Local Development with ngrok

Develop cloud services locally by continuously forwarding real data from a Data Connector using ngrok.

Overview

This guide looks at how ngrok can be used to create a local development environment for Data Connectors applications. In summary, the Data Connector is configured to forward events to an URL generated by ngrok, a free proxy that points all incoming traffic to a specified port on localhost where we can receive the events and quickly develop and debug our application,

Prerequisites

• Service Account Your User or Service Account must have the role of Project Developer or higher.

• Local Development It is assumed that you are familiar with the bare minimums of local development, like using a shell.

ngrok

ngrok is a free proxy service that enables forwarding HTTPS traffic from a publicly accessible URL to a port on localhost.

Installation

Most package managers should contain ngrok. If not, see the official download page.

• Debian: apt install ngrok

• Arch: ngrok AUR

• Mac: brew install --cask ngrok

• Windows: The ngrok client binary can be found here.

Starting the Proxy

In your shell of choice, run the ngrok command with the http argument, followed by the port to which you want to forward incoming traffic.

ngrok http 3000 

Once the proxy has started, all traffic directed to the randomly generated HTTPS URL will be directed to the specified port on your localhost.

ngrok by @inconshreveable                                                            (Ctrl+C to quit)

Session Status                online
Session Expires               1 hour, 59 minutes
Version                       2.3.35
Region                        United States (us)
Web Interface                 http://127.0.0.1:4040
Forwarding                    http://5a880278718b.ngrok.io -> http://localhost:3000
Forwarding                    https://5a880278718b.ngrok.io -> http://localhost:3000

Connections                   ttl     opn     rt1     rt5     p50     p90
                              0       0       0.00    0.00    0.00    0.00

If you're using the free version of ngrok, your session will expire after two hours. To restart, rerun the command. Note that this changes the URL.

Localhost Path

All modifications to the generated URL path is reflected on the localhost. This can be useful to know as some local development frameworks, such as the one used by Azure Functions, utilizes pathing.

• Original https://5a880278718b.ngrok.io -> http://localhost:3000

• Added path https://5a880278718b.ngrok.io/some/path -> http://localhost:3000/some/path

Data Connector

To continuously forward events to localhost, create a new Data Connector or use an existing one, setting the following configurations.

• Endpoint URL The generated HTTPS URL of the form https://5a880278718b.ngrok.io as provided by ngrok. Note that the HTTP URL will not be accepted by the Data Connector.

Local Server App

When forwarding Data Connector events to localhost using ngrok, if no response is given to the request, an 502 Bad Gateway error will the thrown.

The following snippets implement a simple local server app that receives the Data Connector request, prints the content, and responds with a success message.

Python 3.11 - Flask

Install the necessary dependencies.

pip install flask==2.3.2

Copy the following snippet to a local file app.py.

app.py

from flask import Flask, request

app = Flask(__name__)


@app.route('/', methods=['POST'])
def print_request_contents():
    print(f'\nHeaders\n-------\n{request.headers}')
    print(f'Body\n----\n{request.get_json()}')
    return 'Success'

Start the local server by running flask run --port 3000 in your shell.

Node.js 20 - Express

Install the necessary dependencies.

npm install express@4.18.2 body-parser@1.20.2

Copy the following snippet to a local file index.js.

index.js

const express = require("express")     // npm install express@4.18.1
const bodyParser = require("body-parser") // npm install body-parser@1.20.0

const app = express()
app.use(bodyParser.urlencoded({ extended: false }))
app.use(bodyParser.json())

app.post("/", (req, res) => {
    console.dir(req.body, { depth: null })
    res.send("Success")
})

app.listen(3000)

Start the local server by running node index.js in your shell.

Sensor Emulator

Our sensors emit events every periodic heartbeat or when touched. When developing an application locally, being able to emit events at will can be quite useful. Our Sensor Emulator lets you create an emulated device for which events can be sent using a click of a button in DT Studio or the REST API.

You can read about how to create and emit emulated events on our Sensor Emulator page.