The Unison HTTP library

This project contains the following:

Basic types for working with the HTTP protocol. Notably this includes definitions of an HttpRequest and HttpResponse.
An HTTP client. See client.README.
An HTTP server. See server.README.
WebSocket support. See websockets.README. WebSocket functionality is also baked into the client and server.

Attribution and licensing

See LICENSE for license information.

The WebSocket support in this library was originally authored by @alvaroc1 in @alvaroc1/websocket and has been merged into this library with Alvaro's permission.

Basic types

HttpRequest

The HttpRequest type represents an HTTP request as defined by RFC 2616.

It could be used to represent either a request received by a server, or a request to be sent by a client.

Constructing an `HttpRequest`

There are a number of helper methods for constructing an HttpRequest:

HttpRequest.get
HttpRequest.post
HttpRequest.put
HttpRequest.delete

uri = parseOrBug "https://post.it/here"
body = "{\"Hello\": \"World\"}" |> Body.fromText
HttpRequest.post uri body

Or you could construct a HttpRequest directly using its constructor:

uri = parseOrBug "http://some.where"
HttpRequest POST Version.http11 uri Headers.empty Body.empty

A request can also be decoded from Bytes using the HttpRequest.fromBytes and HttpRequest.fromStream functions:

HttpRequest.fromBytes : Bytes ->{Exception} HttpRequest

HttpRequest.fromStream : '{g, Stream Bytes} a
                         ->{g, Exception} HttpRequest

Headers

You can add a header to an HttpRequest using the HttpRequest.addHeader function:

HttpRequest.addHeader : Text -> Text -> HttpRequest -> HttpRequest

To set a header to a specific set of values, use setHeader (a header with multiple values will be sent as multiple headers):

setHeader : Text -> [Text] -> HttpRequest -> HttpRequest

Given an HttpRequest req, you can get the headers using the HttpRequest.headers field:

HttpRequest.headers : HttpRequest -> Headers

You can modify the headers of an HttpRequest using the HttpRequest.headers.modify function:

HttpRequest.headers.modify : (Headers ->{g} Headers)
                             -> HttpRequest
                             ->{g} HttpRequest

Body

The body of an HttpRequest is represented by a Body value. You can get the body of an HttpRequest using the HttpRequest.body field:

HttpRequest.body : HttpRequest -> Body

You can set the body of an HttpRequest using the HttpRequest.body.set function:

HttpRequest.body.set : Body -> HttpRequest -> HttpRequest

Other fields

The other fields of an HttpRequest are:

HttpRequest.method - the HTTP method of the request
HttpRequest.version - the HTTP version of the request
HttpRequest.uri - the URI of the request

HttpResponse

The HttpResponse type represents an HTTP response as defined by RFC 7231.

It could be used to represent either a response received by a client, or a response to be sent by a server.

Constructing an `HttpResponse`

There are a number of helper methods for constructing common HttpResponses:

ok
HttpResponse.notFound
noContent
badRequest

Or you could construct a HttpResponse directly using its constructor:

HttpResponse
  (Status 200 "OK") Version.http11 Headers.empty (Body Bytes.empty)⧨HttpResponse
  (Status 200 "OK")
  (Version 1 1)
  (Headers (Map.fromList []))
  (Body 0xs)

HTTP client

This library can be used to make HTTP requests and inspect their responses.

Basic Usage

Here is a basic example of fetching the unison-lang.org home page:

client.examples.simple : '{IO, Exception} HttpResponse

client.examples.simple : '{IO, Exception} HttpResponse
client.examples.simple =
  do
    Random.run do
      Threads.run do
        Http.run do
          Http.get (parseOrBug "https://www.unison-lang.org")

Below is an example of making a simple HTTP request and getting back a response. It uses the & helper for creating a RawQuery (which will be converted to a URI query string).

examples.query : '{IO, Exception} HttpResponse

examples.query : '{IO, Exception} HttpResponse
examples.query =
  do
    Random.run do
      Threads.run do
        use Path /
        google = Authority None (HostName "www.google.com") None
        path = root / "search"
        query = Query.empty & ("q", "Unison Programming Language")
        uri =
          URI
            Scheme.https
            (Some google)
            path
            (fromQuery query)
            Fragment.empty
        Http.run do Http.get uri

Pooled Usage

If you are making many requests to the same host, you can greatly benefit from using a connection pool that can re-use connections to the same host. To make use of connection pooling, you have to start a Client, which pass to pooledHandler. You can create as many handlers as you want, and they will share the same connection pool.

pooled : '{IO, Exception} Nat

pooled : '{IO, Exception} Nat
pooled =
  do
    use Http get
    _ =
      do
        {{
        we create an interrupt that we can use to shut down the
        client's background threads.
        }}
    Random.run do
      Threads.run do
        _ =
          do
            {{
            create a client that we can re-use for multiple handlers.
            }}
        client = Client.start Client.Config.default
        _resp1 = 
          runPooled client do get (parseOrBug "https://example.com")
        (_, _) =
          runPooled client do
            ( get (parseOrBug "https://example.com")
            , get (parseOrBug "https://example.com")
            )
        _ =
          do
            {{
            If we check the connection pool's stats, we should only
            see one connection created.
            }}
        getStats client |> totalCreated

Response Status

By default, Http.run does not return a Failure for a non-success HTTP status code (such as 500 Internal Server Error). It is left up to the user to determine whether they want to treat a 404 as an error or as an expected case which they should handle accordingly (for example by returning None). You can use HttpResponse.isSuccess to check whether a response has a success code. In the future we may want to provide some helper methods for common use-cases of status code handling.

Response Body

The response body is treated as raw bytes.

type Body

type Body = Body Bytes

HttpResponse.body : HttpResponse -> Body

This library handles decoding chunked and compressed responses but it is up to the user to further interpret those bytes. For example you may want to use fromUtf8 if you are expecting a text response, and/or you may want to use a JSON library to parse the response as JSON. In the future we may add more helper methods for common use-cases.

URI Encoding

You should not attempt to URI-encode the segments in the Path or the keys/values in the RawQuery. This library will automatically encode these values when serializing the HTTP request.

Trailing Slash

According to the HTTP specification, http://www.unison-lang.org/docs/quickstart and http://www.unison-lang.org/docs/quickstart/ (with a trailing slash) are two different URIs. The URI without the trailing slash has two path segments: docs and quickstart. The URI with the trailing slash technically has a third path segment that is an empty string. Therefore if you need to create a path with a trailing slash you can add an empty segment to the end:

trailingSlash : Path

trailingSlash : Path
trailingSlash =
  use Path /
  root / "docs" / "language-reference" / ""

unsafeRun! do fromUtf8 (Path.encode trailingSlash)⧨"/docs/language-reference/"

Inspiration

This library was heavily inspired by the excellent http4s Scala library.

HTTP server

This library allows you to run an HTTP server.

Quickstart

example.main : '{IO, Exception} ()

example.main : '{IO, Exception} ()
example.main = do
  use Nat *
  config = server.Config.Config None (Port "8081") None
  routes = Routes.default <<< alohaHandler <<< helloHandler
  Random.run do
    Threads.run do
      Config.serve routes config
      printLine "started server on port 8081"
      sleepMicroseconds (24 * 60 * 60 * 1000000)

a Handler represents a function which handles one "Route" of the server. Handlers come in two types, Handler which handles HTTP 1.1 requests and WebSocketHandler which handles WebSocket requests.

Http Handlers

Http Handlers are functions from a HttpRequest to a HttpResponse.

A Handler is a Handler which uses g Effects in order to produce a HttpResponse If a Handler uses the Abort ability, it will indicate that this handler is not interested in the given HttpRequest and this request should be sent to the next handler. If a Handler uses the Exception ability, it indicates a Server error which will be sent to the user using the 500 handler.

Creating a Handler

A handler is just a function from a HttpRequest to a HttpResponse, so one could pattern match on the HttpRequest and return a HttpResponse:

helloHandler2 : Handler IO

helloHandler2 : Handler IO
helloHandler2 = 
  Handler cases
    HttpRequest GET _ (URI _ _ (Path ["hello"]) _ _) _ _ ->
      ok (Body (Text.toUtf8 "Hello World"))
    _ -> abort

However, this is not very convenient, so we provide a Handler type which is a wrapper around a function from HttpRequest to HttpResponse. This type provides a number of helper functions to make it easier to create handlers, for example, the above pattern match can be rewritten using the Routes.get function as a pattern guard:

helloHandler : Handler IO

helloHandler : Handler IO
helloHandler = 
  Handler cases
    req| Routes.get (root Path./ "hello") req  ->
      ok (Body (Text.toUtf8 "Hello World"))
    _ -> abort

This example shows how can provide different responses based on Headers:

alohaHandler : Handler IO

alohaHandler : Handler IO
alohaHandler = 
  Handler cases
    req
      | Boolean.and
        (Routes.get (root Path./ "aloha") req)
        (withHeader "Accept" "application/json" req)  ->
        ok (Body (Text.toUtf8 "{\"aloha\": \"World\"}"))
      | Routes.get (root Path./ "aloha") req                                                             ->
        ok (Body (Text.toUtf8 "Aloha, world"))
    _ -> abort

WebSocket Handlers

WebSocket Handlers are functions that take a Http Request and return a WebSocketHandler.

A WebSocketHandler is a pair of functions to handle a WebSocket connection.

The first function is called when the WebSocket connection is closed.

The second function is called after a WebSocket connection has been successfully negotiated. It takes, as input, a WebSocket. It can use WebSocket.send, WebSocket.receive, WebSocket.close to interact with the websocket.

Note: When the handler function returns, the underlying Connection.Deprecated will be closed.

There are are a couple of helpers for creating a WebSocketHandler; sync, async

sync :

Creates a WebSocketHandler that reads and writes messages synchronously. This is useful when you want to receive messages synchronously and block sending when you are waiting for a message.

Example:

Receives 10 messages
Sends a single response acknowledging the 10 messages
Sends a response to each of the 10 messages

sync.doc.example1 : '{IO, Exception} ()

sync.doc.example1 : '{IO, Exception} ()
sync.doc.example1 =
  do
    use Path /
    use Text ++
    config = server.Config.Config None (Port "8081") None
    socketHandler = 
      sync do
        messages = fill' 10 do ask
        emit
          (TextMessage
            ("You sent me 10 messages: " ++ toDebugText messages))
        flipped.deprecated messages cases
          TextMessage msg -> emit (TextMessage ("You said: " ++ msg))
          _               -> ()
    socketRouteHandler : Handler IO
    socketRouteHandler = HandlerWebSocket cases
      req | Routes.get (root / "socket") req -> socketHandler
      _   -> abort
    routes : Routes IO
    routes = Routes.default <<< socketRouteHandler
    Random.run do
      Threads.run do
        Config.serve routes config
        printLine
          "started server on port 8081. Press <enter> to stop."
        ignore readLine()

async :

Creates a WebSocketHandler that forks a thread that continously reads incoming web socket messages calls the provided onMessage on them. This is useful when you want to receive messages asynchronously and not block sending when you are waiting for a message.

Example:

Receives messages and responds to them, all on a forked thread
Send a message every 4 seconds

async.doc.example1 : '{IO, Exception} ()

async.doc.example1 : '{IO, Exception} ()
async.doc.example1 = do
  use Path /
  use Text ++
  config = server.Config.Config None (Port "8081") None
  socketHandler = async (cases
    TextMessage msg -> emit (TextMessage ("You said: " ++ msg))
    _               -> ()) do
    abilities.repeat 100 do
      sleepMicroseconds 4000000
      emit (TextMessage "Regularly scheduled message for you")
  socketRouteHandler : Handler IO
  socketRouteHandler = HandlerWebSocket cases
    req | Routes.get (root / "socket") req -> socketHandler
    _   -> abort
  routes : Routes IO
  routes = Routes.default <<< socketRouteHandler
  Random.run do
    Threads.run do
      Config.serve routes config
      printLine "started server on port 8081. Press <enter> to stop."
      ignore readLine()

Routes is a mechanism for finding a Handler for a given HttpRequest.

Routes are made up of a list of Handlers, a "notFound" function and a "error" function

When a HttpRequest is received list of handlers will be tried one by one to handle a given HttpRequest. The first HttpResponse produced by a Handler will be returned to the client. If one of the handlers rasises an exception, it will be passed to the "error" function to produce an HttpResponse. If none of the handlers are able to handle the HttpRequest, the "notFound" function will be used to produce an HttpResponse.

Creating Routes

Routes.default returns a Routes with a default "notFound" and "error" functions, and no other Handlers, so every request would get a 404 response. <<< is a function which takes a Handler and a Routes and returns a new Routes with the Handler added to the list of handlers.

so default <<< helloHandler creates a Routes that will only responsd to GET reqeusts to "/hello"

WebSockets

Client and server-side WebSockets are supported.

On the client side, you will typically use Http.webSocket and HttpWebSocket.handler to create a WebSocket connection.

On the server side you will typically create a WebSocketHandler to handle a WebSocket connection.

However, some lower-level WebSocket functionality is provided for advanced use cases. Here is an example that uses some of the lower-level functionality:

websockets.example : '{IO, Exception} ()

websockets.example : '{IO, Exception} ()
websockets.example =
  do
    use Message text
    use Nat *
    use WebSocket send
    handleConnection connection = 
      withConnection connection do
        request = HttpRequest.decode()
        emit (HttpResponse.encode (upgradeResponse request))
        ws =
          threadSafeWebSocket
            false connection Server (1024 * 1024) Bytes.empty
        message = WebSocket.receive ws
        Debug.trace "Received" message
        send ws (text "From SERVER")
        send ws (text "From SERVER 2")
    Random.run do
      Threads.run do
        boundSocket = Socket.server None (Port "9011")
        finalizer
          (_ -> let
            (BoundServerSocket socket) = boundSocket
            Socket.close socket)
        listeningSocket = boundSocket |> Socket.listen
        connection =
          Socket.accept listeningSocket
            |> Connection.socket
            |> interruptibleConnection
        finalizer (_ -> Connection.close connection)
        handleConnection connection