Connecting to Tapo lamps from F#

23 October 2021

Introduction

I am a C# programmer by day, but F# enthusiast by night. I love how the language allows me to be explicit in important things, but implicit in things that the compiler can figure out on it's own.

Recently, I have bought a pair of Tapo lights. Why did I buy them? The primary reason is that they have an API to play with. Unfortunately, I have not found public documentation for it, but there are good people out there that have built a library for Tapo products in Python. Shout out to @K4CZP3R and their repository tapo-p100-python. Their work has helped me tremendously to understand how I am supposed to talk to my lights.

Well, let me demonstrate the results. All the code is at the KHome.TapoLights github repository. After I built the API, I have decided to test it out on something. For that purpose, I built a simple color picker that reads the pixel color under my mouse pointer and sends the color to my lamps. Here is the final result (powered by F#):

How did I do that?

Introduction
Protocol
Adding a color picker
Final words

Protocol

What is a typical Request-Response for the device?

The API is very RPC-like. We provide the method name and parameters, and the lamp gives us the operation result.

The request looks like this:

json
{
   "method": "method_name",
   "params": {
      ... // Some data
   },
   "requestTimeMils": 1634482990705
}

Response:

json
{
  "error_code": 0, // 0 - means success
  "result": {
    ... // Result Data
  }
}

Basic API layer

Now that we know the format, let's define the request/response types.

f#
type TapoRequest<'P> = { Method: string; Params: 'P; RequestTimeMils: int64 }
type TapoResponse<'R> = { [<JsonPropertyName("error_code")>] ErrorCode: int; Result: 'R }

// A useful helper to create the request type
let makeTapoRequest<'TParams> (method: string) (parameters: 'TParams): TapoRequest<'TParams> =
  { Method = method; Params = parameters; RequestTimeMils = nowMillis() }

In .NET, there is a built-in JSON serialization library called System.Text.Json. With it, we can easily serialize the requests to bytes and deserialize the response from bytes:

f#
let private serializerOptions = JsonSerializerOptions(JsonSerializerDefaults.Web)

let serialize<'TValue> (obj: 'TValue) = JsonSerializer.SerializeToUtf8Bytes (obj, serializerOptions)

let deserialize<'TValue> serialized =
  let mutable jsonUtfReader = Utf8JsonReader(ReadOnlySpan<byte>(serialized));
  JsonSerializer.Deserialize<'TValue> (&jsonUtfReader, serializerOptions)

And finally, we need to make requests to the API. For that, let's use .NET HttpClient APIs.

f#
let postAsync<'TRequest, 'TResult> (httpClient: HttpClient) (request: TapoRequest<'TRequest>) =
  task {
    let serialized = serialize request
    let content = new ByteArrayContent(serialized)
    content.Headers.ContentType <- MediaTypeHeaderValue("application/json")
    
    let! resp = httpClient.PostAsync("", content)
    
    // .NET has a handy method to deserialize JSON right from the response stream called ReadFromJsonAsync
    let! deserialized = resp.Content.ReadFromJsonAsync<TapoResponse<'TResult>>(serializerOptions)
    
    return deserialized
  }

From the TapoResponse<'T> type definition, we see that an API call can be unsuccessful. How do we handle that? F# active patterns are a wonderful tool to encode two distinct states of the response - failed and succeeded. I like the idea of a simple function that gives us the result of the operation or the error code depending on the response state:

f#
let (|SuccessfulTapoResponse|FailedTapoResponse|) (tapoResponse: TapoResponse<'T>) =
  if tapoResponse.ErrorCode = 0
  then SuccessfulTapoResponse tapoResponse.Result
  else FailedTapoResponse tapoResponse.ErrorCode

// Example usage code:
match response with
| SuccessfulTapoResponse result -> // do stuff with result
| FailedTapoResponse errorCode -> // handle error

Handshake

Firstly, the device expects us to perform a handshake with it. What does a handshake entail?

Almost all communication with the lamp is encrypted. The handshake is a way for us to negotiate which key we are going to use for that encryption.

When we do our first request, we have no idea how we are going to encrypt stuff. So the device expects us to provide a RSA key that it will then use to encrypt the response. The response will contain encrypted AES key parts that we will use for ALL of our communication.

Diagram depicting the handshake request — Handshake

Diagram depicting the handshake response — Handshake

Let's do that in F#.

Our types for the handshake call should represent it receiving an RSA key and returning AES key parts.

f#
module Handshake =  
  type Params = { Key: string }
  type Result = { Key: string }
  
  let makeRequest (parameters: Params) =
    API.makeTapoRequest "handshake" parameters

Since Tapo lights accept requests via HTTP, the easiest way to talk to an HTTP API in .NET is to spin up an HttpClient.

f#
let client = new HttpClient()
client.Timeout <- TimeSpan.FromSeconds(5.0)
client.BaseAddress <- Uri($"http://{ip}/app")

Then, we proceed to implement the handshake API call:

f#
let toHandshake publicKey =
  // .NET RSA doesn't wrap the key with the BEGIN/END parts, so we do it ourselves
  let publicKeyStr = Convert.ToBase64String(publicKey)
  let publicKeyWrapped = $"-----BEGIN PUBLIC KEY-----\n{publicKeyStr}\n-----END PUBLIC KEY-----\n"

  Handshake.makeRequest { Key = publicKeyWrapped }

// Prepating RSA key
let key = RSA.Create(1024)
let publicKey = key.ExportSubjectPublicKeyInfo()

let! handshakeResponse =
  toHandshake publicKey // Creating request object
  |> API.postAsync<_, Handshake.Result> client // Making HTTP Request

// Extracting the result from response object
let handshakeResult =
  match handshakeResponse with
  | API.SuccessfulTapoResponse result -> result
  | API.FailedTapoResponse errCode -> failwith $"received errorCode = {errCode} on handshake"

// Decrypting the AES parts
let encryptionParts =
  key.Decrypt(Convert.FromBase64String(handshakeResult.Key), RSAEncryptionPadding.Pkcs1)

Secure passthrough

Earlier I mentioned that all communication with the device will be encrypted. That is implemented by passing all our requests and responses through an AES encryption.

Fellows that designed the device decided to do that by including a special API call - securePassthrough. All the API call does - is wraps the full request JSON into an encrypted base64 string. The response is returned in the same format.

Diagram depicting the securePassthrough request — SecurePassthrough

Diagram depicting the securePassthrough response — SecurePassthrough

To support that, we need to construct actual AES encryptors and decryptors from out key parts:

f#
// I am not an expert in encryption
// So this key and iv extraction is basically me implementing it like in the Python library
let key = [|
  for i in 0..15 do
    yield encryptionParts.[i]
|]

let iv = [|
  for i in 0..15 do
    yield encryptionParts.[i + 16]
|]

// .NET has build-int AES support, and we can leverage that
let aes = Aes.Create()
let encryptor = aes.CreateEncryptor(key, iv)
let decryptor = aes.CreateDecryptor(key, iv)

Now, with these encryptor and decryptor we can finally implement support for securePassthrough API call. We start with types:

f#
module SecurePassthrough =
  type Params = { Request: string }
  type Result = { Response: string }
  
  let makeRequest (parameters: Params) =
      API.makeTapoRequest "securePassthrough" parameters

Then, we make little helpers that help us encode and decode the request and response:

f#
let securePassthroughEncode (encryptor: ICryptoTransform) request  =
  // We get the raw json of the request
  let requestSerializedBytes = 
    API.serialize request
  
  // We encrypt it and turn into base64
  let secured =
    encryptor.TransformFinalBlock(requestSerializedBytes, 0, requestSerializedBytes.Length)
    |> Convert.ToBase64String
  
  SecurePassthrough.makeRequest { Request = secured }

let securePassthroughDecode<'TResult> (decryptor: ICryptoTransform) (response: SecurePassthrough.Result)  =
  // We get the raw secured bytes from base64
  let responseBytes =
    response.Response
    |> Convert.FromBase64String
  
  // Then, we decrypt and deserialize raw json
  let deserialized =
    decryptor.TransformFinalBlock(responseBytes, 0, responseBytes.Length)
    |> API.deserialize<API.TapoResponse<'TResult>>
  
  deserialized

To access the lamps, we need to log in via the TP-Link account. The login API method accepts the username and password and returns a token which can be used to authenticate all future requests.

Diagram depicting the login request — Login

Diagram depicting the login response — Login

f#
module LoginDevice =
  type Params = { Username: string; Password: string  }
  type Result = { Token: string; }
  
  let makeRequest (parameters: Params) =
    API.makeTapoRequest "login_device" parameters

For some reason, the login should be hashed and in a hex string. Well, whatever, let's do that:

f#
let hashedUsernameHexBytes =
  SHA1.HashData(username |> Encoding.UTF8.GetBytes)
  |> Convert.ToHexString
  |> toLower
  |> Encoding.UTF8.GetBytes

Finally, we can do the login request (remember (!) now we secure everything by passing the request through securePassthrough API method):

f#
let shaUsernameBase64 = 
  Convert.ToBase64String(hashedUsernameHexBytes, Base64FormattingOptions.InsertLineBreaks)
let passwordBase64 = 
  Convert.ToBase64String(password |> Encoding.UTF8.GetBytes, Base64FormattingOptions.InsertLineBreaks)

// Make login API call
let! loginResponseSecured =
  LoginDevice.makeRequest { Username = shaUsernameBase64; Password = passwordBase64 }
  |> securePassthroughEncode encryptor
  |> API.postAsync<_, SecurePassthrough.Result> client

// Get the secured response and decrypt it it
let loginResponse =
  match loginResponseSecured with
  | API.SuccessfulTapoResponse result -> securePassthroughDecode<LoginDevice.Result> decryptor result
  | API.FailedTapoResponse errCode -> failwith $"received errorCode = {errCode} on secured login"

// Get the Token from the response
return
  match loginResponse with
  | API.SuccessfulTapoResponse result -> ({
      Client = client
      Token = result.Token
      Encryptor = encryptor
      Decryptor = decryptor
  })
  | API.FailedTapoResponse errorCode -> failwith $"received errorCode = {errorCode} on login"

Hooray! Now that we have the token, can finally do stuff with the device.

Set device info

The main thing that interests me is how to change the state of the lamp. To do that, the device supports set_device_info API call. You can change the brightness, or the color of the lamp.

Set Device Info request type definition:

f#
module SetDeviceInfo =
  type DeviceInfo = {
      Color : {| Hue: int; Saturation: int |} option
      Brightness: int option
  }
  
  type Params = IDictionary<string, obj>
  type Result = unit
  
  let makeRequest (parameters: DeviceInfo) =
      
    // This is me trying to support optional values 
    // and sending the ones that the user wanted to change only
    let parametersDictionary: Params =
      seq {
        ("brightness", parameters.Brightness)
        ("hue", parameters.Color |> Option.map (fun c -> c.Hue))
        ("saturation", parameters.Color |> Option.map (fun c -> c.Saturation))
      }
      |> Seq.map (fun (k,v) -> Option.map (fun vSome -> (k, vSome :> obj)) v)
      |> Seq.choose id
      |> dict
    
    API.makeTapoRequest "set_device_info" parametersDictionary

And the usage code looks like this:

f#
type SetColorOptions = {
    Brightness: int
    Hue: int
    Saturation: int
}
let setColor device setColorOptions =
  task {
    let! securedResponse =
      SetDeviceInfo.makeRequest 
        { 
          Brightness = Some setColorOptions.Brightness; 
          Color = Some {| Hue = setColorOptions.Hue; Saturation = setColorOptions.Saturation |}  
        }
      |> securePassthroughEncode device.Encryptor
      |> API.postAuthenticatedAsync<_, SecurePassthrough.Result> device.Client device.Token
    ()
  }

We expose the setColor function to the module user, so that they will be able to manipulate the lamp in this way.

Adding a color picker

To test my implementation out, I decided to do something fancy. As I described in my introduction section, we will try to match the lamp color with current mouse pointer location. To do that, we will use GetCursorPos and GetPixel Windows API calls. Firstly, I will define a tiny interop layer:

f#
module WindowsInterop

  type HDC = int
  type HWND = int
  type COLORREF = uint

  [<Struct>]
  [<StructLayout(LayoutKind.Sequential)>]
  type POINT = { x: int; y: int }

  // To get the Device Context for the screen
  [<DllImport("User32", ExactSpelling = true, SetLastError = true, ThrowOnUnmappableChar = true)>]
  extern HDC GetDC(HWND hWnd)

  // To cleanup the DC after ourselved
  [<DllImport("User32", ExactSpelling = true, SetLastError = true, ThrowOnUnmappableChar = true)>]
  extern int ReleaseDC(HWND hWnd, HDC hdc)

  // Gets current cursor position on the screen
  [<DllImport("User32", ExactSpelling = true, SetLastError = true, ThrowOnUnmappableChar = true)>]
  extern bool GetCursorPos(POINT& lpPoint)

  // Gets color of the pixel under our cursot  
  [<DllImport("Gdi32", ExactSpelling = true, SetLastError = true, ThrowOnUnmappableChar = true)>]
  extern COLORREF GetPixel(HDC hdc, int x, int y)

Then, I will get the color of the mouse location in an infinite loop:

f#
let dc = WindowsInterop.GetDC 0
while true do
    let mutable pos = Unchecked.defaultof<WindowsInterop.POINT>
    WindowsInterop.GetCursorPos &pos |> ignore
    let color = WindowsInterop.GetPixel(dc, pos.x, pos.y)

Now, the problem is that the lamp accepts HSV (hue-saturation-value, value is also known as brightness) color format, but Windows API gives me an RGB (red-green-blue) color. To overcome that limitation, we need to convert rgb to hsv.

Firstly, let's extract rgb bits from the uint32 value Windows gave us.

f#
while true do
   ...
   let r = (color &&& 255u)
   let g = ((color >>> 8) &&& 255u)
   let b = ((color >>> 16) &&& 255u)

Then, let's implement a toHsv function. I will leave it out of this blogpost, as the implementation is literally me typing in the formula of conversion. If you are interested in the function, check it out at my github repo.

Finally, we get our hsv values and push them to my lamp

f#
while true do
   ...
   let (h, s, v) = toHsv r g b
   
   let setColorOptions: Tapo.SetColorOptions = { Brightness = v; Hue = h; Saturation = s }
            
   let sw = Stopwatch.StartNew();
   let! _ = Task.WhenAll([
       Tapo.setColor authenticatedLeft setColorOptions;
       Tapo.setColor authenticatedRight setColorOptions
     ])

And that's it, we have our color-changing lamp!

Final words

This little project was very fun! Feel free to take any code I wrote here (or in the github repo), but I would be grateful if you included credit of where you took it.

What I learned during this ordeal is that you should always try to extract some time to do something that is fun for you.

That will boost your skills and give you some experience in an area you have not worked in.

Tags: fsharp tapo smart-appliances

Kostiantyn's Blog