How to read and process yaml files in POSIX using yq and jq

Prerequisites

yq and jq tools installed

Lets assume we have following yaml file called snapshots.yaml where the fields reference and version should be read for each item in the snapshots key in POSIX script

documentation:
  snapshots:
    - reference: master
      version: latest
    - reference: '2024.1'
      version: '1.2.1'
    - reference: '2023.12'
      version: '1.0.1'

In order to read them in POSIX script lets use the tool yq and jq, it provide support to process yaml and json contents respectively.

Use following command to read documentation.snapshots yaml content as json

yq eval -o=j -I=0 '.documentation.snapshots[]' snapshots.yaml

this will produce the following result, where each line will be a json object

{"reference":"master","version":"latest"}
{"reference":"2024.1","version":"1.2.1"}
{"reference":"2023.12","version":"1.0.1"}

these result can be iterated in POSIX script and each json line can be processed using the jq tool to read the fields reference and version like this.

echo '{"reference":"master","version":"latest"}' | jq -r .reference
echo '{"reference":"master","version":"latest"}' | jq -r .version

With all that the complete POSIX script would look like this.

#!/usr/bin/env sh
set -eu

for snapshot in $(yq eval -o=j -I=0 '.documentation.snapshots[]' snapshots.yaml); do
  reference=$(echo "$snapshot" | jq -r .reference)
  version=$(echo "$snapshot" | jq -r .version)

  echo "Reference: $reference, Version: $version"
done

Result

Reference: master, Version: latest
Reference: 2024.1, Version: 1.2.1
Reference: 2023.12, Version: 1.0.1

Create Azure AKS cluster with static egress IP using Terraform

Lets see how we can use a static public IP as a egress IP for the AKS cluster

Prerequisite

• Azure Account
• Configured Azure CLI for the Azure Subscription
• Terraform AKS cluster project 

Create IP resource

We first need to create a Public IP Address for us to use it in our AKS cluster.

Login to Azure portal  > Create a Resource Group

Choose your subscription and provide a name for the resource group E.g. Name StaticIpExample

Under the newly created StaticIpExample resource group, create a Public IP address resource.

Provide a name. E.g. Name: MyStaticIp, and make sure subscription and resource group are selected to the correct value. Once verified create the resource.

This will create a public IP for you.

Load the Public IP into Terraform

Open aks-cluster.tf file from the Terraform project and add following new block into the file.

data "azurerm_public_ip" "egress" {
  name                = "MyStaticIp"
  resource_group_name = "StaticIpExample"
}

once added, run terraform plan and verify the changes. This change will load the azurerm_public_ip resource named MyStaticIp from StaticIpExample resource group.

Use the static IP in AKS cluster

Now the above loaded static IP should be used in the AKS cluster declaration to use it as egress IP.

Open aks-cluster.tf file and add following block into the resource "azurerm_kubernetes_cluster" "default" {  block.

network_profile {
  network_plugin = "kubenet"
  load_balancer_profile {
    outbound_ip_address_ids = [data.azurerm_public_ip.egress.id]
  }
}

The block sets the kubenet as the default network plugin and sets the already defined static public IP as the outbound IP address for the load balancer.

Provide Access to AKS Cluster

If the Public IP address resource and AKS cluster are in two different resource groups then the AKS cluster needs to be provided with the access to use the IP address in the different resource group.

Create User Assigned Managed Identity resource in the resource group where the Public IP address resource present, in our case StaticIpExample, and provide a name E.g. Name: IpIdentity.

Open Public IP address resource MyStaticIp, navigate to Access Control (IAM) > Add role assignment > choose Contributor role under privileged administrator roles (or choose a role best match for you) > Next > Choose Managed Identity > Select newly created Managed Identity > Next > Review + assign

This will provide access to Managed Identity IpIdentity to handle Public IP address resource MyStaticIp.

Load the created managed identity into Terraform.

Open the aks-cluster.tf file and add following block

data "azurerm_user_assigned_identity" "ip_identity" {
  name                = "IpIdentity"
  resource_group_name = "StaticIpExample"
}

above declared managed identity data should be used in AKS cluster to be able to access the Public IP address resource present in the different resource group.

Open aks-cluster.tf file and add following block into the resource "azurerm_kubernetes_cluster" "default" {  block.

identity {
  type = "UserAssigned"
  identity_ids = [
    data.azurerm_user_assigned_identity.ip_identity.id
  ]
}

that's all and terraform plan then terraform apply should now create an AKS cluster which uses our defined static Ip address as its egress IP. 

This will make sure the cluster egress IP doesn't change when cluster deleted and re-created again.

Why Expo Go app fails to connect with the Expo app server running on Windows?

Expo is a framework build around react-native framework to provide better developer experience and tools when building a cross-platform react-native application. The framework comes with the handy app called Expo Go, which can be used to test your application on your android or iphone device without the need to build any native packages out of your application.

Workflow to open the app on the physical device is simple, as you just have to follow these steps

1. Install the Expo Go app on your android or iOS device
2. Run the development server of your app using the command npx expo start
3. Open the development server url from the Expo Go app (by scanning the QR code or manually)

However in Windows, even though the development server is running, Expo Go app might failed to load the app with following error

Uncaught Error: java.net.SocketTimeoutException: failed to connect to after 10000ms

It could be due to one of the following reasons.

Device and windows are in the different network

Connect both windows and the device which has Expo Go app to the same network. If both are in different networks then Expo Go app will not be able to find the development server running on Windows.

E.g. both the device and Windows will be in the same network if it is connected to the same router.

Inbound connection for the development server port is not allowed in Firewall

Open the Windows firewall as an administrator > select Inbound rules > select New rule > Choose port > Next > Select Specific local port and pass the development server port as value > Allow the connection > Choose all domain > Provide a name > and finish.

Port number of the development server will be printed along with the npx expo start command output.

Inbound connection to the node.exe application is not allowed in Firewall

If adding an inbound rule for development port still does not work, then it could be because the access for node.exe is blocked from firewall inbound rule.
Open the Windows firewall > select Inbound rules.

Search for any rule that blocks the access for node.exe and delete that rule, with that Expo Go app should be able to load the app from the development server.

Create Android Emulator Using Command Line Tools

Recently I had the need to create an android emulator to test my tiny new app, it is no secret that Android Studio is the go to IDE in this case, because it has the SDK Manager which can be used to create an Android emulator easily.

But what if we don’t want to install Android Studio just for the purpose of creating the emulator, Can we do that without using the Android Studio?🤔 The answer is yes, there exists a command line sdkmanager for this purpose, so lets use it to create an emulator.

Go ahead and download it for your platform, In my case I downloaded it for Windows and the steps explained here are tried on Windows in PowerShell.

Install sdkmanager CLI

Follow the steps explained in sdkmanager-tools to install sdkmaager CLI. If followed properly you should have a newly created folder with the subfolder cmdline-tools in it. The folder which contains the cmdline-tools subfolder is the Android Home or Android SDK Root folder, therefore its a good idea to setup the environment variables ANDROID_HOME and ANDROID_SDK_ROOT for this folder location.

You can setup these environment variables in Windows by using Environment Variables view. To open up the view search for Edit environment variables for your account in the search bar.

With that now you should be able to list the packages from sdkmanager by executing following command.

cd $Env:ANDROID_HOME ; .\cmdline-tools\latest\bin\sdkmanager --list

Install platform-tools

The command line tool Android Debug Bridge (adb), which comes with platform-tools, is used to communicate with device to install/debug app therefore it should also be installed.

Install the platform-tools using the sdkmanager CLI by executing following command.

cd $Env:ANDROID_HOME ; .\cmdline-tools\latest\bin\sdkmanager platform-tools

Install system-image and platform

Emulator requires system image and platform. Use the latest or preferred version of these packages from sdkmanager.

I used the version android 14/API 34 to install these packages, you can list all possible packages by invoking sdkmanager --list.

cd $Env:ANDROID_HOME ; .\cmdline-tools\latest\bin\sdkmanager "system-images;android-34;default;x86_64" "platforms;android-34"

Create emulator

Now the installed system image and platform can be used to create an emulator. When creating an emulator it is preferred to create it using a specific device skin, it brings the width/height and resolution similar to that targeted device.

All the possible devices can be listed using the command. 

cd $Env:ANDROID_HOME ; .\cmdline-tools\latest\bin\avdmanager list device

Create an emulator called my-pixel using the above installed system-image, platform and pixel-7 device skin.

cd $Env:ANDROID_HOME ; .\cmdline-tools\latest\bin\avdmanager create avd --name "my-pixel" --package "system-images;android-34;default;x86_64" --device "pixel_7"

Start the emulator

Install the emulator package, which contains emulator.exe, if it is not installed already(if you don't see emulator subfolder in ANDROID_HOME folder) then use following command to install it.

cd $Env:ANDROID_HOME ; .\cmdline-tools\latest\bin\sdkmanager "emulator"

Once installed start the above created emulator named my-pixel using the following command.

cd $Env:ANDROID_HOME ; .\emulator\emulator -avd my-pixel

Hooray now we have created an Android emulator using only the command line tools 🙌