Monday, January 30, 2017

More on Kubernetes/OpenShift JSON Path

In a past post: http://sferich888.blogspot.com/2017/01/learning-using-jsonpath-with-openshift.html

I discussed what json path could do. Today I learned that filters (are only capable of doing simple comparisons (==, !=, <=, >=) type operations.

The tl;dr version is (see regex):

https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/client-go/util/jsonpath/parser.go#L338

 However because Kubernetes (and by proxy OpenShift) use the jsonpath extension from exponent-io and the golang re2 (regex engine).

The above "filter" operation to find and compare parts of the path are only (as of right now) capable of simple operations.





Thursday, January 26, 2017

Learning / Using jsonpath with OpenShift and Kuberneties

I work as a support engineer on OpenShift and I spend a lot of time trying to get information out of the cluster and often times have to use that to information as an input of some other task I am doing.

Because OpenShift / Kuberneties (cli) have such a wide range of outputs:
-o, --output='': Output format. One of: json|yaml|wide|name|go-template=...|go-template-file=...|jsonpath=...|jsonpath-file=... See golang template [http://golang.org/pkg/text/template/#pkg-overview] and jsonpath template [http://releases.k8s.io/release-1.3/docs/user-guide/jsonpath.md].
I have a wide range of options to choose from when trying to complete my tasks. However one is far superior than all others (IMO).
jsonpath
  • Note: Many places on the web reference: http://goessner.net/articles/JsonPath/ as the source for where jsonpath started. 
Why is this better than the others?
  1. Well for starters it lets me get just the output I need. 
  2. It lets me format the data as I see fit. 
  3. It simple to understand and use (compared to go-template)
The best way to show you how easy it is to use is to show you some examples.

Lets start with something simple. Like getting the name of pod.
So if you have the following:
# [oc|kubectl] get pods
NAME                       READY     STATUS    RESTARTS   AGE
docker-registry-11-57g56   1/1       Running   0          ---
registry-console-2-49irw   1/1       Running   0          ---
router-1-vinzu             1/1       Running   0          ---
One might just do something like:
# [oc|kubectl] get pods --no-headers | awk '{print $1}'
Simple enough, but if you had to pair this with another command you might start to see issues, with command nesting.

A simpler way (IMO) to get the same information is to do something like:
# [oc|kubectl] get pods -o jsonpath='{.items[*].spec.containers[*].name}{"\n"}'
  • Note: the '{"\n"}' in this command is so that a new line is added at the end of the output, and your terminal does not displaying on the same line as the output.
This allows me just to pull the exact data I want out of the system, and in this case (return it as I want - as a list of name).

If I wanted to do something more complicated, such as get a specific service name and IP from the system, I could easily format the output, and filter (a set of objects) to get me data on a specific object.
# [oc|kubectl] get services -o jsonpath='{.items[?(@.metadata.name=="registry-console")].metadata.name}: {.items[?(@.metadata.name=="registry-console")].spec.clusterIP}{"\n"}'
Example output:
registry-console: 172.30.12.188
As a result of output like this, I can get the name of the service, as well as the IP that corresponds to the service. I also get it formatted, exactly how I want it, without having to do complex command stringing (with bash). 

One of the only downsides to using jsonpath is the docs, around the syntax.

If you read, over https://kubernetes.io/docs/user-guide/jsonpath/ there is enough, to do damage and shoot yourself in the foot (only to find out your just doing it wrong).

So to help simplify that, I'll look at the same examples above, but with the json they provide.

Whenever starting out its best to just run:
# [oc|kubectl] get <object> -o json > file.json
Simply put, it's simpler to read the json, and then traverse through it to define your jsonpath syntax.

Example JSON  
this is a fedora paste bin an may not longer be around, sorry # [oc|kubectl] get pods -o json > example.json is how to create this)

If you look at my example json, you can see in both examples, that because the CLI gives me information on multiple objects, it creates and returns a List (which if you used to python terminology its a dictionary object, that defines a key, whose name is "items" that's value is an array):
{
     "kind": "List",
     "apiVersion": "v1",
     "metadata": {},
     "items": [
         {
     ...
Because of this, you often have to start all jsonpath filters with:
'{.items[...
As your going to be accessing the items array, from this output. However you might not need to do this. Say for example if you got the output of a specific pod.
# [oc|kubectl] get pod registry-console-2-49irw -o jsonpath='{.spec.containers[*].name}'
Because, this command returns a Pod. I don't have to access the member of list (or array).
{
    "kind": "Pod",
    "apiVersion": "v1",
    "metadata": {
...
So the first major tip I have with jsonpath is learning how [*] and ['index'] work. ['index'] is exactly what you would expect if you were working with a python dictionary, and when you try and access as key, to get its value. * is a special character, with [] that lets you traverse all the keys (indexes) in the array.

In 90% of what I do, [*] works, as shown above .items is one of the most important thing I have to traverse. So understanding that I am simply looking at the keys, for each member of the array, when specify the next '.' following [*] makes understanding the syntax much simpler.

The next most complicated concepts to learn with jsonpath are filters, and the 'current object' operator. As shown in the example above, ?() is used to create a filter, or a simple 'if TRUE' type of syntax. In the example I used, I used this with in an array [], to identify a pod by its name. The only way this works, is if I can access the 'current object' as I pass through the array. This is what the @ operator does.

With these concepts, your almost cretin to understand jsonpath, and you'll be able to get almost any data out of OpenShift or Kuberneties.

However, one item I did not show an example of, there is another way to traverse arrays. This method uses the range and end keywords.

If we go back to the services example, you can traverse the service and create output (that look like arrays) with something like:
# [oc|kubectl] get services -o jsonpath='{range .items[*]}[{.metadata.name},{.status.capacity}] {end}{"\n"}'
With this you might find it more natural, when traversing the array. However, due to the syntax length (I often don't use this method).

Sunday, January 8, 2017

Fedora and Native Script

I have decided to look into mobile application development. Due to my hardware limitations (I don't own a Mac), I am limited to Android Application development.

However as Java is not my favorite language and I want the ability to possibly port my application over to IOS, I have decided to look into writing an application using NativeScript. This will allow me to learn javascript (better) as well as write an application that has the possibility of being portable to Android and IOS.

To get started, I needed to download and install all of the required dependencies. The NativeScript website has a guide however no instructions for Fedora.

Because I have a newly installed Fedora 25 system, I decided to see if Fedora's DevAssistant could help me get the required dependencies.
  • Note: The Fedora Magazine has a good guide for installing Android Developer Studio. 
    • Installing Android Developer Sudio is a LONG process, as you can dowload up to 50GB+ of material.
    • Your going to need Disk Space for the SDK and the Emulators. 
To get all the needed components, I did the following:
sudo dnf install gcc-c++.x86_64

sudo dnf install devassistant devassistant-ui
da pkg install android-studio
da crt android-studio --name Test

### Setup bashrc
#### edit: ./.bashrc and add the following.

export JAVA_HOME=$(alternatives --list | grep java_sdk_openjdk | awk '{print $3}')

export ANDROID_HOME=${HOME}/android-studio/android-sdk-linux
export PATH=$PATH:${ANDROID_HOME}/tools/
export PATH=$PATH:${ANDROID_HOME}/platform-tools/
### End of file. 

rm -rf Test    ## The project crated by DevAssistant is not needed.

sudo $ANDROID_HOME/tools/android update sdk --filter tools,platform-tools,android-23,build-tools-23.0.3,extra-android-m2repository,extra-google-m2repository,extra-android-support --all --no-ui

sudo npm install -g nativescript --unsafe-perm

tns doctor    ### Make sure it does not report any errors
With this in place I am ready to start the development of my project!