Its the little things (like opening a web browser)

Sometimes as your developing a solution/script for a problem your faced with interesting challenges where the dumbest workaround (opening a web browser and authenticating with a service) fixes your issue.

Making this simple for a user; who is transitioning mediums (from a cli to a web browser) - this can be interesting technical challenge.

My solution (in python, because that's what I am writing my script in):

    args = parser.parse_args()
    try:
        ### Authenticate with some service, Example below
        ### service = SERVICE(args.server, auth=(username, password)) 
    except SERVICE_Error as e:
        if "ERROR_KEY" in e.text:  # Your key identifier where you know you have 'this one particular issue' 
            print("ERROR: log in to {} using 'incognito' mode in your browser to reset the CAPCHA failed login counter".format(args.server))
            import webbrowser; wb = webbrowser.get()
            if 'chrome' in wb.name:
                wb.remote_args.insert(1, "--incognito")
            elif 'firefox' in wb.name:
                wb.remote_args.insert(1, "--private-window")
            print("INFO: Opening default web-browser in --incognito/--private-window for you! Login to reset the failed log-in counter.")
            print("INFO: After logging in on the web-browser rerun the cli script and authenticate correctly")
            wb.open_new(args.server)

Creating a KVM Lab

In the course of my day job, I often have to create VM's to reproduce issues. With my local system this is quite simple, however how a hypervisor is setup to quickly deploy VMS' with commands, like the ones below, require 3 simple primitives to get the VM's to behave the way you expect.

In short you need a Network (I use a NAT network, because I like the isolation), a Storage Pool (I use an LVM backed pool for performance), and DNS (so you don't have to be rain man to access your systems).

Before I get to the Hypervisor setup, you might ask how I create these VM's:

Creating a VM using a qcow cloud image, and cloud-init

sudo virt-install --name rhel7.5 --vcpus 2 --ram 4096 --disk Downloads/rhel-server-7.5-x86_64.qcow --disk cidata.iso,device=cdrom --disk pool=default,size=15,bus=virtio --network network=default --import --noautoconsole

Creating a VM using and ISO and a KickStart File

sudo virt-install -n rhel7.5 --vcpus 2 -r 4096 --os-type=linux --disk pool=default,size=10,bus=scsi --disk pool=default,size=15,bus=virtio, --network network=default--location=Downloads/rhel-server-7.5-x86_64-dvd.iso --initrd-inject=kickstart.ks -x "ks=file:/kickstart.ks" --noautoconsole

As you can see, with this setup I start from a common install source (base) set of images, that you can pull from a provider (RHEL, CentOS, or Fedora, etc). The second part  that you will find interesting about these commands is the network that I bind the VM's to, and the disks that I create for the VM's to use.

In both examples, and extra 15GB disk is provided to each VM (usually allocated to /dev/vdb, because these are virtio disks). This disk, is created in my default storage pool.

To see what pools you have and/or what resources they provide, you can use the following:

[sudo] virsh pool-list
[sudo] virsh pool-info default

If you need to create a storage pool (an LVM pool, using a blank block device) you can run the following:

[sudo] virsh pool-define-as --name virt-pool --type logical --source-format lvm2 --target /dev/DEVICE

Should you need to, clean up from creating such a pool, the following commands completely clean up from this:

[sudo] virsh pool-destroy virt-pool
[sudo] virsh pool-undefine virt-pool
[sudo] virsh pool-delete virt-pool[sudo] vgremove virt-pool

The network that the VM's are attached too is similarly defined:

[sudo] virsh net-create network_config.xml

File Example:

  minilab
 
 
 
 
 
 
   
     
   
 

As you can see from this, the VMs I put in this network dhcp boot, using an IP from the defined range(.1 to .253).

Should you need to, clean up from creating such a network, the following commands completely clean up from this:

[sudo] virsh net-destroy minilab
[sudo] virsh net-undefine minilab
[sudo] virsh net-delete minilab

However, even with these tools (the network, and the storage pool) yet another component is needed, to make your lab work as it would in say AWS, GCP, or Azure. This missing component is DNS.

To setup DNS on the hypervisor, the simplest thing to do is have NetworkManager use dnsmasq as a caching server, and point at the Network above's dnsmasq instance to get DNS names for the VM's on the network.

You can enable dnsmask with NetworkManager in 2 ways:

[main]
dns=dnsmasq

You can place this in either /etc/NetworkManager/conf.d/localdns.conf or /etc/NetworkManager/NetworkManager.conf and simply restart NetworkManager, to being using this.

However .... you still need to define what DNS servers you want to reference.  This is done by placing a forwarder configuration (as stated above) that points to the dnsmasq instances (per network) started by libvirt.

Example Configuration:

server=/minilab.lan/192.168.100.254

• Note: This configuration is finky (as in make sure you read the man page, and don't mess it up as debugging can be extremely challenging and painful)

This is placed in /etc/NetworkManager/dnsmasq.d/libvirt_dnsmasq.conf or a similarly named file in the .d directory.

While this process gives you fine grain control over your system and the VM's that you provision, its not overly FAST for creating large lab deployments (more than 2-3 vms), or helpful if you plan to create / delete your labs (as I do).

As a result automation is almost always needed. To simplify the pocess above, https://github.com/sferich888/fedora-minilab/ helps, by allowing me to define an inventory file (lab) that can be re-used or modified to fit my needs.

CA Certificates and python Requests with Fedora

So you want to make an API call with python you say....
You say the API is secured with https (or an TLS/SSL certificate)....

Should be simple right?

No not so much. The short story to this is that Fedora by default wants to use the system certificates stored in /etc/pki/tls/certs/ca-bundle.crt (you can see how this works, by looking at the /usr/lib/python2.7/site-packages/requests/certs.py file.

However if you have the certifi python package being installed (either with pip, or via rpm).

• (pip) certifi (2018.1.18) - Python package for providing Mozilla's CA Bundle. 
• (RPM) python2-certifi.noarch 

You CA certificate changes! Changes to /usr/lib/python2.7/site-packages/certifi/cacert.pem

As you can see from: /usr/lib/python2.7/site-packages/requests/certs.py or:$ python -c "from requests import certs; help(certs)"

We on fedora/rhel/centos default to /etc/pki/tls/certs/ca-bundle.crt for our certificates. This also assumes that using this certificate as a CA works.

You can use the following to test (provided you have a cluster_name):

$ curl https://api.example.com/context --cacert /etc/pki/tls/certs/ca-bundle.crt

However with the certifi package installed, it no longer does, so if you go looking for a solution on the inter-webs you might get fun solutions like:

• r = requests.get(url, verify=cafile)
• r = requests.get(url, verifiy=False)
• export REQUESTS_CA_BUNDLE=cafile

All of these can be found as part of https://stackoverflow.com/questions/10667960/python-requests-throwing-sslerror

However as a fedora user, I feel the best options are to remove certifi so that it no longer conflicts with the default OS CA path. (should correct the SSL issues):

sudo pip uninstall certifi

OR

sudo [dnf|yum] uninstall python2-certifi

So verify where your request CA is with:

$ python -c "from requests import certs; print certs.where()"

And if its not pointing where it should, it might be that OS is allowing the requests package to use an optional add on to set the path for you, based on what that package provides for the CA.

tl;dr don't rely on an optional dependency, use your hosts CA so that you have consistency between your tools.

Fixing Powerline on Fedora (IE: Configuring it Properly)

So I recently got a new work system, and had to re-build my system. As I use powerline to help me visually see whats going on in my bash shell, https://fedoramagazine.org/add-power-terminal-powerline/ proved to be VERY helpful.

However one of the things you will not see if you follow this are, git or additional prompts (context queues). To get those you need to set/ configure the proper theme (default_leftonly).

If you read the docs: https://powerline.readthedocs.io/en/latest/configuration.html
You like me will likely be wondering, how do I do what?

So the long and short of how to configure the proper theme, is by running:

# mkdir -p ~/.config/powerline
# cat <<-'EOF' > ~/.config/powerline/config.json
{
     "ext": {
         "shell": {
             "theme": "default_leftonly"
         }
     }
}
EOF
# exec bash

More on Kubernetes/OpenShift JSON Path

In a past post: http://sferich888.blogspot.com/2017/01/learning-using-jsonpath-with-openshift.html

I discussed what json path could do. Today I learned that filters (are only capable of doing simple comparisons (==, !=, <=, >=) type operations.

The tl;dr version is (see regex):

https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/client-go/util/jsonpath/parser.go#L338

 However because Kubernetes (and by proxy OpenShift) use the jsonpath extension from exponent-io and the golang re2 (regex engine).

The above "filter" operation to find and compare parts of the path are only (as of right now) capable of simple operations.

Fedora and Native Script

I have decided to look into mobile application development. Due to my hardware limitations (I don't own a Mac), I am limited to Android Application development.

However as Java is not my favorite language and I want the ability to possibly port my application over to IOS, I have decided to look into writing an application using NativeScript. This will allow me to learn javascript (better) as well as write an application that has the possibility of being portable to Android and IOS.

To get started, I needed to download and install all of the required dependencies. The NativeScript website has a guide however no instructions for Fedora.

Because I have a newly installed Fedora 25 system, I decided to see if Fedora's DevAssistant could help me get the required dependencies.

• Note: The Fedora Magazine has a good guide for installing Android Developer Studio. 
• Installing Android Developer Sudio is a LONG process, as you can dowload up to 50GB+ of material.
• Your going to need Disk Space for the SDK and the Emulators. 

To get all the needed components, I did the following:

sudo dnf install gcc-c++.x86_64

sudo dnf install devassistant devassistant-ui
da pkg install android-studio
da crt android-studio --name Test

### Setup bashrc
#### edit: ./.bashrc and add the following.

export JAVA_HOME=$(alternatives --list | grep java_sdk_openjdk | awk '{print $3}')

export ANDROID_HOME=${HOME}/android-studio/android-sdk-linux
export PATH=$PATH:${ANDROID_HOME}/tools/
export PATH=$PATH:${ANDROID_HOME}/platform-tools/
### End of file. 

rm -rf Test    ## The project crated by DevAssistant is not needed.

sudo $ANDROID_HOME/tools/android update sdk --filter tools,platform-tools,android-23,build-tools-23.0.3,extra-android-m2repository,extra-google-m2repository,extra-android-support --all --no-ui

sudo npm install -g nativescript --unsafe-perm

tns doctor    ### Make sure it does not report any errors

With this in place I am ready to start the development of my project!

Using Docker to reposync RHEL content to a systems local storage.

I recently got the opportunity to go on site with a customer, and as such would be on a plane for some period of time (5+hrs). Because of this long section of down time, I wanted to use the time to do some work. However the work I want to focus on requires that I have resources. I plan to work on scripting or automating the deployments of OpenShift with KVM,  but due to my flight (travel accommodations) I will not be connected, or don't want to rely on slow / unreliable internet speeds for package downloads.

Because Satellite (a repository manager) is bulking and my laptops resources are limited.  I decided that hosting the content with a web server an syncing it would be the fastest and most light way to accomplish my goals.

However, as I run Fedora, getting the repositories (packages) was going to require some ingenuity, because reposync only works if you can attach to the repositories, which requires a RHEL server (with a subscription), this means installing a VM, which takes time.

Luckily we have containers! Which means that instead of installing RHEL, in a VM, and mounting a filesystem into the VM. Or turning this VM into the hosting server for the content. I can simply mount my storage (in my case this was an SC card), as part of a RHEL7 container, and use the container and its tools to do what I need to get the content.

$ sudo docker pull registry.access.redhat.com/rhel7
$ sudo docker run -it --rm -v /var/run/<user>/<drive_name>:/opt rhel7 /bin/bash

From here the steps to get the content may vary, but the basic process goes like:

# subscription-manager register --username <RHN_USERNAME> --auto-attach
[Password]: 

• Note: Extra effort, to enable channels might be needed, depending on how auto-attach does, at selecting your subscriptions.

# rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

# for repo in rhel-7-server-rpms rhel-7-server-extras-rpms rhel-7-server-ose-3.3-rpm; do reposync --gpgcheck -l --repoid=${repo} --download_path=/opt/ --downloadcomps --download-metadata
# for repo in rhel-7-server-rpms rhel-7-server-extras-rpms rhel-7-server-ose-3.3-rpm; do createrepo -v /opt/${repo} -o /opt/${repo} -g /opt/${repo}/comps.xml; done

Note: I based the repositories (and some of the process) I would need off of - https://docs.openshift.com/container-platform/3.3/install_config/install/disconnected_install.html#disconnected-syncing-repos

Because of this method (using Docker), I don't need waste time installing a VM to do these operations, or use unnecessary resources to host this content. This leaves me more room for the VM's I plan to run as part of my scripting / testing, and allowed me to get the content faster than I originally expected.