Virtualization - Hypervisors, Linux Containers, Docker

Virtualization

The topics that I am going to cover now includes

1)Virtual Machines

2)Linux Container

3)Docker

I will clarify what are common among them and how do they differ.

To start with lets discuss about Virtualization.

In traditional architecture we have one Operating Instance running over the native hardware.

But in virtualization we can have several such operating systems running over native hardware.

To achieve this a hypervisor layer is introduced which facilitates interaction of the OS instances with the native hardware. This gives a look and feel as if several machines are running parallely independent of each other. The existence of these machines are virtual and hence we call them virtual machines.

This can come in two falvours:

1)If the hypervisor layer sits on top of another OS we call it "Type 2 " hypervisor

2)If there is no host OS we call it "Type 1" hypervisor.

Linux containers and Namespace

Linux kernel comes with hypervisor facility and we call it KVM (Kernel Virtual Machine).

The essence of virtualization is isolation . We have several isolated virtual spaces where we can run processes independent of each other. To achieve this Linux Kernel comes with another light weight solution and we call them Linux Containers. Unlike virtual machines where we can run any type of Operating system Linux Containers sits on top of the native Linux kernel of your Linux system. Thus each container although provide isolation from each other, they are identical with respect to the Linux kernel they are using. To achieve this Linux kernel provide different namespaces to each container and as a result isolation is possible.

On contrast , VM provides more isolation but at the expense of more resource. Containers are lightweight but they can in some cases be affected by other processes on the host.So decision of choosing one over another depends upon the level of isolation that we want to achieve.

Note that we can use containers inside Virtual Machines.Its like dream inside another dream.

Docker

                                   

Docker is container orchestration. It manages Linux containers. It provides a framework over Linux Containers which increases the ease and comfort to use Linux Containers.Unlike hypervisors where in each VM instance we run Operating Systems, when we use Docker the related binaries required to run the applications are downloaded. Then the applications are managed by the docker engine.

    

Important Docker commands

INSTALLATION AND RUNNING DOCKER DAEMON

In ubuntu machine you can install docker by
sudo apt install docker.io

If the docker service is not enabled you need to enable it,  start it and check the status. 

systemctl start docker

systemctl enable docker

systemctl status docker

GET A DOCKER IMAGE FROM PUBLIC REPOSITORY
Docker hub provides container templates or base images which you can instead of writing your container from scratch.

docker search fedora  #Search the Docker Hub for images

docker pull debian  #Pull docker image from repository.

INFO ABOUT DOCKER

docker info #Displays system wide information about docker. Shows stats like number of containers running, paused,stopped.

docker version # Shows the version of the docker

QUERY IMAGES
docker images #Lists imges in host machine
docker search ubuntu  #Search for image named ubuntu from dokcer repositories

RUN CONTAINER

docker run --name myubuntu -it ubuntu /bin/bash

Now we are in container

This conatiner has minimum installations. You can install important tools like vim,gcc, nettools,ssh.

apt-get update
apt-get -y install net-tools
apt-get -y nstall vim
apt-get -y install gcc
apt-get -y install iputils-ping
apt-get -y install ssh
useradd -m -d /home/dummy_user/ -s /bin/bash -G sudo dummy_user
passwd dummy_user

If you type exit then the container will stop immediately because it will have no commands to run.
To get out of the container without exiting press
Press Ctr+P+Q 

To run the container in background (detached mode) use -d flag.

ATTACH TO CONTAINER RUNNING IN BACKGROUND

docker attach

SSH TO  CONTAINER WITH DIFFERENT USER IDS

docker exec -it --user dummy_user /bin/bash

/bin/bash tells to give the access to bash shell after login.

SSH TO  HOST/ANY MACHINE FROM CONTAINER

We have installed ssh and can login to any machine as usual.

CONTAINER LIFE CYCLES
                        


docker pause
docker unpause
docker start
docker stop
docker kill

CLEANING
To remove a container
docker rm
You can prune all stopped container or downloaded images using the command
system prune -a

STATISTICS OF THE CONATINERS
docker ps -a //Lists containers

docker inspect //Configuration of the container in JSON 
docker top //Displays Linux top for each conatiner

docker logs //Shows the logs of the command running inside the container

CREATING DOCKER IMAGE FROM EXISTING CONTAINER
To create an image out of contianer you must have to stop the container.
docker stop
docker commit

DOCKER FILE

To automate image creation in a batch you have Docker file.
Just like Makefile you can specify rules how to make a image.
Create a file Dockerfile and write the below contents to create our custom image of type just as we have created above.

Dockerfile:

FROM ubuntu # The base image for this image will be ubuntu
#Execute following commands after the image is created.
RUN apt-get update && apt-get -y install net-tools && apt-get -y install vim && apt-get -y install gcc && apt-get -y install iputils-ping && apt-get -y install ssh
RUN useradd -m -d /home/dummy_user/ -s /bin/bash -G sudo dummy_user #RUN -> Executed during building image
CMD [“echo”,”Image created”] #CMD ->Executed during container creation

Then
docker build -t mylinux:latest . // builds image with name mylinux with latest tag

Then start a conatiner of this image just like for any other image.

DOCKER COMPOSE

Tool for defining and running multi-container docker applications

Use yaml files to configure application services (docker-compose.yaml)

Can start/stop all services with a single command.

Can scale up selected services whenever required.

Step1:

Install docker compose.

pip install -U docker-compose

docker-compose -v #Shows you the version

Step2

Create docker compose file at any location on your system.

Standard name of file is docker-compose.yaml

docker-compose.yaml Example:

version: '3'

services:

    web:

        image:ngnix
        ports:
       -"8080:80"

   database:

       image:redis

Define the services and the image from which they will be derived.

Make sure that the version of docker compose specified above is compatible with our docker engine.

Expose port 80 of ngnix to port 8080 of host machine.

Step3

Check validity of file by running 

docker -compose config

Step4

Run docker-compose.yml file by the below command to run it in detached mode.

docker-compose up -d

To stop everything use 

docker-compose down

Step5 (optional to scale up services)

docker-compose up -d --scale database=4

This will create 4 instances of databases.

DOCKER NETWORK

There are 3 basic types of networking available in Docker.

1)Bridge - It is the default private network created by Docker to which gets attached to. 

For example when you type "docker run ubuntu", by default it gets attached to the bridge network.

All docker containers can talk to each other using the private IPs and can communicate with the Internet through the host.

To access the containers from outside world map port of the container to the port of the host.

Use -p :

2)Host -  It takes out isolation between host and the container. You can attach the container to host network like 

docker run ubuntu --network=host

If you run a webserver in container in port 8080 , then this container is available externally with this port. Here you do not need to use the -p option. This means you can not run another web server in the same port.

3)None - This means the conatiner is isolated from other conatiners or any other network.

docker run ubuntu --network=none

Other Commands

docker commit #Create custom image.The conatiner is saved as image and is available locally.

#Create container

docker run -it --name

#Docker bridge exist in Host. Its a sort of hub in host which connects all containers in the host.

#On host type

ip link show #Will list virtual interfaces

brctl show docker 0 #Will show virtual interfaces are connected to the docker bridge

#2 interfaces are created per container. One connected to docker bridge & other conncted to the container

#There is NAT running on docker

ip netns list #Lists network namespaces. The Id of the name spcace is same as container id.

#Isolation inside container. Host is able to see entire thing inside container but not the opposite. Namespace provides the isolation

docker exec -it #Attach the command in the container.

  

docker ps -qa|xargs -n 1 docker rm #Delete all container. Issuing docker rm one by one on all the containers.

docker image rm #It deletes image. If you dont specify "image", then container is deleted.

docker run --name -v /host_dir:/docker_dir:ro #Give access to host resources to the container. Host dir is mapped to the container dir & container dir has the given permissions. -v ->To map volume. -p -> To map ports

docker kill

docker port # 80/tcp -> IP:8080 . TCP port 80 of container is exposed as 8080 in host over IP.

#You can attach a process running in container to the host and vice versa.

Export or give it to customer

docker export -o

Now import that tardocker import

docker run --name -d

docker search repository #You can create your own repository & search inside it