Skip to content

Latest commit

 

History

History
478 lines (391 loc) · 14.9 KB

managing_pods.adoc

File metadata and controls

478 lines (391 loc) · 14.9 KB

Managing Pods

Overview

This topic describes the management of pods, including managing their networks, limiting their run-once duration, and limiting what they can access, and how much bandwidth they can use.

Managing Pod Networks

When your cluster is configured to use the ovs-multitenant SDN plug-in, you can manage the separate pod overlay networks for projects using the administrator CLI.

Joining Project Networks

To join projects to an existing project network:

$ oadm pod-network join-projects --to=<project1> <project2> <project3>

In the above example, all the pods and services in <project2> and <project3> can now access any pods and services in <project1> and vice versa.

Alternatively, instead of specifying specific project names, you can use the --selector=<project_selector> option.

Isolating Project Networks

To isolate the project network in the cluster and vice versa, run:

$ oadm pod-network isolate-projects <project1> <project2>

In the above example, all of the pods and services in <project1> and <project2> can not access any pods and services from other non-global projects in the cluster and vice versa.

Alternatively, instead of specifying specific project names, you can use the --selector=<project_selector> option.

Making Project Networks Global

To allow projects to access all pods and services in the cluster and vice versa:

$ oadm pod-network make-projects-global <project1> <project2>

In the above example, all the pods and services in <project1> and <project2> can now access any pods and services in the cluster and vice versa.

Alternatively, instead of specifying specific project names, you can use the --selector=<project_selector> option.

Controlling Egress Traffic

As an {product-title} cluster administrator, you can control egress traffic using either a router or firewall methods.

Limiting Pod Access with an Egress Router

The {product-title} egress router runs a service that redirects traffic to a specified remote server, using a private source IP address that is not used for anything else. The service allow pods to talk to servers that are set up to only allow access from whitelisted IP addresses.

Important

If you are deploying {product-title} on Red Hat OpenStack Platform, you need to whitelist the IP and MAC addresses on your Openstack environment, otherwise communication will fail:

neutron port-update $neutron_port_uuid \
  --allowed_address_pairs list=true \
  type=dict mac_address=1<mac_address>,ip_address=<ip_address>

Deploying an Egress Router Pod

Example 1. Example Pod Definition for an Egress Router
apiVersion: v1
kind: Pod
metadata:
  name: egress-1
  labels:
    name: egress-1
  annotations:
    pod.network.openshift.io/assign-macvlan: "true"
spec:
  containers:
  - name: egress-router
    image: openshift/origin-egress-router
    securityContext:
      privileged: true
    env:
    - name: EGRESS_SOURCE (1)
      value: 192.168.12.99
    - name: EGRESS_GATEWAY (2)
      value: 192.168.12.1
    - name: EGRESS_DESTINATION (3)
      value: 203.0.113.25
  nodeSelector:
    site: springfield-1 (4)

  1. IP address on the node subnet reserved by the cluster administrator for use by this pod.

  2. Same value as the default gateway used by the node itself.

  3. Connections to the pod are redirected to 203.0.113.25, with a source IP address of 192.168.12.99

  4. The pod will only be deployed to nodes with the label site springfield-1.

The pod.network.openshift.io/assign-macvlan annotation creates a Macvlan network interface on the primary network interface, and then moves it into the pod’s network name space before starting the egress-router container.

Note

Preserve the the quotation marks around "true". Omitting them will result in errors.

The pod contains a single container, using the openshift/origin-egress-router image, and that container is run privileged so that it can configure the Macvlan interface and set up iptables rules.

The environment variables tell the egress-router image what addresses to use; it will configure the Macvlan interface to use EGRESS_SOURCE as its IP address, with EGRESS_GATEWAY as its gateway.

NAT rules are set up so that connections to any TCP or UDP port on the pod’s cluster IP address are redirected to the same port on EGRESS_DESTINATION.

If only some of the nodes in your cluster are capable of claiming the specified source IP address and using the specified gateway, you can specify a nodeName or nodeSelector indicating which nodes are acceptable.

Deploying an Egress Router Service

Though not strictly necessary, you normally want to create a service pointing to the egress router:

apiVersion: v1
kind: Service
metadata:
  name: egress-1
spec:
  ports:
  - name: http
    port: 80
  - name: https
    port: 443
  type: ClusterIP
  selector:
    name: egress-1

Your pods can now connect to this service. Their connections are redirected to the corresponding ports on the external server, using the reserved egress IP address.

Limiting Pod Access with Egress Firewall

As an {product-title} cluster administrator, you can use egress policy to limit the external addresses that some or all pods can access from within the cluster, so that:

  • A pod can only talk to internal hosts, and cannot initiate connections to the public Internet.

    Or,

  • A pod can only talk to the public Internet, and cannot initiate connections to internal hosts (outside the cluster).

    Or,

  • A pod cannot reach specified internal subnets/hosts that it should have no reason to contact.

For example, you can configure projects with different egress policies, allowing <project A> access to a specified IP range, but denying the same access to <project B>.

Caution

You must have the ovs-multitenant plug-in enabled in order to limit pod access via egress policy.

Project administrators can neither create EgressNetworkPolicy objects, nor edit the ones you create in their project. There are also several other restrictions on where EgressNetworkPolicy may be created:

  1. The default project (and any other project that has been made global via oadm pod-network make-projects-global) cannot have egress policy.

  2. If you merge two projects together (via oadm pod-network join-projects), then you cannot use egress policy in any of the joined projects.

  3. No project may have more than one egress policy object.

Violating any of these restrictions will result in broken egress policy for the project, and may cause all external network traffic to be dropped.

Configuring Pod Access Limits

To configure pod access limits, you must use the oc command or the REST API. You can use oc [create|replace|delete] to manipulate EgressNetworkPolicy objects. The api/swagger-spec/oapi-v1.json file has API-level details on how the objects actually work.

To configure pod access limits:

  1. Navigate to the project you want to affect.

  2. Create a JSON file for the pod limit policy:

    # oc create -f <policy>.json

  3. Configure the JSON file with policy details. For example:

    {
    "kind": "EgressNetworkPolicy",
    "apiVersion": "v1",
    "metadata": {
        "name": "default"
    },
    "spec": {
        "egress": [
            {
                "type": "Allow",
                "to": {
                    "cidrSelector": "1.2.3.0/24"
                }
            },
            {
                "type": "Deny",
                "to": {
                    "cidrSelector": "0.0.0.0/0"
                }
            }
        ]
    }
}

    When the example above is added in a project, it allows traffic to 1.2.3.0/24, but denies access to all other external IP addresses. (Traffic to other pods is not affected, because the policy only applies to external traffic.)

Note that the rules in an EgressNetworkPolicy are checked in order, and the first one that matches takes effect. So if the two rules in the above example were swapped, then traffic would not be allowed to 1.2.3.0/24, because the 0.0.0.0/0 rule would be checked first, and it would match and deny all traffic.

Limiting the Bandwidth Available to Pods

You can apply quality-of-service traffic shaping to a pod and effectively limit its available bandwidth. Egress traffic (from the pod) is handled by policing, which simply drops packets in excess of the configured rate. Ingress traffic (to the pod) is handled by shaping queued packets to effectively handle data. The limits you place on a pod do not affect the bandwidth of other pods.

To limit the bandwidth on a pod:

  1. Write an object definition JSON file, and specify the data traffic speed using kubernetes.io/ingress-bandwidth and kubernetes.io/egress-bandwidth annotations. For example, to limit both pod egress and ingress bandwidth to 10M/s:

    Example 2. Limited Pod Object Definition
    {
    "kind": "Pod",
    "spec": {
        "containers": [
            {
                "image": "nginx",
                "name": "nginx"
            }
        ]
    },
    "apiVersion": "v1",
    "metadata": {
        "name": "iperf-slow",
        "annotations": {
            "kubernetes.io/ingress-bandwidth": "10M",
            "kubernetes.io/egress-bandwidth": "10M"
        }
    }
}

  2. Create the pod using the object definition:

    oc create -f <file_or_dir_path>

Setting Pod Disruption Budgets

A pod disruption budget is part of the Kubernetes API, which can be managed with oc commands like other object types. They allow the specification of safety constraints on pods during operations, such as draining a node for maintenance.

PodDisruptionBudget is an API object that specifies the minimum number or percentage of replicas that must be up at a time. Setting these in projects can be helpful during node maintenance (such as scaling a cluster down or a cluster upgrade) and is only honored on voluntary evictions (not on node failures).

A PodDisruptionBudget object’s configuration consists of the following key parts:

  • A label selector, which is a label query over a set of pods.

  • An availability level, which specifies the minimum number of pods that must be available simultaneously.

The following is an example of a PodDisruptionBudget resource:

apiVersion: policy/v1alpha1 (1)
kind: PodDisruptionBudget
metadata:
  name: my-pdb
spec:
  selector:  (2)
    matchLabels:
      foo: bar
  minAvailable: 2  (3)

  1. PodDisruptionBudget is part of the policy/v1alpha API group.

  2. A label query over a set of resources. The result of matchLabels and matchExpressions are logically conjoined.

  3. The minimum number of pods that must be available simultaneously. This can be either an integer or a string specifying a percentage (for example, 20%).

If you created a YAML file with the above object definition, you could add it to project with the following:

$ oc create -f </path/to/file> -n <project_name>

You can check for pod disruption budgets across all projects with the following:

$ oc get poddisruptionbudget --all-namespaces

NAMESPACE         NAME          MIN-AVAILABLE   SELECTOR
another-project   another-pdb   4               bar=foo
test-project      my-pdb        2               foo=bar

The PodDisruptionBudget is considered healthy when there are at least minAvailable pods running in the system. Every pod above that limit can be evicted.