Chef

Table Of Contents

chef-client

A chef-client is an agent that runs locally on every node that is under management by Chef. When a chef-client is run, it will perform all of the steps that are required to bring the node into the expected state, including:

  • Registering and authenticating the node with the Chef server
  • Building the node object
  • Synchronizing cookbooks
  • Compiling the resource collection by loading each of the required cookbooks, including recipes, attributes, and all other dependencies
  • Taking the appropriate and required actions to configure the node
  • Looking for exceptions and notifications, handling each as required

Note

The chef-client executable can be run as a daemon.

Node Types

A node is any physical, virtual, or cloud machine that is configured to be maintained by a chef-client.

There following types of nodes can be managed by the chef-client:

Feature Description
_images/icon_node_cloud.png A cloud-based node is hosted in an external cloud-based service, such as Amazon Virtual Private Cloud, OpenStack, Rackspace, Google Compute Engine, Linode, or Microsoft Azure. Plugins are available for knife that provide support for external cloud-based services. knife can use these plugins to create instances on cloud-based services. Once created, the chef-client can be used to deploy, configure, and maintain those instances.
_images/icon_node_physical.png A physical node is typically a server or a virtual machine, but it can be any active device attached to a network that is capable of sending, receiving, and forwarding information over a communications channel. In other words, a physical node is any active device attached to a network that can run a chef-client and also allow that chef-client to communicate with a Chef server.
_images/icon_node_virtual.png A virtual node is a machine that runs only as a software implementation, but otherwise behaves much like a physical machine.
_images/icon_node_networking.png A network node is any networking device—a switch, a router—that is being managed by a chef-client, such as networking devices by Juniper Networks, Arista, Cisco, and F5. Use Chef to automate common network configurations, such physical and logical Ethernet link properties and VLANs, on these devices.

Some important components of nodes include:

Feature Description
_images/icon_chef_client.png

A chef-client is an agent that runs locally on every node that is under management by Chef. When a chef-client is run, it will perform all of the steps that are required to bring the node into the expected state, including:

  • Registering and authenticating the node with the Chef server
  • Building the node object
  • Synchronizing cookbooks
  • Compiling the resource collection by loading each of the required cookbooks, including recipes, attributes, and all other dependencies
  • Taking the appropriate and required actions to configure the node
  • Looking for exceptions and notifications, handling each as required
_images/icon_ohai.png

Ohai is a tool that is used to detect attributes on a node, and then provide these attributes to the chef-client at the start of every chef-client run. Ohai is required by the chef-client and must be present on a node. (Ohai is installed on a node as part of the chef-client install process.)

The types of attributes Ohai collects include (but are not limited to):

  • Platform details
  • Network usage
  • Memory usage
  • CPU data
  • Kernel data
  • Host names
  • Fully qualified domain names
  • Other configuration details

Attributes that are collected by Ohai are automatic attributes, in that these attributes are used by the chef-client to ensure that these attributes remain unchanged after the chef-client is done configuring the node.

The chef-client Run

A “chef-client run” is the term used to describe a series of steps that are taken by the chef-client when it is configuring a node. The following diagram shows the various stages that occur during the chef-client run, and then the list below the diagram describes in greater detail each of those stages.

_images/chef_run.png

During every chef-client run, the following happens:

Stages Description
Get configuration data The chef-client gets process configuration data from the client.rb file on the node, and then gets node configuration data from Ohai. One important piece of configuration data is the name of the node, which is found in the node_name attribute in the client.rb file or is provided by Ohai. If Ohai provides the name of a node, it is typically the FQDN for the node, which is always unique within an organization.
Authenticate to the Chef Server The chef-client authenticates to the Chef server using an RSA private key and the Chef server API. The name of the node is required as part of the authentication process to the Chef server. If this is the first chef-client run for a node, the chef-validator will be used to generate the RSA private key.
Get, rebuild the node object The chef-client pulls down the node object from the Chef server. If this is the first chef-client run for the node, there will not be a node object to pull down from the Chef server. After the node object is pulled down from the Chef server, the chef-client rebuilds the node object. If this is the first chef-client run for the node, the rebuilt node object will contain only the default run-list. For any subsequent chef-client run, the rebuilt node object will also contain the run-list from the previous chef-client run.
Expand the run-list The chef-client expands the run-list from the rebuilt node object, compiling a full and complete list of roles and recipes that will be applied to the node, placing the roles and recipes in the same exact order they will be applied. (The run-list is stored in each node object’s JSON file, grouped under run_list.)
Synchronize cookbooks The chef-client asks the Chef server for a list of all cookbook files (including recipes, templates, resources, providers, attributes, libraries, and definitions) that will be required to do every action identified in the run-list for the rebuilt node object. The Chef server provides to the chef-client a list of all of those files. The chef-client compares this list to the cookbook files cached on the node (from previous chef-client runs), and then downloads a copy of every file that has changed since the previous chef-client run, along with any new files.
Reset node attributes All attributes in the rebuilt node object are reset. All attributes from attribute files, environments, roles, and Ohai are loaded. Attributes that are defined in attribute files are first loaded according to cookbook order. For each cookbook, attributes in the default.rb file are loaded first, and then additional attribute files (if present) are loaded in lexical sort order. All attributes in the rebuilt node object are updated with the attribute data according to attribute precedence. When all of the attributes are updated, the rebuilt node object is complete.
Compile the resource collection The chef-client identifies each resource in the node object and builds the resource collection. Libraries are loaded first to ensure that all language extensions and Ruby classes are available to all resources. Next, attributes are loaded, followed by lightweight resources, and then all definitions (to ensure that any pseudo-resources within definitions are available). Finally, all recipes are loaded in the order specified by the expanded run-list. This is also referred to as the “compile phase”.
Converge the node The chef-client configures the system based on the information that has been collected. Each resource is executed in the order identified by the run-list, and then by the order in which each resource is listed in each recipe. Each resource in the resource collection is mapped to a provider. The provider examines the node, and then does the steps necessary to complete the action. And then the next resource is processed. Each action configures a specific part of the system. This process is also referred to as convergence. This is also referred to as the “execution phase”.
Update the node object, process exception and report handlers

When all of the actions identified by resources in the resource collection have been done, and when the chef-client run finished successfully, the chef-client updates the node object on the Chef server with the node object that was built during this chef-client run. (This node object will be pulled down by the chef-client during the next chef-client run.) This makes the node object (and the data in the node object) available for search.

The chef-client always checks the resource collection for the presence of exception and report handlers. If any are present, each one is processed appropriately.

Stop, wait for the next run When everything is configured and the chef-client run is complete, the chef-client stops and waits until the next time it is asked to run.

chef-validator

Every request made by the chef-client to the Chef server must be an authenticated request using the Chef server API and a private key. When the chef-client makes a request to the Chef server, the chef-client authenticates each request using a private key located in /etc/chef/client.pem.

However, during the first chef-client run, this private key does not exist. Instead, the chef-client will attempt to use the private key assigned to the chef-validator, located in /etc/chef/validation.pem. (If, for any reason, the chef-validator is unable to make an authenticated request to the Chef server, the initial chef-client run will fail.)

During the initial chef-client run, the chef-client will register with the Chef server using the private key assigned to the chef-validator, after which the chef-client will obtain a client.pem private key for all future authentication requests to the Chef server.

After the initial chef-client run has completed successfully, the chef-validator is no longer required and may be deleted from the node. Use the delete_validation recipe found in the chef-client cookbook (https://github.com/opscode-cookbooks/chef-client) to remove the chef-validator.

SSL Certificates

An SSL certificate is used between the chef-client and the Chef server to ensure that each node has access to the right data.

Signed Header Authentication

Signed header authentication is used to validate communications between the Chef server and any node that is being managed by the Chef server. An API client manages each authentication request. A public and private key pair is used for the authentication itself. The public key is stored in the database on the Chef server. The private key is stored locally on each node and is kept separate from node data (typically in the /etc/chef/client.pem directory). Each request to the Chef server by a node must include a request signature in the HTTP headers. This signature is computed from a hash of request content and is encrypted using the private key.

About Bootstrap Operations

The knife bootstrap command is a common way to install the chef-client on a node. The default for this approach assumes that node can access the Chef website so that it may download the chef-client package from that location.

The omnibus installer will detect the version of the operating system, and then install the appropriate version of the chef-client using a single command to install the chef-client and all of its dependencies, including an embedded version of Ruby, RubyGems, OpenSSL, key-value stores, parsers, libraries, and command line utilities.

The omnibus installer puts everything into a unique directory (/opt/chef/) so that the chef-client will not interfere with other applications that may be running on the target machine. Once installed, the chef-client requires a few more configuration steps before it can perform its first chef-client run on a node.

The following diagram shows the stages of the bootstrap operation, and then the list below the diagram describes in greater detail each of those stages.

_images/chef_bootstrap.png

During a knife bootstrap bootstrap operation, the following happens:

Stages Description
$ knife bootstrap

On UNIX- and Linux-based machines: The knife bootstrap command is issued from a workstation. The hostname, IP address, or FQDN of the target node is issued as part of this command. An SSH connection is established with the target node using port 22. A shell script is assembled using the chef-full.erb (the default bootstrap template), and is then executed on the target node.

On Microsoft Windows machines: The knife bootstrap windows winrm command is issued from a workstation. (This command is part of the knife windows plugin.) The hostname, IP address, or FQDN of the target node is issued as part of this command. A connection is established with the target node using WinRM over port 5985. (WinRM must be enabled with the corresponding firewall rules in place.)

Get the install script from Chef

On UNIX- and Linux-based machines: The shell script that is derived from the chef-full.erb bootstrap template will make a request to the Chef website to get the most recent version of a second shell script (install.sh).

On Microsoft Windows machines: The batch file that is derived from the windows-chef-client-msi.erb bootstrap template will make a request to the Chef website to get the .msi installer.

Get the chef-client package from Chef The second shell script (or batch file) then gathers system-specific information and determines the correct package for the chef-client. The second shell script (or batch file) makes a request to the Chef website, and then downloads the appropriate package from s3.amazonaws.com.
Install the chef-client The chef-client is installed on the target node.
Start the chef-client run

On UNIX- and Linux-based machines: The second shell script executes the chef-client binary with a set of initial settings stored within first-boot.json on the node. first-boot.json is generated from the workstation as part of the initial knife bootstrap command.

On Microsoft Windows machines: The batch file that is derived from the windows-chef-client-msi.erb bootstrap template executes the chef-client binary with a set of initial settings stored within first-boot.json on the node. first-boot.json is generated from the workstation as part of the initial knife bootstrap command.

Complete the chef-client run

The chef-client run proceeds, using HTTPS (port 443), and registers the node with the Chef server.

The first chef-client run, by default, contains an empty run-list. A run-list can be specified as part of the initial bootstrap operation using the --run-list option as part of the knife bootstrap command.