Chef

Table Of Contents

remote_file

A resource defines the desired state for a single configuration item present on a node that is under management by Chef. A resource collection—one (or more) individual resources—defines the desired state for the entire node. During every chef-client run, the current state of each resource is tested, after which the chef-client will take any steps that are necessary to repair the node and bring it back into the desired state.

Use the remote_file resource to transfer a file from a remote location using file specificity. This resource is similar to the file resource.

Note

Fetching files from the files/ directory in a cookbook should be done with the cookbook_file resource.

Syntax

The syntax for using the remote_file resource in a recipe is as follows:

remote_file "name" do
  attribute "value" # see attributes section below
  ...
  action :action # see actions section below
end

where

  • remote_file tells the chef-client to use the Chef::Provider::File::RemoteFile provider during the chef-client run
  • name is the name of the resource block; when the path attribute is not specified as part of a recipe, name is also the path to the remote file
  • attribute is zero (or more) of the attributes that are available for this resource
  • :action is the step that the resource will ask the provider to take during the chef-client run

The following is an example of how the remote_file resource can work when used in a recipe:

remote_file "#{Chef::Config[:file_cache_path]}/large-file.tar.gz" do
  source "http://www.example.org/large-file.tar.gz"
end

Actions

This resource has the following actions:

Action Description
:create Default. Use to create a file. If a file already exists (but does not match), use to update that file to match.
:create_if_missing Use to create a file only if the file does not exist. (When the file exists, nothing happens.)
:delete Use to delete a file.
:touch Use to touch a file. This updates the access (atime) and file modification (mtime) times for a file. (This action may be used with this resource, but is typically only used with the file resource.)

Attributes

This resource has the following attributes:

Attribute Description
atomic_update Use to perform atomic file updates on a per-resource basis. Set to true for atomic file updates. Set to false for non-atomic file updates. (This setting overrides file_atomic_update, which is a global setting found in the client.rb file.) Default value: true.
backup The number of backups to be kept. Set to false to prevent backups from being kept. Default value: 5.
checksum Optional, see use_conditional_get. The SHA-256 checksum of the file. Use to prevent the remote_file resource from re-downloading a file. When the local file matches the checksum, the chef-client will not download it.
force_unlink Use to specify how the chef-client handles certain situations when the target file turns out not to be a file. For example, when a target file is actually a symlink. Set to true to have the chef-client delete the non-file target and replace it with the specified file. Set to false for the chef-client to raise an error. Default value: false.
ftp_active_mode Use to specify if the chef-client will use active or passive FTP. Set to true to use active FTP. Default value: false.
group A string or ID that identifies the group owner by group name, including fully qualified group names such as domain\group or group@domain. If this value is not specified, existing groups will remain unchanged and new group assignments will use the default POSIX group (if available).
headers A Hash of custom headers. Default value: {}.
inherits Microsoft Windows only. Use to specify that a file inherits rights from its parent directory. Default value: true.
manage_symlink_source Use to have the chef-client detect and manage the source file for a symlink. Possible values: nil, true, or false. When this value is set to nil, the chef-client will manage a symlink’s source file and emit a warning. When this value is set to true, the chef-client will manage a symlink’s source file and not emit a warning. Default value: nil. The default value will be changed to false in a future version.
mode

A quoted string that defines the octal mode for a file. If mode is not specified and if the file already exists, the existing mode on the file is used. If mode is not specified, the file does not exist, and the :create action is specified, the chef-client will assume a mask value of "0777" and then apply the umask for the system on which the file will be created to the mask value. For example, if the umask on a system is "022", the chef-client would use the default value of "0755".

The behavior is different depending on the platform.

UNIX- and Linux-based systems: A quoted string that defines the octal mode that is passed to chmod. If the value is specified as a quoted string, it will work exactly as if the chmod command was passed. If the value is specified as an integer, prepend a zero (0) to the value to ensure it is interpreted as an octal number. For example, to assign read, write, and execute rights for all users, use "0777" or "777"; for the same rights, plus the sticky bit, use "01777" or "1777".

Microsoft Windows: A quoted string that defines the octal mode that is translated into rights for Microsoft Windows security. Values up to "0777" are allowed (no sticky bits) and mean the same in Microsoft Windows as they do in UNIX, where 4 equals GENERIC_READ, 2 equals GENERIC_WRITE, and 1 equals GENERIC_EXECUTE. This attribute cannot be used to set :full_control. This attribute has no effect if not specified, but when this attribute and rights are both specified, the effects will be cumulative.

owner A string or ID that identifies the group owner by user name, including fully qualified user names such as domain\user or user@domain. If this value is not specified, existing owners will remain unchanged and new owner assignments will use the current user (when necessary).
path The path to the file. Using a fully qualified path is recommended, but is not always required. Default value: the name of the resource block. (See “Syntax” section above for more information.)
provider Optional. Use to explicitly specify a provider. (See “Providers” section below for more information.)
rights Microsoft Windows only. The permissions for users and groups in a Microsoft Windows environment. For example: rights <permissions>, <principal>, <options> where <permissions> specifies the rights granted to the principal, <principal> is the group or user name, and <options> is a Hash with one (or more) advanced rights options.
source

Required. The location (URI) of the source file. This value may also specify HTTP (http://), FTP (ftp://), or local (file://) source file locations. There are many ways to define the location of a source file. By using a path:

source "http://couchdb.apache.org/img/sketch.png"

By using a node attribute:

source node['nginx']['foo123']['url']

By using attributes to define paths:

source "#{node['python']['url']}/#{version}/Python-#{version}.tar.bz2"

By defining multiple paths for multiple locations:

source "http://seapower/spring.png", "http://seapower/has.png", "http://seapower/sprung.png"

By defining those same multiple paths as an array:

source ["http://seapower/spring.png", "http://seapower/has.png", "http://seapower/sprung.png"]

When multiple paths are specified, the chef-client will attempt to download the files in the order listed, stopping after the first successful download.

use_conditional_get Use to enable conditional HTTP requests by using a conditional GET (with the If-Modified-Since header) or an opaque identifier (ETag). To use If-Modified-Since headers, use_last_modified must also be set to true. To use ETag headers, use_etag must also be set to true. Default value: true.
use_etag Use to enable ETag headers. Set to false to disable ETag headers. To use this setting, use_conditional_get must also be set to true. Default value: true.
use_last_modified Use to enable If-Modified-Since headers. Set to false to disable If-Modified-Since headers. To use this setting, use_conditional_get must also be set to true. Default value: true.

Warning

For a machine on which SELinux is enabled, the chef-client will create files that correctly match the default policy settings only when the cookbook that defines the action also conforms to the same policy.

Atomic File Updates

Atomic updates are used with file-based resources to help ensure that file updates can be made when updating a binary or if disk space runs out.

Atomic updates are enabled by default. They can be managed globally using the file_atomic_update attribute in the client.rb file. They can be managed on a per-resource basis using the atomic_update attribute that is available with the cookbook_file, file, remote_file, and template resources.

Note

On certain platforms, and after a file has been moved into place, the chef-client may modify file permissions to support features specific to those platforms. On platforms with SELinux enabled, the chef-client will fix up the security contexts after a file has been moved into the correct location by running the restorecon command. On the Microsoft Windows platform, the chef-client will create files so that ACL inheritance works as expected.

Windows File Security

To support Microsoft Windows security, the template, file, remote_file, cookbook_file, directory, and remote_directory resources support the use of inheritance and access control lists (ACLs) within recipes.

Access Control Lists (ACLs)

The rights attribute can be used in a recipe to manage access control lists (ACLs), which allow permissions to be given to multiple users and groups. The syntax for the rights attribute is as follows:

rights permission, principal, option_type => value

where

  • permission is used to specify which rights will be granted to the principal. The possible values are: :read, :write, read_execute, :modify, :full_control, and :deny.

    These permissions are cumulative. If :write is specified, then it includes :read. If :full_control is specified, then it includes both :write and :read. If :deny is specified, then the user or group will not have rights to the object.

    (For those who know the Microsoft Windows API: :read corresponds to GENERIC_READ; :write corresponds to GENERIC_WRITE; :read_execute corresponds to GENERIC_READ and GENERIC_EXECUTE; :modify corresponds to GENERIC_WRITE, GENERIC_READ, GENERIC_EXECUTE, and DELETE; :full_control corresponds to GENERIC_ALL, which allows a user to change the owner and other metadata about a file.)

  • principal is used to specify a group or user name. This is identical to what is entered in the login box for Microsoft Windows, such as user_name, domain\user_name, or user_name@fully_qualified_domain_name. The chef-client does not need to know if a principal is a user or a group.

  • option_type is a hash that contains advanced rights options. For example, the rights to a directory that only applies to the first level of children might look something like: rights :write, "domain\group_name", :one_level_deep => true. Possible option types:

    Option Type Description
    :applies_to_children Use to specify how permissions are applied to children. Possible values: true to inherit both child directories and files; false to not inherit any child directories or files; :containers_only to inherit only child directories (and not files); :objects_only to recursively inherit files (and not child directories).
    :applies_to_self Indicates whether a permission is applied to the parent directory. Possible values: true to apply to the parent directory or file and its children; false to not apply only to child directories and files.
    :one_level_deep Indicates the depth to which permissions will be applied. Possible values: true to apply only to the first level of children; false to apply to all children.

The rights attribute can be used as many times as necessary; the chef-client will apply them to the file or directory as required. For example:

resource "x.txt" do
  rights :read, "Everyone"
  rights :write, "domain\group"
  rights :full_control, "group_name_or_user_name"
  rights :full_control, "user_name", :applies_to_children => true
end

or:

rights :read, ["Administrators","Everyone"]
rights :deny, ["Julian", "Lewis"]
rights :full_control, "Users", :applies_to_children => true
rights :write, "Sally", :applies_to_children => :containers_only, :applies_to_self => false, :one_level_deep => true

Some other important things to know when using the rights attribute:

  • Order independence. It doesn’t matter if rights :deny, ["Julian", "Lewis"] is placed before or after rights :read, ["Julian", "Lewis"], both Julian and Lewis will be unable to read the document.
  • Only inherited rights remain. All existing explicit rights on the object are removed and replaced.
  • If rights are not specified, nothing will be changed. The chef-client does not clear out the rights on a file or directory if rights are not specified.
  • Changing inherited rights can be expensive. Microsoft Windows will propagate rights to all children recursively due to inheritance. This is a normal aspect of Microsoft Windows, so consider the frequency with which this type of action is necessary and take steps to control this type of action if performance is the primary consideration.

Inheritance

By default, a file or directory inherits rights from its parent directory. Most of the time this is the preferred behavior, but sometimes it may be necessary to take steps to more specifically control rights. The inherits attribute can be used to specifically tell the chef-client to apply (or not apply) inherited rights from its parent directory.

For example, the following example specifies the rights for a directory:

directory 'C:\mordor' do
  rights :read, 'MORDOR\Minions'
  rights :full_control, 'MORDOR\Sauron'
end

and then the following example specifies how to use inheritance to deny access to the child directory:

directory 'C:\mordor\mount_doom' do
  rights :full_control, 'MORDOR\Sauron'
  inherits false # Sauron is the only person who should have any sort of access
end

If the :deny permission were to be used instead, something could slip through unless all users and groups were denied.

Another example also shows how to specify rights for a directory:

directory 'C:\mordor' do
  rights :read, 'MORDOR\Minions'
  rights :full_control, 'MORDOR\Sauron'
  rights :write, 'SHIRE\Frodo' # Who put that there I didn't put that there
end

but then not use the inherits attribute to deny those rights on a child directory:

directory 'C:\mordor\mount_doom' do
  rights :deny, 'MORDOR\Minions' # Oops, not specific enough
end

Because the inherits attribute is not specified, the chef-client will default it to true, which will ensure that security settings for existing files remain unchanged.

Prevent Re-downloads

To prevent the chef-client from re-downloading files that are already present on a node, use one of the following attributes in a recipe: use_conditional_get (default) or checksum.

  • The use_conditional_get attribute is the default behavior of the chef-client. If the remote file is located on a server that supports ETag and/or If-Modified-Since headers, the chef-client will use a conditional GET to determine if the file has been updated. If the file has been updated, the chef-client will re-download the file.
  • The checksum attribute will ask the chef-client to compare the checksum for the local file to the one at the remote location. If they match, the chef-client will not re-download the file. Using a local checksum for comparison requires that the local checksum be the correct checksum.

The desired approach just depends on the desired workflow. For example, if a node requires a new file every day, using the checksum approach would require that the local checksum be updated and/or verified every day as well, in order to ensure that the local checksum was the correct one. Using a conditional GET in this scenario will greatly simplify the management required to ensure files are being updated accurately.

Providers

The chef-client will attempt to determine the correct provider during the chef-client run, and then choose the best/correct provider based on configuration data collected at the start of the chef-client run. In general, a specific provider does not need to be specified. In situations where a specific provider must be specified, there are two approaches:

  • Using a provider’s short name as the name of the resource, e.g. short_name "foo" do
  • Using the provider attribute to specify the long name as an attribute of a resource, e.g. provider Chef::Provider::Long::Name

Whenever possible, try using the short name first, and then use the long name when necessary.

This resource has the following providers:

Long name Short name Notes
Chef::Provider::File::RemoteFile remote_file The default provider for all platforms.

Examples

The following examples demonstrate various approaches for using resources in recipes. If you want to see examples of how Chef uses resources in recipes, take a closer look at the cookbooks that Chef authors and maintains: https://github.com/opscode-cookbooks.

Transfer a file from a URL

remote_file "/tmp/testfile" do
  source "http://www.example.com/tempfiles/testfile"
  mode '0644'
  checksum "3a7dac00b1" # A SHA256 (or portion thereof) of the file.
end

Install a file from a remote location using bash

The following is an example of how to install the foo123 module for Nginx. This module adds shell-style functionality to an Nginx configuration file and does the following:

  • Declares three variables
  • Gets the Nginx file from a remote location
  • Installs the file using Bash to the path specified by the src_filepath variable
#  the following code sample is similar to the ``upload_progress_module`` recipe in the ``nginx`` cookbook: https://github.com/opscode-cookbooks/nginx

src_filename = "foo123-nginx-module-v#{node['nginx']['foo123']['version']}.tar.gz"
src_filepath = "#{Chef::Config['file_cache_path']}/#{src_filename}"
extract_path = "#{Chef::Config['file_cache_path']}/nginx_foo123_module/#{node['nginx']['foo123']['checksum']}"

remote_file src_filepath do
  source node['nginx']['foo123']['url']
  checksum node['nginx']['foo123']['checksum']
  owner 'root'
  group 'root'
  mode '0644'
end

bash 'extract_module' do
  cwd ::File.dirname(src_filepath)
  code <<-EOH
    mkdir -p #{extract_path}
    tar xzf #{src_filename} -C #{extract_path}
    mv #{extract_path}/*/* #{extract_path}/
    EOH
  not_if { ::File.exists?(extract_path) }
end

Store certain settings

The following recipe shows how an attributes file can be used to store certain settings. An attributes file is located in the attributes/ directory in the same cookbook as the recipe which calls the attributes file. In this example, the attributes file specifies certain settings for Python that are then used across all nodes against which this recipe will run.

Python packages have versions, installation directories, URLs, and checksum files. An attributes file that exists to support this type of recipe would include settings like the following:

default['python']['version'] = '2.7.1'

if python['install_method'] == 'package'
  default['python']['prefix_dir'] = '/usr'
else
  default['python']['prefix_dir'] = '/usr/local'
end

default['python']['url'] = 'http://www.python.org/ftp/python'
default['python']['checksum'] = '80e387...85fd61'

and then the methods in the recipe may refer to these values. A recipe that is used to install Python will need to do the following:

  • Identify each package to be installed (implied in this example, not shown)
  • Define variables for the package version and the install_path
  • Get the package from a remote location, but only if the package does not already exist on the target system
  • Use the bash resource to install the package on the node, but only when the package is not already installed
#  the following code sample comes from the ``oc-nginx`` cookbook on |github|: https://github.com/cookbooks/oc-nginx

version = node['python']['version']
install_path = "#{node['python']['prefix_dir']}/lib/python#{version.split(/(^\d+\.\d+)/)[1]}"

remote_file "#{Chef::Config[:file_cache_path]}/Python-#{version}.tar.bz2" do
  source "#{node['python']['url']}/#{version}/Python-#{version}.tar.bz2"
  checksum node['python']['checksum']
  mode '0644'
  not_if { ::File.exists?(install_path) }
end

bash "build-and-install-python" do
  cwd Chef::Config[:file_cache_path]
  code <<-EOF
    tar -jxvf Python-#{version}.tar.bz2
    (cd Python-#{version} && ./configure #{configure_options})
    (cd Python-#{version} && make && make install)
  EOF
  not_if { ::File.exists?(install_path) }
end

Use the platform_family? method

The following is an example of using the platform_family? method in the Recipe DSL to create a variable that can be used with other resources in the same recipe. In this example, platform_family? is being used to ensure that a specific binary is used for a specific platform before using the remote_file resource to download a file from a remote location, and then using the execute resource to install that file by running a command.

if platform_family?("rhel")
  pip_binary = "/usr/bin/pip"
else
  pip_binary = "/usr/local/bin/pip"
end

remote_file "#{Chef::Config[:file_cache_path]}/distribute_setup.py" do
  source "http://python-distribute.org/distribute_setup.py"
  mode '0644'
  not_if { ::File.exists?(pip_binary) }
end

execute "install-pip" do
  cwd Chef::Config[:file_cache_path]
  command <<-EOF
    # command for installing Python goes here
    EOF
  not_if { ::File.exists?(pip_binary) }
end

where a command for installing Python might look something like:

#{node['python']['binary']} distribute_setup.py
#{::File.dirname(pip_binary)}/easy_install pip

Specify local Windows file path as a valid URI

When specifying a local Microsoft Windows file path as a valid file URI, an additional forward slash (/) is required. For example:

remote_file "file:///c:/path/to/file" do
  ...       # other attributes
end