1. Foreman 1.17 Manual

Foreman Architecture

A Foreman installation will always contain a central foreman instance that is responsible for providing the Web based GUI, node configurations, initial host configuration files, etc. However, if the foreman installation supports unattended installations then other operations need to be performed to fully automate this process. The smart proxy manages remote services and is generally installed with all Foreman installations to manage TFTP, DHCP, DNS, Puppet, Puppet CA, Ansible, Salt, and Chef.

Smart-Proxy

A Smart-Proxy is located on or near a machine that performs a specific function and helps foreman orchestrate the process of commissioning a new host. Placing the proxy on or near to the actual service will also help reduce latencies in large distributed organizations.

Release notes for 1.17

Headline features

Warning for unsupported PXE loader combinations

UI detects unsupported PXE loader combination and shows a warning when users try to use it for provisioning. (#19554)

Global HTTP proxy for outgoing requests

It is now possible to configure HTTP proxy for outgoing requests directly in Foreman and use it to connect to a compute resource. (#20201, 12216)

Replace horizontal navbar with vertical navigation

Vertical navigation gives Foreman fresh and modern look which aims to achieve better user experience. (#18492)

Expose underlying disk options for oVirt provider

oVirt API 3.x supports additionale options for disk creation (disk interface (virtio/virtio_scsi/ide), ‘wipe after delete’) and these are now supported in Foreman. (#21762)

API for personal access tokens

New API endpoints for personal access tokens were introduced. (#21394)

Select related vmware network when selecting a subnet

The VMWare network that matches the subnet is selected automatically when the subnet is changed. (#21676)

Host interfaces auditing

With auditing of host interfaces, it is now possible to see an IP address of a host that has been destroyed. (#21374)

Release notes for 1.17.4

• Properly serialize Array/hash values for Lookup Values (#24640)
• Don’t remove permissions from default role (#24697)

Release notes for 1.17.3

Compute resources

• VMware “Could not match network interface” on VM creation (#23909)
• Can’t save OpenStack OSP 12 Compute Resource (#23818)
• Failed to update a compute vmware (VMware) instance node.test.domain: undefined method ‘[]’ for nil:NilClass (#19990)

DB migrations

• Nil eror in 20170112175131_migrate_template_to_parameters_macros migration (#23679)

Host groups

• Hostgroup creation/edition issue (#22010)

Smart variables

• All parameters for class overridden by default (#24355)
• Editing Smart Class Parameter of type YAML creates malformed value in release 1.17.0 (#23382)

Users

• No route matches {:action=>”edit”, :controller=>”users”, :id=>4} (#23782)

Roles and Permissions

• Missing permissions seed entry for HostgroupClass (#22983)

Security

• fips mode breaks ESXi deployment (#23621)

Other

• ENC with smart class parameters is slow (regression with 1.16) (#23608)

Release notes for 1.17.2

1.17.2 was not released due to a critical regression in VMware provisioning.

Release notes for 1.17.1

Compute resources

• Unable to create a VM on 1.17.0 on vCenter 6.5 (#23378)
• undefined method vlan when listing vsphere networks (#23276)
• Cannot create VM in a vCenter. NoMethodError. (#23335)
• Changing power state gives: NameError: uninitialized constant Fog::Compute::Ovirt (#23212)

Mailers

• Daily reports throw exception (#23358)

Templates

• undefined method ‘#freeze’ for String::Jail (String) on Ruby 2.5.1 (#23273)

Other

• Remove pin on ruby2ruby < 2.4.0 (#22334)

Release notes for 1.17.0

API

• API Hosts GET returning also owner_name (#20516)
• Administer -> About page should have a bullet link to the local API docs (#20035)
• Provide description for Hammer command to disable/enable notifications for a host. (#19383)

Audit Log

• [Audit] Puppet Environment (#21878)
• Audit host interface changes (#21374)

Authentication

• Pages do not refresh after idle session timeout (#19315)
• Netgroup LDAP Authentication in ldap_fluff (#16112)
• Support for ‘+’ in usernames (#7778)

Authorization

• Provide some description to built-in roles (#20737)

Compute resources

• VM: Show Link to Associated Host (#20822)
• allow compute resource connections to use a proxy (#12216)
• Update fog-core to 1.43.0 (#17377)

Compute resources - OpenStack

• Openstack v3 support (#12054)

Compute resources - VMware

• Support VMWare VMRC Console (#22062)
• Select related vmware network when selecting a subnet (#21676)
• Additional SCSI Controller with per-disk configuration (#4509)

Compute resources - oVirt

• Expose underlying disk options for oVirt provider (#21762)
• Add oVirt instance types / flavors support (#5441)

Core

• Log http ports used by modules during startup (#15535)

Database

• Gain speed with additional indices (#20608)

Facts

• Allow searching facts by host_id (#21606)

Host creation

• Provide warning for unsupported PXE loader combinations (#19554)
• Change the order of tabs in host creation form (#20369)

Host groups

• Add description field to host group (#19125)

Inventory

• Colors used for Failed and for Failed Restarts should be different (#20654)
• API to fetch list of hosts without full host details (#20072)

JavaScript stack

• Update patternfly-react version to 0.18.2 (#21990)
• Update patternfly-react version to 0.13.0 (#21838)
• Include babel-plugin-transform-class-properties so we can use static class variables (#21563)
• Use patternfly-react npm package (#21130)
• Update react to v16.2.0 (#22105)
• Reduce the lodash affect on the bundle (#22101)
• update react to use new MIT license (#21117)
• implement bookmark creation using react (#20839)
• as a developer I would like to create new forms using react/redux (#20838)
• update storybook to 3.x (#20193)
• add storybook deployer task (#20142)
• upgrade to webpack 2 (#19478)
• add the ability to auto run js tests when a test is changed. (#19303)
• Add 3rd party js libraries in plugins (#21180)
• Upgrade to latest recommended babel preset env (#21462)

Logging

• Add logging to smart proxy requests (ProxyAPI) (#10982)

Network

• Fetch a value for a given parameter name (from a provisioning snippet) on a subnet (#19977)
• Add comment field for subnets (#19148)

Notifications

• Add close button to notifications drawer (#21202)
• cache notification drawer state (#20485)
• Allow to mass mark notification group as read (#19192)
• Use Dynflow as an ActiveJob backend (#18618)

Orchestration

• Add a setting that allows the user to enable orchestration when puppet facts are uploaded. (#21565)
• Add VM boot failure faster feedback loop for openstack (#19459)

Organizations and Locations

• Add host count to location API call (#21012)

Plugins-Engines

• Pagelets - add tab extension point for hostgroup edit page (#21658)
• Pagelets - add extension point for host show page (#21579)
• Ability for engine/plugin to extend RABL responses (#3763)
• Add an option to create plugin migrations with properly marked scope (#19589)
• Lock plugin roles (#19039)
• Expose React components from plugins (#20469)

Rails

• Add support for running under Ruby on Rails 5.0 (#19050)

Reporting

• Add scoped_search on host id for reports (#21602)
• Unify the way how we display dates (#19047)

Search

• Filter roles by assigned permissions in UI (#20047)
• Add the ability to search for a puppet class in the configuration groups (#7499)

Security

• Protection from Brute Force Password Attacks (#4238)

Settings

• settings page should show when non-default settings are selected (#21238)
• Global HTTP proxy for outgoing requests (#20201)

Smart Variables

• Separate Smart variables and Smart parameters in Host/HostGroup -> Parameters tab. (#12829)
• Allow searching smart class parameters by environment (#20083)
• Prioritize attribute order in puppet classes limited to 255 chars (#19934)
• Inform user that all Puppet Class sub entities affect saving the whole puppet class (#19582)

Templates

• Append “clone” when cloning provisioning templates (#20669)
• Find is not working in the template editor (#20573)

Users

• add an api for personal access tokens (#21394)
• User Group Interface - Multiple User Accounts With Same Name are Indistinguisable (#20739)
• Allow # character in user name (#20726)

Web Interface

• Add experimental UI setting to foreman (#21223)
• Please use monospaced font in fullscreen editor (#20537)
• CSV export of UI table for hostgroups (#20255)
• use turbolinks to load dashboard refresh faster (#20000)
• search should use turbolinks (#19995)
• update table pagination to use patternfly guidelines (#19031)

foreman-debug

• Print time in foreman-debug verbose mode (#20835)
• include passenger-memory-stats & passenger-status in foreman-debug (#19497)

Other

• Add support for sha512 grub passwords to provisioning templates (#21875)
• Foreman RSS notification should include a user agent value (#21823)
• Register reducers from plugins (#21137)
• Do not throw exception on API power requests for hosts that does not support them. (#20966)
• Comment field wont show up in host page (#20750)
• Default User role that can also save bookmarks (#20736)
• Allow API GET /api/hostgroups to return hostgroup parameters (#20500)
• Add rebuild_config to hostgroup API (#20487)
• Add notifications for an RSS feed (#18292)
• Recognize br-ex, br-int and br-tun as bridge interfaces (#21104)
• Add shell script to start foreman-webpack (#16160)

A full list of changes in 1.17.0 is available via Redmine

Deprecations

No new deprecations with target version 1.19 were added.

APIv1 will be dropped in the next Foreman version

APIv1 has been deprecated since Foreman 1.9 and Foreman 1.17 is the last version supporting APIv1. We strongly recommend moving to v2 if you are still using v1.

Debian support

Due to Ruby incompatibility, Debian/Jessie support was dropped in Foreman 1.17

Upgrade warnings

• When seeding default roles, permission check and auditiong is ignored (#21011)

1.17.4 is the last release for the 1.17 series.

Contributors

We’d like to thank the following people who contributed to the Foreman 1.17 release:

Abhijeet, Adam Cécile, Adam Růžička, Aggelos Avgerinos, Alex Fisher, Alexander “Ace” Olofsson, Amir Fefer, Amit Karsale, Andrew Kofink, Anthony Chevalet, Anurag, Arjen Zonneveld, Avi Sharvit, Baptiste, Ben Ford, Bernhard Suttner, Bobby Prins, Brett Maton, Brett Maton (C), Bryan Kearney, Charles Ott, Chris Baldwin, Chris Duryee, Chris Roberts, Christian, Christian Becker, Colin Wood, Conor O’Callaghan, Cristian Falcas, Daehyung Lee, Damian Lukowski, Dan Seethaler, Daniel Helgenberger, Daniel Henninger, Daniel Kimsey, Daniel Kuffner, Daniel Lobato Garcia, Daniel Rosenbloom, Daniel Schaaff, David Davis, David Moreau-Simard, Derek, Derek Tamsen, Dirk Götz, Dis McCarthy, Dmitri Dolguikh, Dmitry Galkin, Dominic Cleal, Dominik Hlavac, Duncan Innes, Elias Abacioglu, Eric D. Helms, Evgeni Golov, Ewoud Kohl van Wijngaarden, Fabian von Feilitzsch, Fabien Wernli, Ganesh Nalawade, Gerrit, Greg Sutcliffe, Guido Günther, Han Boetes, Himanshu, Ido Kanner, Isabell Cowan, Ivan Nečas, James Gregory, James Powis, Jan Pazdziora, Jan-Otto Kröpke, Jarkko Oranen, Jo Vandeginste, John Mitsch, June Zhang, Justin Sherrill, Kavita Gaikwad, Kilian, Klaas Demter, Koen Torfs, Konrad Scherer, Lachlan Musicman, Levi Smith, Liza Gilman, Lukáš Zapletal, Manfred Pusch, Marcelo Moreira de Mello, Marek Hulán, Markus Frosch, Marten Veldthuis, Martin Bačovský, Martin Dobrev, Mateusz Gozdek, Mathieu Parent, Matt Jarvis, Matthias Baur, Maxim Burgerhout, Michael Hofer, Michael Moll, Nagoor Shaik, Nathan Ward, Neil Hanlon, Ngibb, Nick George, Nicolas Jehle, Ohad Levy, Ondřej Pražák, Ori Rabin, Pablo Hess, Partha Aji, Pat Riehecky, Patrick Reed, Patrick Schoenfeld, Pavel Matěja, Pavel Moravec, Peiman Jafari, Peter Bittner, Peter Ondrejka, Phillip Reisbeck, Planktonette, Rahul Bajaj, Ranjan Kumar, Robert Flechtner, Ron, Rune Hammersland, Sean O’Keeffe, Sebastian Gräßl, Shimon Shtein, Shira Maximov, Shlomi Zadok, Simon Peeters, Simonas Rupšys, Stef Fen, Stefan Dietrich, Stefan Meyer, Stephen Benjamin, Stephen Dunne III, Steve Traylen, Swapnil Abnave, TheKangaroo, Thomas McKay, Thomas Mueller, Tim Meusel, Timo Goebel, Tom Copeland, Tomas Strachota, Tomer Brisker, Tony Coffman, Trey Dockendorf, Walden Raines, Zach Huntington-Meath, Zachary Huntington-Meath, aagrawal, acreek, dae1804, dlucredativ, downsj2, earsdown, jbnance, maduma, mdellweg, mriedel-pgdx, ripcurld0, rusty, rwf14f, schurzi

As well as all users who helped test releases, report bugs and provide feedback on the project.

2. Quickstart

The Foreman installer is a collection of Puppet modules that installs everything required for a full working Foreman setup. It uses native OS packaging (e.g. RPM and .deb packages) and adds necessary configuration for the complete installation.

Components include the Foreman web UI, Smart Proxy, Passenger, a Puppet master (either Puppet Server or under Passenger), and optionally TFTP, DNS and DHCP servers. It is configurable and the Puppet modules can be read or run in “no-op” mode to see what changes it will make.

Supported platforms

• CentOS, Scientific Linux or Oracle Linux 7, x86_64
• Debian 9 (Stretch), i386/amd64/armhf/aarch64
• Red Hat Enterprise Linux 7, x86_64
• Ubuntu 16.04 (Xenial), i386/amd64/armhf/aarch64

Other operating systems will need to use alternative installation methods (see the manual).

The installation will require 4GB of memory, see System Requirements for more information.

2.1 Installation

The Foreman installer uses Puppet (> 3.x required) to install Foreman. This guide assumes that you have a newly installed operating system, on which the installer will setup Foreman, a Puppet master, and the Smart Proxy by default. It’s not advisable to follow the steps below on an existing system, since the installer will affect the configuration of several components.

Select operating system

To provide specific installation instructions, please select your operating system: -- select operating system -- CentOS, Scientific Linux or Oracle Linux 7 Debian 9 (Stretch) Red Hat Enterprise Linux 7 Ubuntu 16.04 (Xenial)

Repositories

sudo yum-config-manager --enable rhel-7-server-optional-rpms

sudo yum -y install https://yum.puppetlabs.com/puppet5/puppet5-release-el-7.noarch.rpm

sudo yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
sudo yum -y install https://yum.theforeman.org/releases/1.17/el7/x86_64/foreman-release.rpm

sudo yum -y install http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
sudo yum -y install https://yum.theforeman.org/releases/1.17/el7/x86_64/foreman-release.rpm

sudo apt-get -y install ca-certificates
cd /tmp && wget https://apt.puppetlabs.com/puppet5-release-stretch.deb
sudo dpkg -i /tmp/puppet5-release-stretch.deb

echo "deb http://deb.theforeman.org/ stretch 1.17" | sudo tee /etc/apt/sources.list.d/foreman.list
echo "deb http://deb.theforeman.org/ plugins 1.17" | sudo tee -a /etc/apt/sources.list.d/foreman.list
sudo apt-get -y install ca-certificates
wget -q https://deb.theforeman.org/pubkey.gpg -O- | sudo apt-key add -

sudo apt-get -y install ca-certificates
cd /tmp && wget https://apt.puppetlabs.com/puppet5-release-xenial.deb
sudo dpkg -i /tmp/puppet5-release-xenial.deb

echo "deb http://deb.theforeman.org/ xenial 1.17" | sudo tee /etc/apt/sources.list.d/foreman.list
echo "deb http://deb.theforeman.org/ plugins 1.17" | sudo tee -a /etc/apt/sources.list.d/foreman.list
sudo apt-get -y install ca-certificates
wget -q https://deb.theforeman.org/pubkey.gpg -O- | sudo apt-key add -

Downloading the installer

sudo yum -y install foreman-installer

sudo apt-get update && sudo apt-get -y install foreman-installer

Running the installer

The installation run is non-interactive, but the configuration can be customized by supplying any of the options listed in foreman-installer --help, or by running foreman-installer -i for interactive mode. More examples are given in the Installation Options section. Adding -v will disable the progress bar and display all changes. To run the installer, execute:

sudo foreman-installer

After it completes, the installer will print some details about where to find Foreman and the Smart Proxy and Puppet master if they were installed along Foreman. Output should be similar to this:

  * Foreman is running at https://theforeman.example.com
      Initial credentials are admin / 3ekw5xtyXCoXxS29
  * Foreman Proxy is running at https://theforeman.example.com:8443
  * Puppetmaster is running at port 8140
  The full log is at /var/log/foreman-installer/foreman-installer.log

2.2 Puppet Management

After installation, the Foreman installer will have set up a puppet master on the host, fully integrated with Foreman. First run the Puppet agent on the Foreman host which will send the first Puppet report to Foreman, automatically creating the host in Foreman’s database.

sudo puppet agent --test

In Foreman, click on the Hosts tab and your Foreman host should be visible in the list with an “O” status. This indicates its status is OK, with no changes made on the last Puppet run.

Downloading a Puppet module

Next, we’ll install a Puppet module for managing the NTP service from Puppet Forge to our “production” environment (the default):

sudo puppet module install puppetlabs/ntp

In Foreman, go to Configure > Classes and click Import from hostname (top right) to read the available Puppet classes from the puppet master and populate Foreman’s database. The “ntp” class will appear in the Puppet class list if installed correctly.

Using the Puppet module

Click on the “ntp” class in the list, change to the Smart Class Parameters tab and select the servers parameter on the left hand side. Tick the Override checkbox so Foreman manages the “servers” parameter of the class and change the default value if desired, before submitting the page.

• More info: Parameterized classes documentation
• Screencast: Parameterized class support in Foreman

Change back to the Hosts tab and click Edit on the Foreman host. On the Puppet Classes tab, expand the ntp module and click the + icon to add the ntp class to the host, then save the host.

Clicking the YAML button when back on the host page will show the ntp class and the servers parameter, as passed to Puppet via the ENC (external node classifier) interface. Re-run puppet agent --test on the Foreman host to see the NTP service automatically reconfigured by Puppet and the NTP module.

Adding more Puppet-managed hosts

Other hosts with Puppet agents installed can use this puppet master by setting server = foreman.example.com in puppet.conf. Sign their certificates in Foreman by going to Infrastructure > Smart Proxies > Certificates or using puppet cert list and puppet cert sign on the puppet master.

Puppet classes can be added to host groups in Foreman instead of individual hosts, enabling a standard configuration of many hosts simultaneously. Host groups are typically used to represent server roles.

3. Installing Foreman

There are several different methods of installing Foreman. The recommended way is with the puppet based Foreman Installer but you may also use your distribution’s package manager or install directly from source.

3.1 System Requirements

This sections outlines the system requirements for an installation of Foreman. This will cover the hardware requirements, OS requirements and firewall requirements. This includes variations for all supported database types.

3.1.1 Supported Platforms

The following operating systems are supported by the installer, have packages and are tested for deploying Foreman:

• Red Hat Enterprise Linux 7
  • Architectures: x86_64 only
  • EPEL is required
  • Enable the Optional repository/channel:
    • yum-config-manager --enable rhel-7-server-optional-rpms
    • check the above repositories because the command can silently fail when subscription does not provide it: yum repolist
  • Apply all SELinux-related errata.
• CentOS, Scientific Linux or Oracle Linux 7
  • Architectures: x86_64 only
  • EPEL is required
• Ubuntu 16.04 (Xenial)
  • Architectures: i386, amd64, armhf, aarch64
• Debian 9 (Stretch)
  • Architectures: i386, amd64, armhf

It is recommended to apply all OS updates if possible.

All platforms will require Puppet 4 or higher, which may be installed from OS or the Puppet Labs repositories.

Other operating systems will need to use alternative installation methods, such as from source.

The following operating systems are known to install successfully from Foreman:

• RHEL and derivatives (CentOS, Scientific Linux, Oracle Linux) 3+
• Fedora
• Ubuntu
• Debian
• OpenSUSE
• SLES
• CoreOS
• Solaris
• FreeBSD
• Juniper Junos
• Cisco NX-OS

3.1.2 Hardware Requirements

The hardware requirements for Foreman depend primarily on the number of requests that it will receive, which depends on the number of configuration management clients, web UI activity and other systems using the API.

The default installation when including Puppet Server will require:

• 4GB memory
• 2GB disk space

For a bare minimum installation with few clients and no Puppet master, the requirements are:

• 2GB memory
• 1GB disk space

Scaling notes

• The default installation when using Passenger will increase the number of running processes based on the number of simultaneous requests, up to a default maximum of six instances. Each instance will typically require 0.5-1GB of memory, depending on the number of configured plugins.
• When using the Puppet master, consult the requirements outlined in the Puppet System Requirements and Puppet Server System Requirements.
• Disk usage will increase as more data is stored in the database (PostgreSQL by default), mostly for facts and reports. See the reports cronjob configuration to change how they are expired.

3.1.3 Puppet Compatibility

Foreman integrates with Puppet and Facter in a few places, but generally using a recent, stable version will be fine. The exact versions of Puppet, Puppet Server and Facter that Foreman is compatible with are listed below.

Lines indicated with * require Parametrized_Classes_in_ENC in Foreman to be disabled.

Puppet Server compatibility

Puppet Server is the application for serving Puppet agents used by default in Puppet 4/5 AIO installations.

AIO installer compatibility

The Foreman installer supports both AIO and non-AIO configurations when using Puppet 4/5, switching behavior automatically based on the version of Puppet installed (usually during the first run when answers are stored).

When using an AIO installation of Puppet to run the installer, it will default to configuring the Puppet Server application, and when using a non-AIO version of Puppet, it defaults to a traditional Rack/Passenger configuration.

Facter compatibility

Foreman is known to be compatible with all Facter 1.x to 3.x releases.

Puppet Enterprise compatibility

The Foreman installer and packages are generally incompatible with Puppet Enterprise, however with some manual reconfiguration, individual Foreman components such as the smart proxy should work if needed (some further unsupported documentation can be found on the wiki). The installer in particular will conflict with a Puppet Enterprise installation. It is recommended that Foreman is installed using Puppet “open source”.

3.1.4 Browser Compatibility

Using the most recent version of a major browser is highly recommended, as Foreman and the frameworks it uses offer limited support for older versions.

The recommended requirements are as follows for major browsers:

• Google Chrome 54 or higher
• Microsoft Edge
• Microsoft Internet Explorer 10 or higher
• Mozilla Firefox 49 or higher

Other browsers may work unpredictably.

3.1.5 Firewall Configuration

Protect your Foreman environment by blocking all unnecessary and unused ports.

Ports indicated with * are running by default on a Foreman all-in-one installation and should be open.

3.2 Foreman Installer

The Foreman installer is a collection of Puppet modules that installs everything required for a full working Foreman setup. It uses native OS packaging (e.g. RPM and .deb packages) and adds necessary configuration for the complete installation.

Components include the Foreman web UI, Smart Proxy, Passenger (for the puppet master and Foreman itself), and optionally TFTP, DNS and DHCP servers. It is configurable and the Puppet modules can be read or run in “no-op” mode to see what changes it will make.

It’s strongly recommended to use the installer instead of only installing packages, as the installer uses OS packages and it saves a lot of time otherwise spent replicating configuration by hand.

By default it will configure:

• Apache HTTP with SSL (using a Puppet-signed certificate)
• Foreman running under mod_passenger
• Smart Proxy configured for Puppet, TFTP and SSL
• Puppet master running under mod_passenger
• Puppet agent configured
• TFTP server (under xinetd on Red Hat platforms)

Other modules can be enabled, which will also configure:

• ISC DHCP server
• BIND DNS server

It’s recommended to run the installer on a fresh and clean single-purpose system, since the configurations of the aforementioned components is (at least partially) overwritten by the installer.

3.2.1 Installation

Downloading the installer

Follow the instructions in section 2.1 Quickstart installation

Running the installer

The installation run is non-interactive, but the configuration can be customized by supplying any of the options listed in foreman-installer --help, or by running foreman-installer -i for interactive mode. More examples are given in the Installation Options section. Adding -v will disable the progress bar and display all changes, while --noop will run without making any changes. To run the installer, execute:

foreman-installer

After it completes, the installer will print some details about where to find Foreman and the Smart Proxy and Puppet master if they were installed along Foreman. Output should be similar to this:

  * Foreman is running at https://theforeman.example.com
      Initial credentials are admin / 3ekw5xtyXCoXxS29
  * Foreman Proxy is running at https://theforeman.example.com:8443
  * Puppetmaster is running at port 8140
  * The full log is at /var/log/foreman-installer/foreman-installer.log

3.2.2 Installer Options

The installer is a collection of Puppet modules, which have a large number of parameters available to customize the configuration. Parameters can be set by running foreman-installer with arguments, e.g. --foreman-db-type, changing settings in interactive mode or by setting up an answers file.

The precedence for settings is for those set by arguments to foreman-installer or interactive mode, then the answers file, then the Puppet manifest defaults.

foreman-installer arguments

Every parameter available in the installer can be set using command line arguments to foreman-installer. Run foreman-installer --help for most options, or foreman-installer --full-help for a list of every available option.

When running the installer, all arguments passed on the command line will be persisted by default to /etc/foreman-installer/scenarios.d/foreman-answers.yaml and used automatically on subsequent runs, without needing to specify those arguments again. This persistence can be disabled with the -b option.

Interactive mode

The installer also provides a text driven interface to customize configuration parameters, and can be run by executing:

foreman-installer -i

Plugins and compute resources

The installer contains a number of high level modules (e.g. “foreman”, “puppet”) and additionally a number of smaller modules for additional functionality, such as plugins and compute resource support. These can be added with the “–enable” switches, or the default options can be disabled with “–no-enable” switches.

More information about compute resources can be found in the Compute Resources section and plugins in the Plugins section.

Available options

Note: When you can specify a hash, you need to specify each option individually to foreman-installer.

For example the hash {show_diff => true, stringify_facts => false} for --puppet-server-additional-settings becomes --puppet-server-additional-settings=show_diff:true --puppet-server-additional-settings=stringify_facts:false.

Answers file

The answers file describes the classes that will be applied to the host to install Foreman, along with their parameters. The foreman-installer package stores it at /etc/foreman-installer/scenarios.d/foreman-answers.yaml. By default, the all-in-one setup will include Foreman, a puppetmaster, Puppet agent, and the Smart Proxy:

---
foreman:
  custom_repo: true
foreman_proxy:
  custom_repo: true
puppet:
  server: true

Other examples are given in the next section, or /usr/share/foreman-installer/README.md.

Advanced module configuration

Additional configuration options can be given in /etc/foreman-installer/custom-hiera.yaml for some of the Puppet modules that are used internally by Foreman installer. The contents of this file will be passed to Hiera during the Foreman installer execution so can set class parameters for other modules such as apache, mysql, and postgresql.

For example, the TraceEnable option may be controlled by disabling the apache::trace_enable parameter in this file:

apache::trace_enable: Off

Please note that the parameters used by these modules may change between versions of Foreman, so it’s important to check the versions in use and the appropriate module documentation or source code when editing this configuration file. Modifications cannot be supported or migrated by Foreman.

3.2.3 Installation Scenarios

The Foreman installer can accommodate more complex, multi-host setups when supplied with appropriate parameters.

Using an external database server

Per default foreman-installer will install a PostgreSQL database server onto the Foreman host and create its database. An external MySQL or PostgreSQL database server with an already created database can be used with the following arguments:

foreman-installer \
  --foreman-db-type=mysql \
  --foreman-db-manage=false \
  --foreman-db-host=dbserver.example.com \
  --foreman-db-database=dbname \
  --foreman-db-username=dbuser \
  --foreman-db-password=dbpassword

As a post-installation step, to populate the database correctly, run:

foreman-rake db:migrate
foreman-rake db:seed
foreman-rake apipie:cache:index

Setting up Foreman with external Puppet masters

Using the scenarios outlined below, a simple scale-out setup can be created as follows:

1. On the Foreman host, run a complete foreman-installer all-in-one installation to provide Foreman, a Puppet master and smart proxy. This will be the Puppet CA.

For each Puppet master:

1. Generate a new certificate following the steps in the SSL CA section and transfer it to the new Puppet master host
2. Run the standalone Puppet master installation as detailed below

Each Puppet master will register with Foreman as a smart proxy, while the instance running on the Foreman host itself will act as a central Puppet CA. These can be selected while adding new hosts or host groups.

SSL certificate authority setup

The scenarios below assume a single Puppet CA (certificate authority) for all hosts, which is also used for Foreman and smart proxy communications, though more complex deployments are possible. This might be the central Foreman host, or a particular Puppet master.

Other systems require certificates to be generated on the central Puppet CA host, then distributed to them before running foreman-installer (else it may generate a second CA). To prepare these, on the host acting as Puppet CA, run:

# puppet cert generate new-puppetmaster.example.com
Notice: new-puppetmaster.example.com has a waiting certificate request
Notice: Signed certificate request for new-puppetmaster.example.com
Notice: Removing file Puppet::SSL::CertificateRequest new-puppetmaster.example.com at '/var/lib/puppet/ssl/ca/requests/new-puppetmaster.example.com.pem'
Notice: Removing file Puppet::SSL::CertificateRequest new-puppetmaster.example.com at '/var/lib/puppet/ssl/certificate_requests/new-puppetmaster.example.com.pem'

# ls /var/lib/puppet/ssl/*/new-puppetmaster.example.com.pem
/var/lib/puppet/ssl/certs/new-puppetmaster.example.com.pem
/var/lib/puppet/ssl/private_keys/new-puppetmaster.example.com.pem
/var/lib/puppet/ssl/public_keys/new-puppetmaster.example.com.pem

Transfer the following files to the same paths on the new host:

• /var/lib/puppet/ssl/certs/ca.pem
• /var/lib/puppet/ssl/certs/new-puppetmaster.example.com.pem
• /var/lib/puppet/ssl/private_keys/new-puppetmaster.example.com.pem

Under a Puppet 4 AIO installation, substitute above paths with /etc/puppetlabs/puppet/ssl/.

This provides the new host a certificate in the same authority, but doesn’t make it a CA itself. Certificates will continue to be generated on the central Puppet CA host.

Standalone Puppet master

A standalone Puppet master can be configured along with a smart proxy installation, enabling the Puppet infrastructure to be scaled out. A certificate should be generated and copied to the host first to make it part of the same CA, else a new Puppet CA will be generated.

Command line arguments:

foreman-installer \
  --no-enable-foreman \
  --no-enable-foreman-cli \
  --no-enable-foreman-plugin-bootdisk \
  --no-enable-foreman-plugin-setup \
  --enable-puppet \
  --puppet-server-ca=false \
  --puppet-server-foreman-url=https://foreman.example.com \
  --enable-foreman-proxy \
  --foreman-proxy-puppetca=false \
  --foreman-proxy-tftp=false \
  --foreman-proxy-foreman-base-url=https://foreman.example.com \
  --foreman-proxy-trusted-hosts=foreman.example.com \
  --foreman-proxy-oauth-consumer-key=<key here> \
  --foreman-proxy-oauth-consumer-secret=<secret here>

Fill in the OAuth consumer key and secret values from your Foreman instance, retrieve them from Administer > Settings > Authentication, and set the Foreman URLs appropriately. These allow the smart proxy to register automatically with the Foreman instance, or disable with --foreman-proxy-register-in-foreman=false.

PuppetDB integration

An existing PuppetDB server can be integrated into a Puppet master by adding:

foreman-installer \
  [...]
  --puppet-server-puppetdb-host=puppetdb.example.com \
  --puppet-server-reports=foreman,puppetdb \
  --puppet-server-storeconfigs-backend=puppetdb

Be aware that foreman-installer does not setup the PuppetDB server itself. Starting with 1.13 of foreman-installer, only setups using Puppet Labs Puppet 4 AIO packages are supported for PuppetDB integration using these parameters.

Foreman server without the Puppet master

The default “all-in-one” Foreman installation includes a Puppet master, but this can be disabled. Foreman by default uses Puppet’s SSL certificates however, so a certificate must be generated and copied to the host first, or all SSL communications need to be disabled.

Command line arguments:

foreman-installer \
  --puppet-server=false \
  --foreman-proxy-puppet=false \
  --foreman-proxy-puppetca=false

This will still configure the Puppet agent, but this too can be disabled with --no-enable-puppet to disable the whole Puppet module.

Smart proxy for DNS, DHCP etc.

The smart proxy allows management of various network services, such as DNS, DHCP and TFTP. The installer can set up a basic smart proxy service ready to be configured, or it can install and configure BIND or ISC DHCP ready to go. A certificate should be generated and copied to the host first so Foreman can contact the proxy server.

Command line arguments for a basic smart proxy installation:

foreman-installer \
  --no-enable-foreman \
  --no-enable-foreman-cli \
  --no-enable-foreman-plugin-bootdisk \
  --no-enable-foreman-plugin-setup \
  --no-enable-puppet \
  --enable-foreman-proxy \
  --foreman-proxy-tftp=false \
  --foreman-proxy-foreman-base-url=https://foreman.example.com \
  --foreman-proxy-trusted-hosts=foreman.example.com \
  --foreman-proxy-oauth-consumer-key=<key here> \
  --foreman-proxy-oauth-consumer-secret=<secret here>

Fill in the OAuth consumer key and secret values from your Foreman instance, retrieve them from Administer > Settings > Authentication, and set the Foreman URL appropriately. These allow the smart proxy to register automatically with the Foreman instance, or disable with --foreman-proxy-register-in-foreman=false.

Command line arguments for a smart proxy configured with just BIND, setting DNS forwarders and overriding the default forward and reverse DNS zones:

foreman-installer \
  --no-enable-foreman \
  --no-enable-foreman-cli \
  --no-enable-foreman-plugin-bootdisk \
  --no-enable-foreman-plugin-setup \
  --no-enable-puppet \
  --enable-foreman-proxy \
  --foreman-proxy-tftp=false \
  --foreman-proxy-puppet=false \
  --foreman-proxy-puppetca=false \
  --foreman-proxy-dns=true \
  --foreman-proxy-dns-interface=eth0 \
  --foreman-proxy-dns-zone=example.com \
  --foreman-proxy-dns-reverse=0.0.10.in-addr.arpa \
  --foreman-proxy-dns-forwarders=8.8.8.8 \
  --foreman-proxy-dns-forwarders=8.8.4.4 \
  --foreman-proxy-foreman-base-url=https://foreman.example.com \
  --foreman-proxy-trusted-hosts=foreman.example.com \
  --foreman-proxy-oauth-consumer-key=<key here> \
  --foreman-proxy-oauth-consumer-secret=<secret here>

Ensure the dns-interface argument is updated with the correct network interface name for the DNS server to listen on. After configuration, make sure to create Subnet in Foreman under Infrastructure > Subnets for the particular Smart Proxy which registers automatically.

Command line arguments for a smart proxy configured with just ISC DHCP and a single DHCP subnet:

foreman-installer \
  --no-enable-foreman \
  --no-enable-foreman-cli \
  --no-enable-foreman-plugin-bootdisk \
  --no-enable-foreman-plugin-setup \
  --no-enable-puppet \
  --enable-foreman-proxy \
  --foreman-proxy-puppet=false \
  --foreman-proxy-puppetca=false \
  --foreman-proxy-tftp=false \
  --foreman-proxy-dhcp=true \
  --foreman-proxy-dhcp-interface=eth0 \
  --foreman-proxy-dhcp-gateway=10.0.0.1 \
  --foreman-proxy-dhcp-range="10.0.0.100 10.0.0.200" \
  --foreman-proxy-dhcp-nameservers="10.0.1.2,10.0.1.3" \
  --foreman-proxy-foreman-base-url=https://foreman.example.com \
  --foreman-proxy-trusted-hosts=foreman.example.com \
  --foreman-proxy-oauth-consumer-key=<key here> \
  --foreman-proxy-oauth-consumer-secret=<secret here>

Also ensure here that the dhcp-interface argument is updated for the interface to run DHCP on. After configuration, make sure to create a new Subnet (or import from existing) in the Foreman interface.

While it is possible to define the same DHCP range in Foreman, it’s usually good practice to select a range from outside the pool defined in the installer, but still in the subnet. For the example above, it is recommended to define the DHCP range from 10.0.0.1 to 10.0.0.99 in the Foreman UI which gives the following IP address distribution:

• 10.0.0.1 - 10.0.0.99 - addresses reserved during bare-metal provisioning by Foreman
• 10.0.0.100 - 10.0.200 - addresses for dynamic clients in the subnet (discovered hosts, unmanaged hosts)

3.3 Install From Packages

Packages are available for Red Hat and Debian-based distributions. This will install a standalone Foreman service running under WEBrick, which has limited scalability.

The Puppet-based Foreman installer is recommended for most environments, instead of installing only the packages as it will perform full configuration too.

3.3.1 RPM Packages

Foreman is packaged for the following RPM based distributions:

• RHEL and derivatives, version 7

For most users, it’s highly recommended to use the installer as the packages only provide the software and a standalone Foreman service. The installer installs these packages, then additionally configures Foreman to run under Apache and Passenger with PostgreSQL, plus can configure a complete Puppet setup integrated with Foreman.

Pre-requisites: EPEL

All RHEL and derivatives must be subscribed to EPEL to provide additional dependencies. Install epel-release from here to automatically configure it.

Pre-requisites: Software collections

All RHEL and derivatives require software collections from the CentOS SCLorg Special Interest Group (SIG). Foreman uses the rh-ruby24 collection, which is from RHSCL.

Generally we recommend all users to use the CentOS SCLorg SIG’s builds of both, which can be configured by installing the foreman-release-scl package from our repositories (which in turn installs the centos-release-scl* packages). The Foreman installer will use this package by default.

It is also possible to obtain rh-ruby24 from other sources below. These sources are noted only for completeness:

1. CentOS SCLo SIG: yum install centos-release-scl to configure from CentOS Extras, or sclo on mirror.centos.org (docs)
2. Oracle Linux: Software Collection Library documentation
3. RHEL: customers can access RHSCL as a separate repository or child channel (rhel-server-rhscl-7-rpms)
4. Scientific Linux: softwarecollections on ftp.scientificlinux.org

Pre-requisites: Optional repo (RHEL only)

RHEL 7 hosts must also be subscribed to the RHEL 7 Optional repository or child channel in RHN.

To enable the optional repository on a RHEL 7 system using subscription-manager, run:

yum-config-manager --enable rhel-7-server-optional-rpms

Pre-requisites: Puppet

It’s recommended to configure the Puppet Labs repositories to obtain the latest version of Puppet available, instead of the version on EPEL. Either the 3.x open source repository or the Puppet Collection (PC1) repository may be configured:

• Using Puppet Collections: Yum-based Systems (Puppet 4)
• Pre-4.0 Open Source Repositories: Yum-based Systems (Puppet 3)

Available repositories

Four main repos are provided at http://yum.theforeman.org:

• /releases/latest or /releases/VERSION (e.g. /releases/1.17) carries the stable releases and updates of Foreman and its dependencies.
• /rc carries release candidates only in the few weeks prior to a release. Ensure after a release is made that you use the main releases repo instead.
• /nightly carries the latest development builds and as such, may be unstable or occasionally broken.
• /plugins or /releases/VERSION (e.g. /releases/1.17) carries the stable plugin releases.

Under each repo are directories for each distribution, then each architecture.

Release packages

To set up the repository, foreman-release packages are provided which add a repo definition to /etc/yum.repos.d. Install the appropriate release RPM from these lists:

yum localinstall https://yum.theforeman.org/releases/1.17/el7/x86_64/foreman-release.rpm

For release candidate or nightly RPMs, change the URL as appropriate based on the above list of available repositories.

Signing

Release and release candidate packages are signed by the per-release key listed on Security. Nightly packages are not signed.

Available packages

Install foreman and other foreman-* packages to add functionality:

foreman               Foreman server
foreman-proxy         Foreman Smart Proxy
foreman-compute       Additional compute resource support for plugins
foreman-debug         Log and debug collection tools
foreman-ec2           Amazon EC2 provisioning support
foreman-gce           Google Compute Engine provisioning support
foreman-libvirt       libvirt provisioning support
foreman-openstack     OpenStack Nova provisioning support
foreman-ovirt         oVirt/RHEV provisioning support
foreman-rackspace     Rackspace Cloud provisioning support
foreman-vmware        VMware provisioning support
foreman-cli           Foreman CLI utility
foreman-console       Console additions
foreman-selinux       SELinux targeted policy
foreman-mysql2        MySQL database support
foreman-postgresql    PostgreSQL database support
foreman-sqlite        SQLite database support

Setup

1. Configure by editing /etc/foreman/settings.yaml and /etc/foreman/database.yml
2. After changing the database, migrate it: sudo -u foreman /usr/share/foreman/extras/dbmigrate
3. Start the foreman service or set up passenger: service foreman start

Upgrade

See upgrade instructions

3.3.2 Software Collections

The RPMs use a packaging technique called Software Collections, or SCL. This provides Ruby and all dependencies required to run Foreman separately from the version of Ruby provided by the distribution.

A “tfm” (The Foreman) collection is provided by the Foreman project which ships all dependencies for the main Foreman application, and depends on the “tfm-ror51” and “rh-ruby24” collections which provide newer versions of Ruby and Ruby on Rails. Packages that are part of these collections will have “tfm-“, “tfm-ror51-“ and “rh-ruby24-“ prefixes, allowing them to be easily identified, and will install entirely underneath /opt/theforeman and /opt/rh.

The system Ruby version is left alone and will still be used for packages provided both by the distribution, and other third parties who target the system Ruby (e.g. Puppet packages).

Foreman currently uses SCL only on RHEL and EL clones where a newer version of Ruby is desired. Fedora only uses the distro version of Ruby.

In order to run rake commands for Foreman, or scripts that run in the same environment, tfm-rake and tfm-ruby wrappers are provided as alternatives for the usual rake or ruby. In order to run a series of commands or a script directly within the software collection, scl enable tfm 'other command' can be used. It is also possible to run scl enable tfm bash to get a shell within the context of the SCL. Foreman rake tasks should be run with foreman-rake, which automates using Foreman’s rake environment, changes user etc.

More general information about software collections is available from these links:

• RHEL packaging: Software Collections (SCLs)
• Fedora Contributor Documentation: Software Collections Guide

Passenger under the SCL

When running Foreman under Passenger (the default installer setup), a specific configuration is needed for SCL (on EL), since Foreman operates under the SCL Ruby and other apps such as the puppetmaster will use the system Ruby. Passenger 4 is shipped in the Foreman repos as it can be configured with separate Ruby binaries per VirtualHost. The full configuration is described below.

The following packages must be installed:

• mod_passenger
• tfm-rubygem-passenger
• tfm-rubygem-passenger-native
• tfm-rubygem-passenger-native-libs
• rubygem-passenger
• rubygem-passenger-native
• rubygem-passenger-native-libs

Ensure all version numbers match and are at least 4.0. mod_passenger provides the Apache module, while there are two copies of the Ruby components, one for the SCL Ruby (tfm-rubygem-*) and one for the system Ruby (rubygem-*).

The /etc/httpd/conf.d/passenger.conf file is supplied by mod_passenger and should contain:

LoadModule passenger_module modules/mod_passenger.so
<IfModule mod_passenger.c>
   PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-4.0.5
   PassengerRuby /usr/bin/ruby
</IfModule>

Check for .rpmsave or .rpmnew config backup files if this isn’t correct. Note that this refers to the system Ruby paths by default, allowing everything except Foreman (i.e. the puppetmaster) to run under the system Ruby.

Next, the Foreman config file at /etc/httpd/conf.d/foreman.conf must contain this entry in both HTTP and HTTPS VirtualHost sections:

PassengerRuby /usr/bin/tfm-ruby

Ensure both RailsAutoDetect and RakeAutoDetect config entries are removed from foreman.conf and puppet.conf when using Passenger 4, since they have been deprecated.

When successfully configured, two Passenger RackApp processes will be visible and by inspecting the open files, the Ruby version being loaded can be confirmed:

# ps auxww | grep RackApp
foreman  16627  0.1 15.4 466980 157196 ?       Sl   07:35   0:09 Passenger RackApp: /usr/share/foreman
puppet   16697  0.8 11.3 253080 115720 ?       Sl   07:35   1:13 Passenger RackApp: /etc/puppet/rack
# lsof -p 16697 | grep libruby
ruby    16697 puppet  mem    REG              253,0   951224  272286 /usr/lib64/libruby.so.1.8.7
# lsof -p 16627 | grep libruby
ruby    16627 foreman  mem    REG              253,0  2041096  171190 /opt/rh/ruby193/root/usr/lib64/libruby.so.1.9.1

3.3.3 Debian Packages

The Foreman packages should work on the following Debian-based Linux distributions:

Distributions

• Debian Linux 9.0 (Stretch), i386, amd64, armhf
• Ubuntu Linux 16.04 LTS (Xenial Xerus), i386, amd64, armhf, aarch64
• Ubuntu Linux 14.04 LTS (Trusty Tahr), i386, amd64

If you encounter any errors during the installation, please file a bug report!

Apt Configuration

Add one of the following lines to your /etc/apt/sources.list (alternatively in a separate file in /etc/apt/sources.list.d/foreman.list):

# Stable packages

# Debian Jessie
deb http://deb.theforeman.org/ stretch 1.17
# Ubuntu 16.04 Trusty
deb http://deb.theforeman.org/ xenial 1.17
# Ubuntu 14.04 Trusty
deb http://deb.theforeman.org/ trusty 1.17

# Nightly builds. Beware: HERE BE DRAGONS

# Debian Jessie
deb http://deb.theforeman.org/ stretch nightly
# Ubuntu 16.04 Precise
deb http://deb.theforeman.org/ xenial nightly
# Ubuntu 14.04 Precise
deb http://deb.theforeman.org/ trusty nightly

You may also want some plugins from the plugin repo (required for the Foreman Installer):

# Plugins compatible with Stable
deb http://deb.theforeman.org/ plugins 1.17
# Plugins compatible with Nightly
deb http://deb.theforeman.org/ plugins nightly

The public key for secure APT can be downloaded here

You can add this key with

apt-key add pubkey.gpg

or combine downloading and registering:

wget -q https://deb.theforeman.org/pubkey.gpg -O- | apt-key add -

The key fingerprint is

AE0A F310 E2EA 96B6 B6F4 BD72 6F86 00B9 5632 78F6
Foreman Automatic Signing Key (2016) <packages@theforeman.org>

Remember to update your package lists!

apt-get update

Install packages

The packages are now split by gem dependencies - there are 11 packages to choose from. These are:

Main package:

• foreman

Database gems - you need at least one of these:

• foreman-sqlite3
• foreman-mysql2
• foreman-pgsql

Optional functionality:

• foreman-console
• foreman-compute
• foreman-debug
• foreman-ec2
• foreman-gce
• foreman-libvirt
• foreman-openstack
• foreman-ovirt
• foreman-rackspace
• foreman-test
• foreman-vmware

Command line interface:

• ruby-hammer-cli
• ruby-hammer-cli-foreman

Installation instructions are:

# Install packages  (adjust additional packages as needed)
apt-get install foreman foreman-sqlite3 foreman-libvirt

# Copy sample db config to /etc
cp /usr/share/foreman/config/database.yml.example /etc/foreman/database.yml

# Review settings and DB config
vi /etc/foreman/settings.yaml /etc/foreman/database.yml

# Perform initial DB setup
foreman-rake db:migrate
foreman-rake db:seed

The packages should auto-run db:migrate and db:seed if /etc/foreman/database.yml exists. So the initial DB setup is only needed during first install, upgrades should just work.

Upgrade

See upgrade instructions

3.4 Install From Source

Installing the latest development code: Foreman uses Bundler to install dependencies and get up and running. This is the preferred way to get Foreman if you want to benefit from the latest improvements. By using the git repository you can also upgrade more easily. You will need to have Bundler installed manually for now (check your distribution repositories, or install it via rubygems).

Foreman will run with the following requirements (aside from rubygem dependencies):

• Ruby 2.0, 2.1, 2.2, 2.3 or 2.4
• NodeJS 4.0 or newer
• NPM 3.0 or newer

You will want to make sure that you have one of the mysql-devel, postgresql-devel, and sqlite-devel libraries installed so the database gems can install properly. Also, you would also need gcc, ruby-devel, libxml-devel, libxslt-devel, libvirt-devel, nodejs, and npm packages.

For RHEL6 or clones, you can issue the following commands to install all required packages:

yum groupinstall "Development Tools" "Development Libraries"
yum -y install gcc-c++ git ruby ruby-devel rubygems \
    libvirt-devel mysql-devel postgresql-devel openssl-devel \
    libxml2-devel sqlite-devel libxslt-devel zlib-devel \
    readline-devel tar nodejs npm

Additionally, it is important that config/database.yml is set to use the correct database in the “production” block. Rails (and subsequently Foreman) will use these connection settings under “production” to manage the database it uses and setup the necessary schema.

git clone https://github.com/theforeman/foreman.git -b develop
cd foreman
cp config/settings.yaml.example config/settings.yaml
cp config/database.yml.example config/database.yml
gem install bundler
# if you don't want to use therubyracer as JavaScript engine, you can use
# --without therubyracer, but make sure to have another supported JS engine installed, i.e. nodejs
#
# depending on database configured in yml file you can skip others
# (we are assuming sqlite to be configured)
bundle install --without mysql2 pg test --path vendor # or postgresql
npm install
# set up database schema, precompile assets and locales
RAILS_ENV=production bundle exec rake db:migrate
RAILS_ENV=production bundle exec rake db:seed assets:precompile locale:pack webpack:compile

The db:seed step will print out the default admin password, record this in order to log in later.

In order to run Foreman you can use the following command inside your git repository:

./bin/rails s -e production

Latest stable version

to get latest “stable” version do:

git clone https://github.com/theforeman/foreman.git -b 1.17-stable

CLI (Hammer)

To install hammer from git checkouts, you will just need rake installed on your system. Clone and install CLI core

$ git clone https://github.com/theforeman/hammer-cli.git
$ cd hammer-cli
$ rake install
$ cd ..

and clone plugin with foreman commands

$ git clone https://github.com/theforeman/hammer-cli-foreman.git
$ cd hammer-cli-foreman
$ rake install
$ cd ..

You can install other hammer plugins via any of the methods mentioned above.

3.5 Configuration

The following sections detail the configuration steps required to get Foreman working in your environment. Lets get started!

3.5.1 Initial Setup

Configuration

Foreman configuration is managed from two places; a configuration file config/settings.yaml and from the SETTINGS/Foreman Settings page. A full description of the configuration options is given at foreman_configuration

Database

Foreman requires a database of its own to operate - database sharing is unsupported. By default, the installer uses PostgreSQL, while a package or source installation will use SQLite. If you want to use other database (e.g. MySQL) please modify the configuration file under config/database.yml.

In all cases, please use the production settings.

to initialize the database schema and content, run:

foreman-rake db:migrate
foreman-rake db:seed

For more information please see the database configuration page here

Import Data from Puppet

At this point, you might want to go through the [[FAQ]] to see how can you import your data into Foreman.

Start The Web Server

if you installed via rpm, just start the foreman service, or start the builtin web server by typing:

RAILS_ENV=production rails server

and point your browser to http://foreman:3000

If you would like to keep the server running, its recommend to setup passenger or use the RPM. Example usage with passenger can be found on GitHub.

Getting your Puppet Reports into Foreman

Read Puppet_Reports to learn how to get your nodes to report to Foreman.

3.5.2 Configuration Options

Configuration is broken into two parts. The /etc/foreman/settings.yaml file and the Administer > Settings page. The configuration file contains a few low-level options that need to be set before Foreman starts but the majority of Foreman customization is managed from within the web interface on the Settings page.

The configuration file can also override those settings specified in the web interface. Any settings added in the config file that are available in the web interface will be made read-only.

The config/settings.yaml file

YAML start

The first non-comment line of this file must be three dashes.

---

login

This boolean option configures whether Foreman requires users to to login. If it is set then each user will be expected to authenticate themselves and all operations will occur, and be audited, under their identity. When this option is false then all activity will be executed under the admin account.

:login: true

require_ssl

This boolean option configures whether Foreman insists on using only https/ssl encrypted communication channels in the web interface. This does not configure the channels used to contact the smart-proxies. Note that certain operations will still accept a http connection even if this is set, for example, the downloading of a finish script.

:require_ssl: true

unattended

This boolean option configures whether Foreman will act as a simple node classifier for puppet, or support the full spectrum of operations required for managing a host’s lifecycle. When set to true then foreman will provide full host building facilities for various operating systems.

:unattended: true

support_jsonp

This boolean options configures whether Foreman will provide support for the JavaScript object notation with padding. When set to true then Foreman will allow to pass a callback parameter to the API calls.

:support_jsonp: false

logging

This options contains a hash of parameters that override the current logging configuration. It supports any of the options that are in logging.yaml (see below), but most usually it’s used to change the log level for debugging.

:logging:
  :level: debug

loggers

This options contains a hash of config options for specific loggers, which cover parts of Foreman functionality. It’s usually used to enable additional types of logging.

Available loggers are:

• app - web requests and all general application logs (default: true)
• audit - additional fact import statistics, numbers of facts added/updated/removed (default: true)
• ldap - high level LDAP queries (e.g. find users in group) and LDAP operations performed (default: false)
• permissions - evaluation of user roles, filters and permissions when loading pages (default: false)
• sql - SQL queries made through Rails ActiveRecord, only debug (default: false)

Uncomment or add a :loggers block to enable or disable loggers:

:loggers:
  :app:
    :enabled: true
  :ldap:
    :enabled: false
  :permissions:
    :enabled: false
  :sql:
    :enabled: false

Some plugins may add their own loggers. See the configuration files in /etc/foreman/plugins/ which should list possibilities and enable them there.

The ‘logging.yaml’ file

This settings file can be found at /etc/foreman/logging.yaml, or config/logging.yaml on a source installation. It controls the default logging locations, formatting and log levels per Rails environment.

In a normal installation, only the “production” environment is relevant - development and test are only used in source installations. The file has comments for the most common configuration options, which can be changed here or overridden from the logging directive in the main settings.yaml config file (see above).

The ‘Administer/Settings’ page

access_unattended_without_build

When enabled, unattended URLs used to fetch templates for individual hosts will be accessible irrespective of the host build state. When disabled, the unattended URLs will only function in build mode to prevent accidental rebuilding etc. Default: false

administrator

When Foreman needs to mail the administrator then this is the email address that it will contact. The domain is determined from Facter, else it will default to the “:domain” setting in /etc/foreman/settings.yaml. Default: root@<your domain>.

always_show_configuration_status

When reporting the configuration status of hosts, usually only hosts with outdated reports, or a Puppet proxy/master set and no reports will be considered out of sync. When true, all hosts will be considered out of sync until a report is received. This setting should be enabled in environments where Foreman is used for reporting without smart proxies. Default: false

authorize_login_delegation

mod_proxy and other load balancers will set a REMOTE_USER environment variable. If this is true , your users will be able to login through an external service and Foreman requests will be authenticated using this REMOTE_USER variable. Default: false

authorize_login_delegation_api

Same as above, but this setting allows REMOTE_USER authentication for API calls as well. Default: false

authorize_login_delegation_auth_source_autocreate

If you have authorize_login_delegation set, new users can be autocreated through your external authentication mechanism by changing this to the name of the Auth Source you want to use to auto create users. Default: ‘’

bmc_credentials_accessible

By default passwords stored on BMC network interfaces will be visible to other users who can view the host via the ENC YAML preview and accessible through templates, for the purposes of configuring BMC interfaces automatically.

When set to false, all BMC passwords will be redacted in template and ENC output, preventing both users from viewing the passwords directly and also from configuration (or access) in Puppet and other config management tools using the ENC interface. Foreman will continue to use the stored password for BMC power operations.

Note that setting this to false also this requires that safemode_render be enabled, else it could be bypassed.

Default: true

clean_up_failed_deployment

During host provisioning onto a compute resource using images or templates and a finish script, this setting controls the behavior of Foreman when the script fails. When true, the new host and virtual machine (on the compute resource) will be deleted if the script fails. When false, the host and virtual machine are left running so the script can be debugged. Default: true

create_new_host_when_facts_are_uploaded

When facts are received from Puppet or other configuration management systems, a corresponding host will be created in Foreman if the certname or hostname is unknown. When false, this behavior is disabled and facts will be discarded from unknown hosts. Default: true See also: create_new_host_when_report_is_uploaded

create_new_host_when_report_is_uploaded

If a report is received from Puppet or other configuration management systems, a corresponding host will be created in Foreman if the hostname is unknown. When false, this behavior is disabled and reports will be discarded from unknown hosts. Default: true See also: create_new_host_when_facts_are_uploaded

db_pending_migration

When you upgrade Foreman using foreman-installer, the database may migrate its data model to the new version. If this is true, the next run of the installer will run these migrations. If it is false, the database will remain untouched.

Default: true

db_pending_seed

When you upgrade Foreman using foreman-installer, the new version may contain some seed data such as operating systems, provisioning templates, roles and more. It will also update any previous seeded data. If this is true, the next run of the installer will seed this data. If it is false, the database will not get this seeded data.

Default: true

default_location

The name of an location that hosts uploading facts into Foreman will be assigned to if they are new or missing an location. This can be used when hosts are created through fact uploads to ensure they’re assigned to the correct location to prevent resource mismatches. For inherited location, the fact should use slash-delimited names, e.g. “USA/New York”. Default: ‘’

default_organization

The name of an organization that hosts uploading facts into Foreman will be assigned to if they are new or missing an organization. This can be used when hosts are created through fact uploads to ensure they’re assigned to the correct organization to prevent resource mismatches. For inherited organization, the fact should use slash-delimited names, e.g. “ACME Inc/Engineering”. Default: ‘’

default_puppet_environment

When Foreman receives a fact upload from a machine that it has not previously come across it will create a host in its database. If the facts from that host did not contain information about the puppet environment then it will assign the default_puppet_environment environment to this host. Default: production

Default_variables_Lookup_Path

A Smart-variable’s match criteria are evaluated in a specific order and if this search order is not provided then Default_variables_Lookup_Path is used. Default: [“fqdn”, “hostgroup”, “os”, “domain”]

delivery_method

The method for sending emails from the Foreman instance, either sendmail (running the command set by sendmail_location) to send mail via the configured local MTA, or smtp for direct connection to an outbound SMTP server (given by settings with the smtp prefix). Default: sendmail

dns_conflict_timeout

When updating a host and DNS conflict detection is performed, each lookup for A and PTR records will be limited to this time in seconds. Default: 3 (seconds) See also: query_local_nameservers

email_reply_address

The return address applied to outgoing emails. Default: Foreman-noreply@<your domain>

email_subject_prefix

The subject line prefix for any emails sent by Foreman. Default: [foreman]

enc_environment

When this is true, Foreman will send the puppet environment in the ENC yaml output. This is meant to fix conflicts between a node’s puppet.conf environment and the environment set in Foreman. On Puppet 3+, agents will take the environment sent by the ENC. When false, the ENC yaml will not contain the environment, the node will not update its environment and use the one at puppet.conf. Default: true

Enable_Smart_Variables_in_ENC

Whether Smart-variables should be included in the yaml node information provided to puppet. Default: true

entries_per_page

The number of entries that will be shown in the web interface for list operations. Default: 20

fix_db_cache

Foreman maintains a cache of permissions and roles. If this is set to true, Foreman will recreate this cache on the next run. Default: false

foreman_url

Emails may contain embedded references to Foreman’s web interface. This option allows the URL prefix to be configured. The FQDN is determined from Facter, else it will default to the “:fqdn” setting in /etc/foreman/settings.yaml. Default: https://FQDN/ or http://FQDN/ (depending on require_ssl) See also: unattended_url

global_(PXELinux/PXEGrub/PXEGrub2)

Default PXELinux/PXEGrub/PXEGrub2 template. This template gets deployed to all configured TFTP servers. For example, this template can be used to make new hosts in a network boot into Foreman Discovery.

• Default: none

host_group_matchers_inheritance

Matchers used in smart variables or class parameters to match host groups can be inherited by children of those matching host groups too (e.g. a matcher for hostgroup=Base will also apply to Base/Web). Set this to false to make matchers only match a particular hostgroup and not its children. Default: true

host_power_status

Controls whether the power status of hosts is shown on the hosts list, which may lead to decreased performance, or if the column is removed. Default: true

host_owner

Defines which is the default owner of newly provisioned hosts. It can be either a user or a user group. If unset, the default owner of the host will be the user who created the host. Default: none

idle_timeout

Users that stay idle (no requests sent to Foreman) for more than this number of minutes will be logged out. Default: 60

interpolate_erb_in_parameters

If true, Foreman variables will be exposed to the ENC. Check Template Writing for a more comprehensive guide on how to create and use these variables in your ERB templates. Default: true

ignore_facts_for_operatingsystem

When Foreman receives facts for a host (from any source, Puppet, Ansible…) it will try to update the operating system to whatever the incoming facts say. This setting allows you to import all facts but ignore those related with operating system. If this is set to true, Foreman will update the operating system of hosts using these facts. Default: false See also: ignored_facts_for_domain

ignore_facts_for_domain

See ignored_facts_for_operatingsystem - this setting is the equivalent for domains. Default: false

ignore_puppet_facts_for_provisioning

If this option is set to true then Foreman will not update the IP and MAC addresses stored on a host’s network interfaces with the values that it receives from facts. It will also include Foreman’s values for IP and MAC to Puppet in its node/ENC information. Default: false See also: ignored_interface_identifiers

ignored_interface_identifiers

When importing facts and updating stored information on network interfaces, any network interface with an identifier (e.g. eth0) that matches any of the items in this list will be ignored and not updated. This can be used to avoid updating special types of interfaces when Foreman has limited or no understanding of them. The contents are an array of strings which may contain * wildcards to match zero or more characters. Default: ['lo', 'usb*', 'vnet*', 'macvtap*', '_vdsmdummy_', 'veth*'] See also: ignore_puppet_facts_for_provisioning

legacy_puppet_hostname

This setting truncates the hostname of your smart proxy to ‘puppet’ if it starts with ‘puppet’. Default: false

libvirt_default_console_access

The IP address that should be used for the console listen address when provisioning new virtual machines via Libvirt. Default: 0.0.0.0

location_fact

The name of a fact from hosts reporting into Foreman which gives the full location name of the host. This can be used when hosts are created through fact uploads to ensure they’re assigned to the correct location to prevent resource mismatches. The location of a host will be updated to the value of the fact on every fact upload. For inherited locations, the fact should use slash-delimited names, e.g. “USA/New York”. Default: foreman_location

local_boot_(PXELinux/PXEGrub/PXEGrub2)

When creating hosts that use a PXE loader, this will be the default template for local boot.

• Default: none

login_delegation_logout_url

If your external authentication system has a logout URL, redirect your users to it here. This setting can be useful if your users sign in Foreman through SSO, and you want them to sign out from all services when they log out Foreman. Default: ‘’

login_text

Specifies text to be displayed on the Foreman login page underneath the version number. Default: ‘’

manage_puppetca

If this option is set to true then Foreman will manage a host’s Puppet certificate signing. If it is set to false then some external mechanism is required to ensure that the host’s certificate request is signed. Default: true

max_trend

Days that trend graphs will capture. Default: 30

modulepath

This it the modulepath that foreman uses when processing puppet modules. It is usually able to determine this itself at runtime but if it is not able to find a value then modulepath is used. Default: /etc/puppet/modules

name_generator_type

Specifies the method used to generate random hostnames when creating a new host, either Random-based, MAC-based for bare metal hosts only, or Off to disable the function and leave the field blank. Default: Random-based

oauth_active

Enables OAuth authentication for API requests. Default: false

oauth_consumer_key

OAuth consumer key Default: none

oauth_consumer_secret

OAuth consumer secret Default: none

oauth_map_users

This allows OAuth users to specify which user their requests map to. When this is false, OAuth requests will map to admin. Default: true

organization_fact

The name of a fact from hosts reporting into Foreman which gives the full organization name of the host. This can be used when hosts are created through fact uploads to ensure they’re assigned to the correct organization to prevent resource mismatches. The organization of a host will be updated to the value of the fact on every fact upload. For inherited organization, the fact should use slash-delimited names, e.g. “ACME Inc/Engineering”. Default: foreman_organization

outofsync_interval

Duration in minutes after the Puppet interval for servers to be classed as out of sync. This is added to the puppet_interval setting and used in pages such as the host list and dashboard. Default: 5

Parametrized_Classes_in_ENC

In Puppet 2.6.5+, the ENC may send a hash of the class’s attributes and values. Before then, the ENC used to send just an array of class names. Set this to true if you are using any version of Puppet equal to or higher than 2.6.5. Default: true

proxy_request_timeout

Timeout in seconds used when making REST requests to a Smart Proxy, e.g. when importing Puppet classes or creating DHCP records. May be set to a larger value when certain operations take a long time. Default: 60

puppet_interval

This is the number of minutes between each run of puppet. Default: 30

puppet_server

The default puppet server hostname. For larger organizations this is often a non fqdn so that a name like puppet can be a different host within each DNS domain. Default: puppet

puppetrun

If this option is set to true then Foreman will be able to trigger a puppet run on any host that it manages. Default: false

query_local_nameservers

If true, Foreman will query the local DNS. When false Foreman will query the SOA/NS authority. Warning! Querying a resolver can cause Foreman to get false positives when checking presence of DNS records due to caching. Default: false See also: dns_conflict_timeout

remote_addr

When Foreman receives client web requests via a smart proxy, proxy or load balancer, the original client source IP address is lost, replaced by the smart proxy, proxy, or load balancers IP instead. For template requests without a token, this causes a failure, because Foreman can’t match the request against a valid host.

Smart proxies, and other devices if configured, can preserve the original client IP within an HTTP X-Forwarded-For header, which Foreman can evaluate and use to match the request against a valid host.

In order to prevent spoofing and provide some level of security, Foreman will only evaluate X-Forwarded-For headers from devices which match the list of IPs configured here.

This is a regular expression, so it can support several load balancers, i.e: (10.0.0.1|127.0.0.1) Default: 127.0.0.1

require_ssl_smart_proxies

When set to true, Foreman requires a client SSL certificate on requests from smart proxies or services on them (e.g. Puppet masters), and will verify the CN of the certificate against the known smart proxies. If false, it uses the reverse DNS of the IP address making the request. require_ssl in config/settings.yaml should be enabled too. For more information about securing the connection between Puppet masters or smart proxies and Foreman, see Section 5.4.1 Default: true

restrict_registered_smart_proxies

When set to true, services such as Puppet masters (or Salt, Chef) need to have a smart proxy registered with the appropriate feature (e.g. Puppet) to access fact/report importers and ENC output. Default: true

root_pass

If a root password is not provided whilst configuring a host or its host group then this encrypted password is used when building the host. Default: ‘’ (To generate a new one you should use: openssl passwd -1 “your_password” )

safemode_render

In the default configuration with safemode_render set to true, access to variables, Foreman internals and any object that is not whitelisted within Foreman will be denied for system security.

When set to false, any object may be accessed by a user with permission to use templating features, either via editing of templates, parameters or smart variables. This permits users full remote code execution on the Foreman server, effectively disabling all authorization if set to false. It is strongly recommended for this setting to be true in most environments.

Default: true

send_welcome_email

New account holders will receive a welcome email when the account is created if this is enabled, including their username and a link to Foreman. Default: false

sendmail_arguments

Arguments given to the sendmail command when sending emails from Foreman. Default: -i

sendmail_location

Path to the sendmail binary, or other sendmail-compatible MTA for outbound email. Default: /usr/sbin/sendmail

smtp_address

Outbound SMTP connections will connect to the SMTP server at this address, either a hostname or IP address. Default: empty value (implying localhost)

smtp_authentication

Outbound SMTP connections will authenticate to the SMTP server using the protocol specified here, either: plain for the PLAIN SMTP mechanism (plain text), login for the LOGIN SMTP mechanism (plain text), cram-md5 for the CRAM-MD5 method (hashed, not plain text), or none to disable authentication. See also: smtp_username, smtp_password. Default: none

smtp_domain

Outbound SMTP connections will use this domain to identify during the HELO/EHLO command. Default: empty value

smtp_enable_starttls_auto

Outbound SMTP connections will automatically switch to TLS mode (via STARTTLS) when the capability is advertised by the SMTP server. This implies verification of TLS/SSL certificates by default (see also: smtp_openssl_verify_mode). Default: true

smtp_openssl_verify_mode

Outbound SMTP connections to a TLS-enabled SMTP server will verify the remote server certificate according to this setting. Either the default (usually peer), none for no verification of the server certificate, or peer for explicitly verifying the server certificate. client_once and fail_if_no_peer_cert have no effect in outbound SMTP connections. Default: Default verification mode (usually peer)

smtp_password

Outbound SMTP connections with authentication enabled will authenticate with this password (see also: smtp_username, smtp_authentication). Default: empty value

smtp_port

Outbound SMTP connections will connect to the SMTP server on this TCP port. Some SMTP servers may prefer port 587 for email submission. Default: 25

smtp_username

Outbound SMTP connections with authentication enabled will identify with this username (see also: smtp_password, smtp_authentication). Default: empty value

ssl_client_cert_env

Environment variable containing the entire PEM-encoded certificate from the client. This environment variable is required when authenticating using Subject Alternative Names and will be preferred over ssl_client_dn_env if available. Under Apache HTTP and mod_ssl, SSLOptions +ExportCertData sets this environment variable. Default: SSL_CLIENT_CERT See also: ssl_client_dn_env

ssl_client_dn_env

Environment variable containing the subject DN from a client SSL certificate. Under Apache HTTP and mod_ssl, SSLOptions +StdEnvVars sets this environment variable. Default: SSL_CLIENT_S_DN See also: ssl_client_cert_env

ssl_client_verify_env

Environment variable containing the verification status of a client SSL certificate Default: SSL_CLIENT_VERIFY

ssl_ca_file

The SSL Certificate Authority file that Foreman will use when connecting to its smart-proxies. Default: The CA file used by puppet

ssl_certificate

The SSL certificate that Foreman will use when connecting to its smart-proxies. Default: The host certificate used by puppet

ssl_priv_key

The SSL private key file that Foreman will use when connecting to its smart-proxies. Default: The private key file used by puppet

token_duration

Time in minutes installation tokens should be valid for, 0 to disable token generation. Default: 360 (6 hours)

trusted_puppetmaster_hosts

Other trusted puppet masters in addition to Smart Proxies to access fact/report importers and ENC output. i.e: [puppetmaster1.yourdomain.com, puppetmaster2.yourdomain.com] Default: []

unattended_url

This controls the URL prefix used in provisioning templates such as TFTP/PXELinux files that refer to the Foreman server. It is usually HTTP rather than HTTPS due to lack of installer support for HTTPS. The FQDN is determined from Facter, else it will default to the “:fqdn” setting in /etc/foreman/settings.yaml. Default: http://FQDN/ See also: foreman_url

update_environment_from_facts

If Foreman receives an environment fact from one of its hosts and if this option is true, it will update the host’s environment with the new value. By default this is not the case as Foreman should manage the host’s environment. Default: false

update_ip_from_built_request

If true, Foreman will update the host IP with the IP that made the ‘build’ request. This request is made at the end of a provisioning cycle to indicate a host has completed the build. Default: false

update_subnets_from_facts

If true, fact imports from Puppet and other config management tools will update the subnet on host network interfaces to match the IP address given in facts, preventing a mismatch. Default: false

use_shortname_for_vms

When false, any hosts created on a compute resource will use the FQDN of the host for the name of the virtual machine. When set to the true, the short name (i.e. without domain) will be used instead. Default: false

use_gravatar

Display user avatars by matching their emails with emails at Gravatar.com Default: false

use_uuid_for_certificates

When enabled, Foreman will generate UUIDs for each host instead of using the hostname as the Puppet certname, which is more reliable with changing hostnames. Note that when disabling this setting, existing stored certnames won’t be changed or discarded until new certificates are requested from a host (i.e. on a rebuild), in order that the existing certificate remains known to Foreman and can be revoked.

websockets_encrypt

When enabled, virtual machine consoles using NoVNC will always be sent over an encrypted WebSocket connection. Requires both websockets_ssl_cert and websockets_ssl_key to be configured too. Default: true if require_ssl is enabled See also: websockets_ssl_cert, websockets_ssl_key

websockets_ssl_cert

Path to the SSL certificate that will be used for the WebSockets server when serving virtual machine consoles. Should be the same as the SSL certificate used for the Foreman web server (e.g. Apache).

websockets_ssl_key

Path to the SSL private key that will be used for the WebSockets server when serving virtual machine consoles. Should be the same as the SSL key used for the Foreman web server (e.g. Apache).

3.5.3 Database Setup

Foreman is a rails application. Therefore, anything that is supported under RAILS (sqlite, mysql, postgresql, …) can be used. See 3.3 Install From Packages for a list of packages for connecting foreman to the databse of your choice. At this time, Oracle DB is known to not work. Patches are welcome!

The database configuration file can be found at:

/etc/foreman/database.yml

SQLite (default)

By default, the database will be created in the db subdirectory.

production:
  adapter: sqlite3
  database: db/production.sqlite3
  pool: 5
  timeout: 5000

MySQL

Edit your config/database.yml and modify:

production:
  adapter: mysql2
  database: foreman
  username: foreman
  password: password
  host: localhost
  socket: "/var/run/mysqld/mysqld.sock"

Afterwards you would need to re-populate your database, simply execute:

foreman-rake db:migrate
foreman-rake db:seed

PostgreSQL

Edit your config/database.yml and modify:

production:
  adapter: postgresql
  database: foreman
  username: foreman
  password: password
  host: localhost

Switching from SQLite to MySQL/PostgreSQL while maintaining existing data

We have a rake task for this. First setup your database.yml to have the sqlite db as production and the mysql/psql db as dev:

production:
  adapter: sqlite3
  database: db/production.sqlite3
  pool: 5
  timeout: 5000

development:
  adapter: postgresql
  database: foreman
  username: foreman
  password: password
  host: localhost

Now migrate both dbs so they’re consistent:

bundle exec rake db:migrate RAILS_ENV=production
bundle exec rake db:migrate RAILS_ENV=development

Now move the data to the new db

bundle exec rake db:convert:prod2dev

(On RPM distros, remove “bundle exec” from the start of these commands.)

Once you’ve migrated to your new database using prod2dev, you should update your database.yml file to point your ‘production’ environment at the new database. You should also update the ‘development’ environment to point to an alternative location (for example, at SQLite) to ensure you don’t accidentally overwrite your production database.

Special notes for migrating to Postgres

Firstly, the default installer setup doesn’t give superuser privileges to the ‘foreman’ user, which then prevents the prod2dev script from temporarily disabling foreign keys. You’ll need to do

sudo -u postgres psql -c 'ALTER USER foreman WITH SUPERUSER'

before the prod2dev call, and

sudo -u postgres psql -c 'ALTER USER foreman WITH NOSUPERUSER'

after it.

Secondly, the psql sequence numbers will be wrong after the prod2dev execution. You can fix them like this:

cat <<EOF > reset.sql
SELECT  'SELECT SETVAL(' ||quote_literal(S.relname)|| ', MAX(' ||quote_ident(C.attname)|| ') ) FROM ' ||quote_ident(T.relname)|| ';'
FROM pg_class AS S, pg_depend AS D, pg_class AS T, pg_attribute AS C
WHERE S.relkind = 'S'
    AND S.oid = D.objid
    AND D.refobjid = T.oid
    AND D.refobjid = C.attrelid
    AND D.refobjsubid = C.attnum
ORDER BY S.relname;
EOF
psql -Atq -f reset.sql -o temp foreman
psql -f temp foreman
rm temp reset.sql

(big thanks to Fixing Sequences for the fix)

3.5.4 Puppet Reports

Foreman uses a custom puppet reports address (similar to tagmail or store) which Puppet will use to upload its report into Foreman. This enables you to see the reports through the web interface as soon as the client finishes its run.

Configuration

Client

Ensure that the puppet clients has the following option in their puppet.conf:

report = true

Without it, no reports will be sent.

Puppet master

First identify the directory containing report processors, e.g.

• AIO installations: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/reports/
• EL6 (e.g. RHEL, CentOS): /usr/lib/ruby/site_ruby/1.8/puppet/reports/
• EL7 or Fedora: /usr/share/ruby/vendor_ruby/puppet/reports/
• Debian or Ubuntu: /usr/lib/ruby/vendor_ruby/puppet/reports/
• other OSes, look for tagmail.rb in the Puppet installation (locate tagmail.rb)

Copy the report processor source to this report directory and name it foreman.rb.

Create a new configuration file at /etc/puppetlabs/puppet/foreman.yaml (Puppet 4 AIO) or /etc/puppet/foreman.yaml (non-AIO) containing:

---
# Update for your Foreman and Puppet master hostname(s)
:url: "https://foreman.example.com"
:ssl_ca: "/var/lib/puppet/ssl/certs/ca.pem"
:ssl_cert: "/var/lib/puppet/ssl/certs/puppet.example.com.pem"
:ssl_key: "/var/lib/puppet/ssl/private_keys/puppet.example.com.pem"

# Advanced settings
:puppetdir: "/var/lib/puppet"
:puppetuser: "puppet"
:facts: true
:timeout: 10
:threads: null

Edit the URL field to point to your Foreman instance, and the SSL fields for the hostname of the Puppet master (which may be the same host). Paths to Puppet’s SSL certificates will be under /etc/puppetlabs/ssl/ when using Puppet 4 with AIO.

Lastly add this report processor to your Puppet master configuration. In your master puppet.conf under the [main] section add:

reports=log, foreman

and restart your Puppet master.

You should start seeing reports coming in under the reports link.

Debugging reports

If reports aren’t showing up in Foreman when an agent is run, there can be a number of reasons. First check through the above configuration steps, and then look at these places to narrow down the cause:

1. Puppet master logs may show an issue either loading or executing the report processor. Check syslog (/var/log/messages or syslog) for puppet-master messages, or /var/log/puppetlabs/puppetserver/.
2. /var/log/foreman/production.log should show a POST "/api/reports" request each time a report is received, and will end in Completed 201 Created when successful. Check for errors within the block of log messages.
3. When viewing reports in Foreman’s UI, be aware that the default search is for “eventful” reports. Clear the search (‘x’) to see reports with no changes.

Expire reports automatically

You will probably want to delete your reports after some time to limit database growth. To do so, you can set a cronjob:

Available conditions:

• days => number of days to keep reports (defaults to 7)
• status => status of the report (defaults to 0 –> “reports with no errors”)

Example:

1. Expires all reports regardless of their status

   foreman-rake reports:expire days=7

2. Expires all non interesting reports after one day

   foreman-rake reports:expire days=1 status=0

3.5.5 Facts and the ENC

Foreman can act as a classifier to Puppet through the External Nodes interface. This is a mechanism provided by Puppet to ask for configuration data from an external service, via a script on the puppetmaster.

The external nodes script we supply also deals with uploading facts from hosts to Foreman, so we will discuss the two things together.

Configuration

Puppet master

Download the ENC script to /etc/puppetlabs/puppet/node.rb (Puppet 4 AIO) or /etc/puppet/node.rb (non-AIO). The name is arbitrary, but must match configuration below, and ensure it’s executable by “puppet” with chmod +x /etc/puppet/node.rb.

Unless it already exists from setting up reporting, create a new configuration file at /etc/puppetlabs/puppet/foreman.yaml (Puppet 4 AIO) or /etc/puppet/foreman.yaml (non-AIO) containing

---
# Update for your Foreman and Puppet master hostname(s)
:url: "https://foreman.example.com"
:ssl_ca: "/var/lib/puppet/ssl/certs/ca.pem"
:ssl_cert: "/var/lib/puppet/ssl/certs/puppet.example.com.pem"
:ssl_key: "/var/lib/puppet/ssl/private_keys/puppet.example.com.pem"

# Advanced settings
:puppetdir: "/var/lib/puppet"
:puppetuser: "puppet"
:facts: true
:timeout: 10
:threads: null

Edit the URL field to point to your Foreman instance, and the SSL fields for the hostname of the Puppet master (which may be the same host). Paths to Puppet’s SSL certificates will be under /etc/puppetlabs/ssl/ when using Puppet 4 with AIO. More information on SSL certificates is at Securing communications with SSL.

Add the following lines to the [master] section of puppet.conf:

[master]
  external_nodes = /etc/puppet/node.rb
  node_terminus  = exec

Restart the Puppet master. When the next agent checks in, the script will upload fact data for this host to Foreman, and download the ENC data.

The --no-environment option can be optionally specified to stop the ENC from being authoritative about the agent’s Puppet environment. This can be useful in development setups where the agent may be run against different environments.

Client

No agent configuration is necessary to use this functionality.

Testing the config

Make sure that the puppet user can execute the ENC script and it works:

sudo -u puppet /etc/puppet/node.rb [the name of a node, eg agent.local]

should output something like:

parameters:
  puppetmaster: puppet
  foreman_env: &id001 production
classes:
  helloworld:
environment: *id001

This output should match the information displayed when you click on the YAML button on the Host page in Foreman.

For further information see the Puppet Labs docs on external nodes

Debugging the ENC

1. If Puppet agents receive empty catalogs, check the puppet.conf master configuration has the ENC script configured. Also check the output of the ENC for the hostname logged by Puppet (which may be different) to see if Foreman is reporting the correct configuration.
2. If the hostname.yaml facts file is missing, this is typically a Puppet misconfiguration. Check /etc/puppet/rack/config.ru has been updated if Puppet has been upgraded.
3. Failures to upload facts or download the ENC data may be a network issue (check the URL and SSL settings) or an error on the Foreman server. Check /var/log/foreman/production.log for two requests, POST "/api/hosts/facts" and GET "/node/client.example.com?format=yml" and for any errors within the block of log messages.

Assigning data to hosts through the ENC

Foreman passes all assoicated parameters, classes,and class parameters, to the Host, including those inherited from host groups, domains, or global settings. See section Managing Puppet for more information on assigning configuration to hosts.

Creating hosts in Foreman with facts

By default, Foreman adds hosts to its database that it learns about through facts, provided the “create_new_host_when_facts_are_uploaded” setting is enabled.

If locations or organizations are enabled, these can be inferred from the “foreman_location” or “foreman_organization” facts as supplied by the host. The names of these facts can be changed with the “location_fact” and “organization_fact” settings respectively. Foreman will update hosts on each fact upload based on the value of these facts.

If these facts aren’t supplied, then the “default_location” and “default_organization” settings can be used to set values globally when a host doesn’t have a location or an organization set.

More information in the Configuration section.

Pushing facts to Foreman when not using the ENC functionality

There are several options for pushing fact data to Foreman if you are using Foreman for reporting/inventory only.

Using node.rb

The ENC script (node.rb) accepts an option to run in ‘batch-mode’. In this mode, the script iterates over the cached fact data stored on the puppet master, and uploads all of it to Foreman.

Download and configure the node.rb script as above, and then call it like this:

sudo -u puppet /etc/puppet/node.rb --push-facts

The following options are available for node.rb’s batch mode:

• --push-facts uploads all facts sequentially which have changed since the last run.
• --push-facts-parallel uploads all facts in parallel which have changed since the last run. The number of threads is specified by the :threads setting or the number of processors.
• --watch-facts runs in the foreground and upload facts based on inotify events, used in conjunction with either –push-facts option.

Direct HTTP upload

As of Foreman 1.3, the fact-upload API endpoint accepts data in pure JSON. You can push data to Foreman as a hash containing:

{
  "name": "fqdn-of-host.domain.com",
  "certname": "optional-certname-of-host.domain.com",
  "facts": {
    "fact1": "string",
    "fact2": "true",
    "fact3": "1.2.3.4",
    ...
  }
}

The ‘certname’ is optional but will be used to location the Host in Foreman if supplied. The ‘facts’ hash must be a flat hash, not nested with other arrays or hashes. See link-to-API-when-its-updated-here for more details.

This body can be POSTed to ‘/api/hosts/facts’ using Foreman API v2. See the node.rb template for an example of constructing and sending data in Ruby.

3.5.6 CLI

The Command Line Interface is based on the hammer framework. The foreman-related commands are defined in plugin hammer_cli_foreman

Format and locations

Configuration is loaded from a set of directories in this order:

• ./config/hammer/ (config dir in CWD)
• /etc/hammer/.
• ~/.hammer/
• custom location specified on command line - -c CONF_FILE_PATH

In each of these directories hammer tries to load cli_config.yml and anything in the cli.modules.d subdirectory which is the recommended location for configuration of hammer modules.

Later directories and files have precedence if they redefine the same option. Files from cli.modules.d are loaded in alphabetical order.

Hammer uses yaml formatting for its configuration. The config file template is contained in the hammer_cli gem.

gem contents hammer_cli|grep cli_config.template.yml

and can be copied to one of the locations above and changed as needed. The packaged version of hammer copies the template to /etc for you.

Plugins

Plugins are disabled by default. You have to edit the config file and enable them manually under modules option, as can be seen in the sample config below.

Plugin specific configuration should be nested under plugin’s name.

Options

• :log_dir: <path> - directory where the logs are stored. The default is /var/log/hammer/ and the log file is named hammer.log
• :log_level: <level> - logging level. One of debug, info, warning, error, fatal
• :log_owner: <owner> - logfile owner
• :log_group: <group> - logfile group
• :log_size: 1048576 - size in bytes, when exceeded the log rotates. Default is 1MB
• :watch_plain: false - turn on/off syntax highlighting of data being logged in debug mode

Sample config

:modules:
    - hammer_cli_foreman

:foreman:
    :host: 'https://localhost/'
    :username: 'admin'
    :password: 'changeme'

:log_dir: '/var/log/foreman/'
:log_level: 'debug'

3.6 Upgrade

Upgrading to Foreman 1.17

Preparation

Before updating to 1.17, make sure you have successfully upgraded to 1.16 first.

Upgrading across more than one version is not supported, so it’s required to upgrade to each intermediate version and follow all upgrade instructions for the previous versions.

Check the list of Supported Platforms when planning to upgrade, as these change between releases. If your OS is no longer supported by Foreman, migrate or upgrade the OS (if supported) using a release of Foreman supported by both OS versions before upgrading Foreman.

Step 1 - Backup

It is recommended that you backup your database and modifications to Foreman files(config/settings.yaml, unattended installations etc). Most upgrades are safe but it never hurts to have a backup just in case.

For more information about how to backup your instance head over to Backup

Step 2 - Perform the upgrade

Before proceeding, it is necessary to shutdown the Foreman instance (e.g. service httpd stop or service apache2 stop usually).

Now it’s time to perform the actual upgrade. This process if different depending on how you downloaded Foreman. You only need to perform one of the following options.

Step 2 (A) - RHEL package (RPM) and installer setups

To upgrade an existing Foreman installation, first update with the appropriate foreman-release package for your operating system:

yum upgrade https://yum.theforeman.org/releases/1.17/el7/x86_64/foreman-release.rpm

Clean up the yum cache:

yum clean all

First upgrade or install the SCL release package to add rh-ruby24 and tfm-ror51 repositories:

yum install foreman-release-scl

Next upgrade all Foreman packages:

yum upgrade tfm\* ruby\* foreman\*

In order to catch all configuration changes and manage them properly you should install and run rpmconf from the EPEL repository along with vim-enhanced (for vimdiff):

rpmconf -a --frontend=vimdiff

You can skip to Step 3.

Step 2 (B) - Debian or Ubuntu package (deb) and installer setups

Upgrading from the last release to 1.17 has been tested on both Debian and Ubuntu. Updating the packages will upgrade the application and automatically migrate the database.

First edit /etc/apt/sources.list.d/foreman.list and change the release number from the previous release or “stable” to 1.17:

deb http://deb.theforeman.org/ stretch 1.17
deb http://deb.theforeman.org/ plugins 1.17

Next upgrade all Foreman packages:

apt-get update
apt-get --only-upgrade install ruby\* foreman\*

You can skip to Step 3.

Step 2 (C) - Downloaded release (tar.bz2)

• Uncompress the new program archive in a new directory.
• Copy your database settings-file config/database.yml into the new config directory.
• If your database is a simple file (e.g. SQLite), don’t forget to make it available in the new directory.

VERY IMPORTANT: do NOT overwrite config/settings.yml with the old one.

Step 2 (D) - git checkouts only

Please note that there can be significant differences between stable and development branches. You MUST take care to select a branch and make sure you get the correct one.

1. Go to the Foreman root directory and run the following command:

For staying on the stable branch:

• git checkout 1.17-stable
• git pull

There may have been changes to the ruby gem or node module versions, to update run the following commands:

bundle update
npm update

The following step is the one that could change the contents of your database. Go to your new Foreman directory (or the git dir), then migrate and update the contents of your database:

foreman-rake db:migrate
foreman-rake db:seed

You should compile i18n strings and precompile assets now:

foreman-rake locale:pack
foreman-rake assets:precompile RAILS_ENV=production
foreman-rake webpack:compile RAILS_ENV=production

Step 3 - Post-upgrade steps

Step 3 (A) - Database migration and cleanup

The database should be migrated already, but you can make sure by executing the migration script again, it should produce no errors or output:

foreman-rake db:migrate
foreman-rake db:seed

You should clear the cache and the existing sessions:

foreman-rake tmp:cache:clear
foreman-rake db:sessions:clear

Step 3 (B) - OpenStack and Rackspace compute resource users

The functionality for OpenStack and Rackspace compute resources has been separated into their own packages in Foreman 1.12, so if you use either compute resource then install either foreman-openstack:

yum install foreman-openstack      # EL/Fedora
apt-get install foreman-openstack  # Debian/Ubuntu

or foreman-rackspace:

yum install foreman-rackspace      # EL/Fedora
apt-get install foreman-rackspace  # Debian/Ubuntu

Optional Step 3 (C) - Run foreman-installer

If you used foreman-installer to set up your existing Foreman instance we recommend running it again after upgrading. Note that the installer can modify config files so this may overwrite your custom changes. You can run the installer in noop mode so you can see what would be changed.

To see what would happen

foreman-installer --noop --dont-save-answers --verbose

You may see ERRORS such as /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[foreman-hostname.domain]: Could not evaluate: Connection refused - connect(2) due to httpd / apache2 service being stopped. These can be safely ignored.

To apply these changes, run the installer again without options

foreman-installer

Step 4 - Restart

Restart the application server (e.g. mongrel, thin, passenger).

On RPM installations, run:

service httpd restart

And on Debian/Ubuntu installations with Passenger, run:

service apache2 restart

Optional Step 5 - Cleanup of RPMs and repositories

On EL6/7, consider removing the unmaintained softwarecollections.org repositories, which are replaced by the CentOS SCLorg repos:

yum erase rhscl-\*

And consider removing unused SCL packages:

yum erase ruby22\* rh-ror42\*

Common issues

See Troubleshooting

4. General Foreman

This section covers general information on using Foreman to manage your infrastructure. It covers the features of the web interface, managing puppet, provisioning systems and the installation and configuration of Foreman Smart Proxies.

4.1 Web Interface

4.1.1 LDAP Authentication

Foreman natively supports LDAP authentication using one or multiple LDAP directories.

Setting up

Go to Administer > LDAP Authentication, click on New LDAP Source and enter the following details about the LDAP server:

• Name: an arbitrary name for the directory
• Server: the LDAP hostname, e.g. ldap.example.com
• LDAPS: check this if you want or need to use LDAPS to access the directory
• Port: the LDAP port (default is 389, or 636 for LDAPS)
• Server type: select the implementation if listed, else choose POSIX

Under the account tab, the details of an account used to read LDAP entries is required if anonymous binds and reads are disabled. This should be a dedicated service account with bind, read and search permissions on the user and group entries in the directory server.

This may use the variable $login which will be replaced by the login of the authenticating user, however this is deprecated and will result in reduced functionality (as it only works at authentication time).

• Account: leave this field empty if your LDAP server can be read anonymously, otherwise enter a user name that has read access
• Account password: password for the account, if defined above and when not using $login
• Base DN: the top level DN of your LDAP directory tree, e.g. dc=example,dc=com
• Group base DN: the top level DN of your LDAP directory tree that contains groups, e.g. ou=Groups,dc=example,dc=com
• Use netgroups: switcher that enables using netgroups instead of standard LDAP group objects, supported only for FreeIPA and POSIX LDAP server types
• LDAP filter (optional): a filter to restrict your LDAP queries, for instance: (memberOf=cn=foreman-users,ou=Groups,dc=example,dc=com). Multiple filters can be combined using the syntax (& (filter1) (filter2)).

Trusting SSL certificates

When configuring an LDAPS connection, the certificate authority needs to be trusted. When using Active Directory Certificate Services, ensure to export the Enterprise PKI CA Certificate using the Base-64 encoded X.509 format.

If your LDAP server uses a certificate chain with intermediate CAs, all of the root and intermediate certificates in the chain must be trusted.

On Red Hat based OSes:

cp example.crt /etc/pki/tls/certs/
ln -s example.crt /etc/pki/tls/certs/$(openssl x509 -noout -hash -in /etc/pki/tls/certs/example.crt).0

On Debian or Ubuntu, also ensure the file has a .crt extension:

cp example.crt /usr/local/share/ca-certificates/
update-ca-certificates

On the fly user creation

By checking Automatically create accounts in Foreman, any LDAP user will have their Foreman account automatically created the first time they log into Foreman.

If organizations/locations are enabled, you can assign multiple organizations/locations to your LDAP authentication sources. This will assign users that are automatically created to the set of organizations/locations associated with the LDAP authentication source. Please notice this assignment happens only when users are created automatically via LDAP, and not upon every login.

Changing the organization/location of a LDAP authentication source will not automatically change these attributes on the users in that authentication source.

To use this feature, the relevant LDAP attributes must be specified on the next tab (e.g. firstname, lastname, email), as these will be used to populate the Foreman account.

Attribute mappings

Foreman needs to know how to map internal user account attributes to their LDAP counterparts, such as login, name, and e-mail. Examples for common directory servers are provided below.

Note that LDAP attribute names are case sensitive.

Foreman also has the ability to use a user’s photo stored in LDAP as their Foreman avatar, by setting the jpegPhoto attribute mapping.

Additional Information:

• Adding Display Pictures/Avatars to Red Hat IDM/FreeIPA
• Using the jpegPhoto attribute in Active Directory

Examples

All of the examples below use a dedicated service account called ‘foreman’. This should be set up with bind, read and search permissions on the user and group entries and with a strong, random password.

Active Directory

Typically either LDAPS on port 636 or LDAP on port 389.

Note that previously we recommended using sAMAccountName as the login name attribute. It turned out that userPrincipalName is a better choice since it does not contain white spaces that can cause issues on user creation.

FreeIPA

Typically either LDAPS on port 636 or LDAP on port 389.

OpenLDAP

Typically LDAP on port 389 and with anonymous queries (leave Account blank), unless configured otherwise.

Linking user groups to LDAP

A Foreman user group can be associated to a group stored in an LDAP server, so membership of the LDAP group automatically adds a user to the Foreman user group. User groups can be associated with roles, enabling users to log into Foreman and be automatically granted permissions via their membership of an LDAP group. Read more about permissions in the Roles and Permissions section.

To configure the association, create or edit a user group via Administer > User groups. The group name may be any value (no direct relation to the LDAP group). Under the Roles tab, select roles granting permissions to Foreman, or tick the Admin checkbox to enable administrator level access.

On the External groups tab, click the Add external user group button to open a new form. In the Name field, enter the exact name of the LDAP group (usually the common name/CN) and select the server from the dropdown list of LDAP authentication sources. Click the Submit button to save changes.

When a user logs in for the first time (assuming on the fly account creation), the ldap:refresh_usergroups cronjob runs (every 30 minutes by default) or the Refresh button is pressed next to the external user group entry, Foreman will synchronize the group membership from LDAP.

Security Disclaimer

Please remember your external user groups will only be refreshed automatically through the ldap:refresh_usergroups cronjob. There can be a lapse of time cronjob runs, in which if the user groups in LDAP change, the user will be assigned to the wrong external user groups. This situation can be quickly fixed by manually running foreman-rake ldap:refresh_usergroups or by refreshing the external user groups in the UI. Otherwise, the problem will eventually get fixed when the cronjob runs again.

Active Directory password changes

When using Active Directory, please be aware that users will be able to log in for up to an hour after a password change using the old password. This is a function of the AD domain controller and not Foreman.

To change this password expiry period, see Microsoft KB906305 for the necessary registry change.

Brute-force protection

Foreman allows only 30 failed attempts in the last 5 minutes per one IP address by default. Any subsequent login attempts are not allowed and error message is shown: “Too many tries, please try again in a few minutes.” If this is triggered by accident, the silent period can be removed by deleting failed login cache entries:

find /usr/share/foreman/tmp/cache -name failed_login\*

This will only work when using the file store Rails cache implementation.

Troubleshooting

If you want to use on the fly user creation, make sure that Foreman can fetch from your LDAP all the required information to create a valid user. For example, on-the-fly user creation won’t work if you don’t have valid email addresses in your directory (you will get an ‘Invalid username/password’ error message when trying to log in).

4.1.2 Roles and Permissions

A user’s access to the features of Foreman are constrained by the permissions that they are granted. These permissions are also used to restrict the set of hosts, host groups and other resources that a user is able to access and modify.

Note: a user with global admin enabled is not restricted by the authorization system. This is the default for installations that do not have :login:true in config/settings.yml.

A logged in user will be granted the Default role role plus one or more additional roles. The permissions and filters associated with these roles are aggregated and determine the final permission set.

Roles may be administered by users with admin privileges or regular users with ‘edit_roles’ permission. In order to add new filters and permissions to a role, regular users must have the ‘create_filters’ permission.

Roles

These may be created, deleted and edited on the Roles page. Each role will contain permission filters, which define the actions allowed in a certain resource. Once your role is created, you can associate it with one or more users and user groups.

There is one built-in system role, ‘Default role’. This is a set of permissions that every user will be granted, in addition to any other roles that they have. Foreman provides you with a set of seeded roles. These roles can be assigned to users but cannot be modified in any way. They serve as a sane set of defaults and a quick starting point. If you wish to base your custom role on one of these, you can clone it and modify the clone.

Roles can be also associated to Locations or Organizations if these are allowed. Unlike other objects this does not mean that Roles would be only available in a particular scope. Roles are always global for the entirety of Foreman. The association means that filters of such role are scoped to a particular Organization or Location. Imagine you want to create a role representing Administrator of Organization A. You can clone an existing Organization admin role and associate it with Organization A. If you later assign this role to some users, they will be granted all admin permissions but only on resources of Organization A. Note that some resources are not scopeable by Organization and Locations. Filters for such resources grant permissions globally.

The seeded Organization admin role is similar to the Manager role. They are both being automatically extended with permissions introduced in new Foreman versions, as well as permissions introduced by plugins. The difference is that Organization admin role does not contain permissions for managing organizations, only for viewing them. Since organization administrator does not usually need to create or modify other organizations, the Organization admin role fits better this scenario.

Filters

Filters are defined within the context of a role, clicking on the ‘filters and permissions’ link. A filter allows a user to choose a resource (Hosts, Host groups, etc…) and the permissions that should be granted for that resource. After a filter has been created, users given a role containing this filter will have the permissions for the resource specified at the filter.

If the filter is marked as ‘Unlimited?’, the permissions created in this filter will apply to all objects in the chosen resource. For instance, if the resource is Host, and the permissions are ‘view’ and ‘index’, and ‘Unlimited?’ is checked, users that have a role with this filter will be able to ‘view’ and ‘index’ all hosts in the system.

When ‘Unlimited?’ is unchecked, a text box allowing to define more granular filtering will be enabled. You can write a search query and permissions in this filter will be applied to the results of that query only. An example of a query for the resource Host could be ‘os = RedHat’. In this case, the permissions in this filter will be applied only to Hosts whose Operating System is set to Red Hat. You can test your search queries at the index page of your resource, in this case that would be ‘/hosts’.

Some example queries for the resource Host:

1. Ownership and domain membership: ‘owner_id = 95 and domain = localdomain’ - Will apply permissions to hosts owned by User with id 95 and in the domain ‘localdomain’

2. Compute resource membership: ‘compute_resource = Openstack’ - Will apply permissions to hosts deployed on compute resource Openstack.

3. Fact filtering: ‘facts.alarmlevel = high’ - Will apply permissions to hosts with a fact ‘alarmlevel’ with value ‘high’. As a fact is only generated during a puppet run, this filter will only refer to machines that have been built and therefore cannot be used to restrict the creation of machines.

These pools of queries can be combined by adding them together or the filters can be used to restrict the selected resource to a smaller and smaller subset of the total. Think of them as set operations.

As already mentioned, a Role can be assigned to Organizations and Locations. In such case, all filters for resources that support such scoping automatically apply the same Organizations and Locations. If you want to combine filters with different Organizations or Locations assignments, you can use ‘Override’ check box. When checked you can override Organizations and Location for a filter. If you uncheck this field, the filter starts inheriting its role Organizations and Locations after submitting again. If you want to reset all role filters to start inheriting, you can use ‘Disable all filters overriding’ button on role’s ‘Filters’ tab.

We recommend managing Organizations and Locations association on Role level to keep the setup simple and clear.

Note: If the “Administrator” check box is checked for a user, filtering will not take effect.

Permissions

These determine the operations that are allowed to be performed upon the resources to which they refer. For a few simple items like bookmarks, this operates as expected - it grants permission for all bookmarks. But for most resources, such as the hosts a user is able to operate on, there is an additional layer of security called filtering.

When editing a filter there is a search field at the bottom that narrows the scope of the permissions granted to a subset of the resource objects. Most permission types support this search field however there are some exceptions. The permission for creating objects can’t be limited by a search query because the object does not exist during creation. Therefore a user is granted the create permission if they are associated with any filter containing this permission (limited by search or not).

Following table lists some of permissions and their impact:

4.1.3 Trends

Trends in Foreman allow you to track changes in your infrastructure over time. It allows you to track both Foreman related information and any puppet facts. The Trend pages give a graph of how the number of hosts with that value have changed over time, and the current hosts list.

There are two pieces to the Trends area, the Trends to track and their corresponding counters. To define trend counters, use the “Add Trend Counter” button on the ‘/trends’ page. Optionally set the “Name” field to over-ride odd puppet fact names to be more readable. Once created you can optionally ‘Edit’ the Trend to change the display names of the underlying values.

Next, to start collecting trend data, set a cron job to execute ‘foreman-rake trends:counter’. Each time the rake task executes it will create 1 tick on the graphs, so you can fine tune the granularity with your cron job. We recommend using the same as your puppet run interval (30 minutes). Here’s an example to run once per hour:

0 * * * * /usr/sbin/foreman-rake trends:counter

Finally note that these trends are the same for all users. So if you delete a Trend category you will loose all history for that trend and the trackers will start all over again. So please be careful when deleting.

4.1.4 Auditing

Foreman supports auditing of almost all changes that happen within Foreman, from both the UI and from the API. Auditing is done at a user level, and is thus ineffective if :login: is set to false, as all audits will be done as the ‘admin’ user.

Basic View

Go to the Audit tab to see a view of what has changed. This view can be filtered by the type of change or by the object that was altered (e.g. search for a hostname to see all changes relating to that host). You also get the parent object - so if a parameter was modified, you can see what host/group that parameter belongs to. The timestamp of the change and the user who performed it will be listed.

Extended Audits for Templates

Template changes also store a diff of the changes, and the ability to roll back to a previous version of the template.

4.1.5 Searching

Each page in Foreman has its own search functionality, which is centred around the resources that it manages, allowing searching based on attributes of the resources in the list or resources that they’re associated to. The search box also features powerful auto-completion to help build up search queries and free text search on many pages. The search functionality can also be used in the API when listing resources, see Customize JSON Responses for details.

General usage

Searching is through “field = value” or free text queries, which can be combined with logical operators (and, or, not) and parentheses to handle more complex logic. To give some examples:

• name = client.example.com on the host list would show the host(s) whose hostname is client.example.com
• hostgroup = "Web servers" and domain != lon.example.com would show hosts in the Web servers host group, but not in the lon.example.com domain
• Web servers would show all hosts with that text anywhere, e.g. as their host group name or in the comment field

The fields available depend on the type of resource that’s being searched, and the names of the attributes vary depending on the context. The “name” field on the host groups list is equivalent to the “hostgroup” field on the hosts list. Requests to add additional searchable fields are welcome, and may be filed in the “Search” category in the bug tracker.

The search engine is provided by the scoped_search library, which maps search queries directly to SQL queries. The Query Language documentation provides A more complete specification of the syntax available.

Bookmarks

Foreman supports the ability to make search bookmarks, allows users to quickly jump to predefined search conditions. Available bookmarks can be selected from the dropdown menu to the right of the search box, or managed from Administer > Bookmarks.

Some of the bookmarks are provided by default, e.g. to search for active or inactive hosts, or to only view reports with events.

To save a query, Use the dropdown menu to the right of the search box and click “Bookmark this search”. When saving, the bookmark can be labeled as public, so all other users are able to see and use it too.

Free text search

If you ignore the auto-completer and just enter text in the search field, Foreman will try searching for that text across multiple fields.

For example, if you just enter 12 in the hosts search box, the results will include all hosts with 12 in their IP address, MAC address or name. In general the fields used for free text search are kept to a minimum for performance and accuracy reasons. It’s preferable to search using a specific field, e.g. when searching for an IP address, use ip ~ 12 instead of 12.

Searching for present/empty values

The “has” operator matches values that are present, e.g. to search for hosts that are on a compute resource, use has compute_resource.

Similarly, this can be negated, so to search for hosts without host groups, you can use not has hostgroup.

Case sensitivity

When querying using = and != operators then exact, case sensitive matches will be returned. When running ~ (like) and !~ (unlike) operators, the matching is case insensitive.

Quoting

In search queries, white spaces are used as a delimiter. Here are some examples of the way a query will be interpreted:

• description ~ "created successfully": list all notifications that contain “created successfully”
• description ~ created successfully: list all notifications that contain “created” and at least one of its text fields contains “successfully”
• description !~ created successfully: list all notifications that doesn’t contain “created” and at least one of its text fields contains “successfully”

In the second and third example, “successfully” is an additional term that is interpreted as a free text search

Wildcards (‘_’, ‘%’, ‘*’)

The ~ and !~ search operators are translated to the LIKE and NOT LIKE SQL queries respectively, which support two basic wildcards, _ and %.

_ is a wildcard for a single character replacement. For example, the search name ~ fo_ will match both “foo” and “for”.

The % and * wildcard will replace zero or more characters. For example, the search name ~ corp% will match both “corp” and “corporation”. The more common ‘*’ wildcard is not a SQL wildcard but may be used instead.

When the ~ or !~ search is processed, a ‘%’ wildcard is automatically added at the beginning and end of the value if no wildcard is used, so it will by default match at any location inside a string. For example, the search name ~ foo is equivalent to name ~ %foo% and the search name ~ foo% will only match “foo” at the beginning of the value.

Date-time search query syntax

Many date and time formats are accepted in search queries. Here are some examples:

• “30 minutes ago”, “1 hour ago”, “2 hours ago”, Today, Yesterday
• “3 weeks ago”, “1 month ago”, “6 days ago”, “July 10,2011”

The date can have different separators, “10-July-2011” will be interpreted in the same way as “10/July/2010” or “10 July 2011” Month names may be the full English name or a three letter abbreviation, e.g. “Jan” will be interpreted as “January”. Many other formats are also acceptable, however it is not recommended to use ambiguous formats such as “3/4/2011”

The valid date time operators are ‘=’, ‘<’ and ‘>’ which are interpreted as ‘at’, ‘before’ and ‘after’ respectively. This is how the search term interpeted:

The right hand part of a date-time condition is parsed and translated into a specific date-time, “30 minutes ago” is translated to “now - 30 minutes”.

• last_report > "2011-07-01 12:57:18 EDT" should be read as created after this time
• In the same way, last_report > "30 minutes ago" should be read as “created after 30 minutes ago” and not “created more then 30 minutes ago”

A search query like installed_at = Yesterday is translated into a period query, it will be translated at runtime to match a range of date-times. For example, if running on Jan 1, it would be translated into “(installed_at >= Jan 1,2012 00:00) and (installed_at < Dec 31,2011 00:00)”.

4.1.6 User Management

Foreman is all about hosts and users interacting with these hosts.

SSH Keys

Each Foreman user can have multiple SSH keys assigned when editing a user. These keys alone do not serve any purpose, but are available for use in provisioning templates and can be accessed via ENC data. They provide an easy way to manage users and login ssh keys on hosts without the need for LDAP. If you want users to be able to login to a host using the data provided in Foreman, you need to include the create_users snippet in your provisioning template. There is a puppet module available to keep user data in sync with Foreman and your hosts.

4.2 Managing Puppet

In this section we’ll look at the various ways we can control and interact with Puppet.

4.2.1 Environments

Puppet environments are mapped directly into Foreman. They can be used at various levels throughout the Foreman interface. Puppet environments are generally used to separate classes from different types of Host, typically allowing changes to a module to tested in one environment (e.g. development) before being pushed to another (e.g production).

Defining environments

There are several ways to create Puppet environments within Foreman.

Importing from Puppet

Foreman can detect all the environments and classes contained on a Puppet master, and import them automatically. To do this, go to Configure > Environments and click on Import from <proxy-name>. Foreman will scan the Puppet master via the Smart Proxy, and display a confirmation of the detected changes. Select the changes you wish to apply and confirm.

More information about configuring the Smart Proxy to read environments and Puppet classes is in the Smart Proxy Puppet section.

Note that the Smart Proxy will only detect environments that contain one or more Puppet classes, so ensure that at least one Puppet module containing a class has been deployed to the Puppet master.

Manual creation

To create an environment by hand, simply go to Configure > Environments and click New Puppet Environment. Give the new environment a name and save. Note that if the environment doesn’t exist on the Puppet master and you subsequently run an import (above), Foreman will prompt for the environment to be deleted.

Assigning environments to hosts

This is done from the Host Edit page, on the Host tab. Selecting an environment will filter the classes visible on the Puppet Classes tab to just the classes in the selected environment.

You can also also mass-assign an environment to a group of hosts - tick the checkboxes of the required hosts in the Hosts list, and then select Change Environment from the Select Action dropdown menu at the top of the page.

Environments with host groups

You can assign an environment to a hostgroup as well. This functions as a form of default - a user creating a new host and selecting the hostgroup will automatically have the environment pre-selected. The user is not prevented from changing the environment of the new host, it simply saves a few clicks if they are happy with it.

4.2.2 Classes

Puppet classes are generally imported from the Puppet Master(s) via the Import button on the Puppet Classes page. They can also be created by hand, and manually associated with a set of environments (for filtering purposes).

Importing Classes

Go to Configure > Classes and click the Import button. This will not be visible unless you have at least one Puppet Master with a puppet-enabled Smart Proxy. Only classes from modules will be imported, and the Puppet manifests must be valid in order for the Smart Proxy to parse them. Use puppet parser validate to test the syntax of Puppet manifests.

More information about configuring the Smart Proxy to read environments and Puppet classes is in the Smart Proxy Puppet section.

The “Hosts” Column

Under Configure > Classes you will also see a column called “Hosts”. This column represents the number of hosts the given module/class has been assigned to. Clicking this figure will list the hosts.

Ignoring classes on import

It’s often to have a module structure like this:

$ tree git/
git/
└── manifests
    ├── init.pp
    ├── install.pp
    ├── params.pp
    └── repo.pp

In this situation, Foreman would offer to create:

git
git::install
git::params
git::repo

However, if we know that the subclasses are not intended for direct consumption, but are only really part of the internal structure of the module, then we would want to exclude those from the import mechanism, so that Foreman only offers to import git. We can achieve this via the file /usr/share/foreman/config/ignored_environments.yml.

This file is read during each import, causing Foreman to ignore changes to the listed environments or Puppet classes that match the expressions in the file. It will not delete any environments or classes already in Foreman.

Entire environments can be ignored with this configuration:

:ignored:
  - development
  - testenv

Classes can be ignored using a set of regular expressions - any class which matches one of them will not be imported. So, for the above example, we might configure:

:filters:
  - !ruby/regexp '/install$/'
  - !ruby/regexp '/params$/'
  - !ruby/regexp '/repo$/'

Regular expression features such as negative lookaheads can be used for more advanced filtering, e.g. to ignore all classes except for those starting with “role::”, the following syntax can be used:

:filters:
  - !ruby/regexp '/^(?!role::)/'

Assigning classes to hosts

To cause Puppet to apply your classes, you will need to assign them to your hosts. This can be achieved in a number of ways - the best method may vary depending on how many classes you intend to assign and whether any parameters need to be overridden.

Individual host assignment

When editing a host, Puppet classes may be assigned directly under the Puppet Classes tab. All classes that are in the Puppet environment selected on the first Host tab will be listed.

Via a host group

Host groups tend to correspond to an infrastructure role as each host may be assigned to a single host group, and typically inherits most of its Puppet classes in this way.

Puppet classes can be assigned by editing the host group and selecting them on the Puppet Classes tab.

Most host group attributes are copied to a host when it is created, however Puppet class associations remain inherited from the host group throughout its lifetime. Any change to a host group’s assigned Puppet classes or parameters will affect any host with that host group set.

The Puppet environment attribute may be different on the host to the host group, which means that Puppet classes assigned to the host group may not exist in the host’s own Puppet environment. Any Puppet classes that are inherited from the host group, but do not exist in the host’s environment will be left out when Foreman renders the ENC (YAML) output. Check under Configure > Puppet classes that the classes are available in both the host group and host environments if they differ.

You can also also mass-assign a host group to a number of hosts - tick the checkboxes of the required hosts in the Hosts list, and then select Change Group from the Select Action dropdown menu at the top of the page.

Using config groups

A config group provides a one-step method of associating many Puppet classes to either a host or host group. Typically this would be used to add a particular application profile or stack in one step.

To create a config group, click on Configure > Config groups, click New Config Group, enter a name and select the desired Puppet classes. When editing either a host or host group, the new config group can be added at the top of the Puppet Classes tab.

Config groups are not specific to an environment and so only those Puppet classes that are in the host’s environment when rendering the ENC (YAML) will be listed. Any classes that are not listed in the environment (as per Configure > Classes) will be left out.

Note that it isn’t possible to use a smart class parameter override with a config group, as a host may have many config groups with no way to define an order of precedence. Overrides should be made on a host group, host or other attribute.

Checking the results

To see how Foreman is passing the classes to Puppet, go to a Host and click the YAML button. You will be shown the exact YAML data sent to the Puppet master - the classes will be in the “classes” hash.

4.2.3 Parameters

Foreman can pass two types of parameters to Puppet via the ENC (External Node Classifier) interface - global parameters (accessible from any manifest), and class parameters (scoped to a single Puppet class). These can be added in a number of ways through Foreman.

Generally speaking, it’s best to use class parameters where possible, as this makes designing, using and sharing Puppet modules and classes easier. The class may clearly specify which parameters it expects, provide sensible defaults and allow users to override them. Foreman is also able to import information about class parameters automatically, making it easier to consume new classes without needing to know and enter the precise names of global parameters.

Types of parameters in Puppet

In Puppet’s DSL, accessing a global parameter or variable is done using $::example (preferred) or $example for a parameter named “example” in Foreman. More information about accessing variables is available in the Puppet Language: Variables documentation. When looking at the ENC (YAML) output from Foreman, a global parameter will look like this:

parameters:
  example: "foo bar"

When using class parameters, a class will first be defined with a parameter and may be accessed either using the local name or fully-qualified, e.g.

class example($setting) {
  notice($setting)
  notice($::example::setting)  # fully-qualified
}

When looking at the ENC (YAML) output from Foreman, a class and class parameter will look like this:

classes:
  example:
    setting: "foo bar"

Types of parameters in Foreman

Global parameters in Foreman can be added in the following places:

• Globally, per-resource (e.g. host group, domain) or per-host
• Smart variables

Class parameters in Foreman can be set in:

• Puppet classes, as a smart class parameter

Global parameters

Host inherit their list of global parameters from the following locations, in order of increasing precedence:

• Globally defined parameters, under Configure > Global parameters. These apply to every host.
• Organization-level parameters if organizations are enabled, under Administer > Organizations > edit > Parameters.
• Location-level parameters if locations are enabled, under Administer > Locations > edit > Parameters.
• Domain-level parameters, under Infrastructure > Domains > edit > Parameters.
• Subnet-level parameters, under Infrastructure > Subnets > edit > Parameters.
• Operating system-level parameters, under Hosts > Operating systems > edit > Parameters.
• Host group-level parameters, under Configure > Host groups > edit > Parameters.
• Host parameters, under Hosts > All hosts > edit > Parameters.

The final (most specific) level of global parameters applies only to a single host. Edit a Host and switch to the Parameters tab, and you will see all of its inherited parameters from the previous levels. You can override any of these previously-defined parameters or define new ones here.

Checking the results

To see how Foreman is passing the parameters to Puppet, go to a Host and click the YAML button. You will be shown the exact YAML data sent to the Puppet master - the parameters will be in the “parameters” hash.

4.2.4 Smart Variables

Smart variables are a tool to provide global parameters (key/value data), normally to your Puppet ENC, depending on a set of rules. They are intended to be a stepping stone to full parameterized classes, when the class hasn’t been parameterized or in special cases when a global parameter is desired. The same matching/rules technology underpins both smart variables and smart class parameters. If in doubt, use smart class parameters.

Smart variables are associated with a Puppet class, but they result in a global parameter. They may have multiple possible values, all depending on hierarchical context or various conditions a user can wish to apply. For example:

Creating a smart variable

Start by going to Configure > Classes and then click one of your classes to edit it.

Click on the Smart Variables tab. If you have any existing smart variables, they will be listed at the left side of the tab.

Click New Variable, and you will be presented with a set of input fields:

Once you’ve created the defaults for your smart variable, you can then add at least one criterion to match against - click the “Add Matcher” button at the bottom of the form and a new row will be added to the matchers table:

Advanced usage

Smart variables are based on the smart matchers technology, and have a number of advanced features such as validation and multiple data types. More about these can be found in the Smart Matchers section.

API usage

It’s also possible to retrieve these values if you’re not using a ENC, via a custom Puppet function or a http request.. You’ll need to retrieve the value from

https://foreman/api/v2/hosts/<fqdn>/smart_variables/<parameter>

You can find a ready-made function for your puppet module here

4.2.5 Parameterized Classes

Parameterized class support permits detecting, importing, and supplying parameters direct to classes which support it, via the ENC. This requires Puppet 2.6.5 or higher.

Setup

By default, parameterized class support is enabled in Foreman. This can be checked from Administer > Settings > Puppet and ensure Parametrized_Classes_in_ENC is set to true.

Now you’ll need to import some parameterized classes from your Puppet master. If you don’t have any parameterized classes in your modules dir, the foreman-installer has several, you can download a few modules from the Puppet Forge. Once you have some parameterized modules, import your classes (see 4.2.2 Classes)

Configure a class

This example will work with the foreman class from the installer. Click on the class, and you should get a page with 3 tabs, like so:

The middle tab, “Smart Class Parameter”, is the important one. Click onto that, and you should see something like this:

On the left, we have a list of possible parameters that the class supports. On the right, we have the configuration options for the parameter selected.

Lets configure the foreman class to change the user the foreman processes run as. Select the user parameter, at the end of the list. Now lets go through the options:

• Puppet Environments / Parameter
  • These can’t be edited, they’re just for information.
• Description
  • Purely information textbox for making notes in. Not passed to Puppet, or reused anywhere else
• Override (important)
  • If this is unchecked, Foreman will not attempt to control this variable, and it will not be passed to Puppet via the ENC.
• Key type
  • The type of data we want to pass. Most commonly a string, but many other data types are supported. There’s no easy way to tell what type of data Puppet is expecting, so you will need to read through the code/documentation that comes with a particular module to find out. Changing the type field requires an appropriately set “Default Value” field.
• Default Value
  • This will be imported from Puppet initially, but if Puppet is using any class inheritance, you’ll get something unhelpful like “${$foreman::params::user}”. This is because Foreman won’t follow the inheritance, so you’ll need to set a sensible default value
• Hidden value
  • Should the values of the smart class parameter be hidden in the UI.

Ok, so let’s configure our user parameter. We want to tick Override, set type to “String” and set the default value to “foreman”, like so:

Default value

Most importantly, the Override option has to be enabled for Foreman to control this variable, otherwise it will never be managed and will not appear in the ENC output.

The Default value will be supplied in the ENC output and should be a supported value, such as a string, YAML or JSON structure or use template features (see following sections). When the Omit checkbox is enabled, no default value will be present in the ENC output unless an override matches. Puppet will instead use the class default or data binding (Hiera) as usual.

The default will be imported from the Puppet manifest initially, but if the class uses an inherited params pattern, it may contain an unhelpful string such as ${$foreman::params::user}. Foreman is unable to parse the actual value in this case as it might change when evaluated. Change the suggested default to the actual value, or tick the Omit checkbox.

Setting up matchers

We’ve configured the default, but that’s not very useful. We need to be able to override the default for hosts or groups of hosts. To do that we need the “Override Value For Specific Hosts” section at the bottom of the page.

Let’s say that any machine in the “development” Puppet environment should use a value of “foremandev” instead of “foreman” for the “user” parameter. Add “environment” to the end of the matchers list, then click the “New Matcher-Value” button, and fill it out like this:

This is a basic configuration - for more complex examples of using matchers, see the Smart Matchers section.

Overriding a parameter for a host

If Foreman manages the value of a class parameter (“override = true”), it’s also possible to update a host-specific override from the host itself. That way you don’t have to grant access to the Puppet Classes page to everyone. From a Host, click Edit, go to the Parameters tab, and you’ll see the variable, the class-scope, and the current value. You can then override the value for that host:

If the value is hidden you can click the unhide button to temporarily see the value while you edit. Clicking the button won’t change the hidden property for the parameter, only show it for editing purpose.

If you go back and look at the Puppet class, you’ll see Foreman has added a matcher for that host:

The same override button is available on a host group’s Parameters tab. For more complex logic, like matching on facts, use the Puppet Class page.

Advanced usage

Smart class parameters are based on the smart matchers technology, and have a number of advanced features such as validation and multiple data types. More about these can be found in the Smart Matchers section.

4.2.6 Smart Matchers

The same smart matching technology underpins both smart variables and smart class parameters, so is described below. It provides the following features for each parameter:

1. A default value that can be sent if no specific match is found.
2. An order of precedence for overrides, based on host attributes or facts.
3. A list of overrides (matchers).
4. Specifying a data type, allowing strings, integers and data structures to be passed natively to Puppet.
5. Optional validation of values.
6. Template processing of values for dynamic content.

Ordering

Overrides are processed in the order of precedence set in the Order field, from most to least specific (first match wins, unless merging is enabled). This is a list of host attributes and fact names that overrides will be checked against. If no override from this list matches, the default value is used.

Example attributes that may be listed are:

• fqdn - host’s FQDN (“host.example.com”)
• hostgroup - full name of the host group, including parents (“Europe/Web servers”). Matchers on host groups can be inherited by their children, see documentation for host_group_matchers_inheritance in configuration options.
• os - name and version of operating system (“RedHat 6.4”)
• domain - host’s domain name (“example.com”)
• location or organization - full name of the location/organization, including parents (“Company/Subsidiary”)
• is_virtual - a fact supplied by Facter

The default order is set under Administer > Settings > Puppet > Default_variables_Lookup_Path and is “fqdn”, “hostgroup”, “os”, “domain”.

Note that there’s a name conflict between the “operatingsystem” fact and Foreman’s attribute “operatingsystem” (same as “os” above), and Foreman’s attribute will be the one that is used, so will include the version number.

Overrides / matchers

Once defaults have been filled in for your parameter, you can then add criteria to match against - click the Add Matcher button under your parameter, and more input fields will appear:

As an example, let’s say that any machine in the “development” puppet environment should use a value of “foremandev” instead of “foreman” for the “user” parameter. Add “environment” to the end of the matchers list, then click the Add Matcher-Value button, and fill it out like this:

The match field currently supports string equality only, the values must match exactly.

Merging overrides

When the data type is a hash or array, ticking Merge overrides will cause values from every override that matches (e.g. an FQDN and domain) to be merged together.

Merging is “deep”, so nested hashes and arrays will gain values rather than being overwritten entirely.

The Merge default option adds the default value as one of the values to merge, it will get the least important priority so one of the other values may overwrite it.

When the data type is an array, the Avoid duplicates option will de-duplicate the resulting array.

Data types

The type of data we want to pass to Puppet can be set in the Parameter type field. Most commonly a string, but many other data types are supported:

• String - Everything is taken as a string.
• Boolean - Common representation of boolean values are accepted, including true, false, yes, no etc.
• Integer - Integer numbers only, can be negative.
• Real - Accept any numerical input.
• Array - A valid JSON or YAML input, that must evaluate to an array.
• Hash - A valid JSON or YAML input, that must evaluate to an object/map/dict/hash.
• YAML - Any valid YAML input.
• JSON - Any valid JSON input.

There’s no easy way to tell what type of data the Puppet manifest is expecting, so you will need to read through the code/documentation that comes with a particular module to find out. Changing the type field requires an appropriately set “Default Value” field.

Complex data

Here’s an example of adding an array parameter. Note the use of YAML in the edit box:

This will be converted to the JSON ["a","b"] syntax when you save. You can also use hashes in YAML or JSON as data types too.

Note that the JSON hash syntax is not the same as Puppet’s hash syntax: {"example":"value"}

Input validation

The Optional input validator section can be used to restrict the allowed values for the parameter. It is important to note that the validation applies to changes made from the Host edit page as well as the Puppet Classes edit page.

The input validation section is hidden by default but can be opened by clicking on its title. When changing the parameter type this section will be automatically expanded to change the validations according to the new type.

For example, to restrict the “user” field to either “foreman” or “foremandev”, tick the Required checkbox, and then set:

• Type: List
• Rule: foreman,foremandev

String validators

At present, the string type cannot be validated - leave the validator field blank, and all strings in the variable will be considered acceptable

Regexp / List validators

By entering a list (comma-separated, no spaces) or a regex (no delimiter required), the value to be assigned to the parameter will be checked against this list. If the value does not match the validator, and error will be raised.

Template variables

Because Foreman offers templating capabilities, you can utilise pre-existing variables, macros and or functions within your parameterized classes. This is especially useful if you need to send a string to Puppet/Chef, but have a need to embed host specific information within the string, such as the host’s FQDN.

Let’s look a quick example situation: we need to configure RabbitMQ and have it use our existing Puppet SSL certs. Using what we’ve learnt above, we jump into the RabbitMQ class and configure the “ssl cert” parameter as such:

As you can see we’re utilising a template variable within the parameter’s string just like we would in a normal template file. The important part of this string, as we’re sure you’ve gathered, is the “@host.name” element. This pulls the FQDN from Foreman’s facts and inserts it into the string.

More information regarding templates can be found on the wiki. This page also contains the pre-existing functions and macros you can use in your templates and parameter classes.

Examples

Example 1 - Simple change to a single host

All our hosts use server.foo for something, except bob.domain.com which uses server2.bar:

Example 2 - Change for a group of hosts (via custom fact) with validation and ordering

Most hosts need to use a port of 80 but all machines with a fact region and value europe need to use 8080. To do this, you have to add the factname (in this example region) to the searchlist:

Note that all machines will get either 80 or 8080 as required, except foo.domain which will generate an error, since 67 is not in the list validator. Note also that foo.domain will match before region, since it is higher in the searchlist. The rule ordering does not matter.

It is also possible to mix conditions, e.g.

4.3 Smart Proxies

The Smart Proxy is a project which provides a restful API to various sub-systems.

Its goal is to provide an API for a higher level orchestration tools (such as Foreman). The Smart proxy provides an easy way to add or extended existing subsystems and APIs using plugins.

Currently supported (Click on the links below for more details).

• DHCP - ISC DHCP and MS DHCP Servers
• DNS - Bind and MS DNS Servers
• Puppet - Any Puppet server from 0.24.x
• Puppet CA - Manage certificate signing, cleaning and autosign on a Puppet CA server
• Realm - Manage host registration to a realm (e.g. FreeIPA)
• Templates - Proxy template requests from hosts in isolated networks
• TFTP - Any UNIX based tftp server

If you require another sub system type or implementation, please add a new feature request or consider writing a plugin.

Once your smart proxy is running, each of the relevant sub systems needs to be configured via the settings.d/* files in the config directory.

4.3.1 Smart Proxy Installation

A smart proxy is an autonomous web-based foreman component that is placed on a host performing a specific function in the host commissioning phase. It receives requests from Foreman to perform operations that are required during the commissioning process and executes them on its behalf. More details can be found on the Foreman Architecture page.

To fully manage the commissioning process then a smart proxy will have to manipulate these services, DHCP, DNS, Puppet CA, Puppet and TFTP. These services may exist on separate machines or several of them may be hosted on the same machine. As each smart proxy instance is capable of managing all the of these services, there is only need for one proxy per host. In the special case of a smart proxy managing a Windows DHCP server, the host machine must be running Windows, it does not need to be the Microsoft DHCP server itself.

Packages

RPM and Debian packages are available, see the Install from Packages section for configuration and install the foreman-proxy package.

Source code

You can get the latest stable code from GitHub (via git).

git clone git://github.com/theforeman/smart-proxy.git -b 1.17-stable

System requirements

The smart proxy will run with the following requirements (aside from rubygem dependencies):

• Ruby 2.0, 2.1, 2.2, 2.3 or 2.4

Windows

The Microsoft smart-proxy installation procedure is very basic compared to the RPM or APT based solution. You need to run smart-proxy from the source as well as install Ruby and Ruby DevKit.

1. Run Ruby Installer and add the matching DevKit to compile native extensions. Make sure to add Ruby to %PATH%, you can select this option in the installer
2. Download / clone the smart proxy repository to a convenient location (see above, Source Code). Make sure to download / checkout the maching branch to your foreman installation
3. Open a command prompt (cmd.exe) and run the following commands in order:
   1. ruby <devKitRoot>\dk.rb init
   2. ruby <devKitRoot>\dk.rb install
   3. gem install --no-ri --no-rdoc bundler
   4. cd <smart-proxy location>
   5. bundle install --without development test krb5 puppet_proxy_legacy bmc libvirt

General configuration

1. Create the SSL certificate and key
   1. Login to your puppetmaster

   2. On the command line, type the following command. Take care not to use an alias nor upper case characters.

      puppet cert generate new-smart-proxy-FQDN
      

   3. Copy the private key, the public certificate and the ca.pem from /var/lib/puppet/ssl on your puppetmaster over to a location accessible by your new smart proxy, e.g. <smart-proxy location>\ssl\ (create the directory if necessary - this location will be referred to by the settings.yml in the next step)
2. Copy settings.yml.example inside config to settings.yml
3. At very least, modify the settings for :bind_host: and :log_file: and SSL, for example:

:bind_host: '0.0.0.0'
:log_file: 'C:\smart-proxy.log'

:trusted_hosts: [ foreman.example.com ]

:ssl_certificate: <smart-proxy location>\ssl\host.example.com.pem
:ssl_private_key: <smart-proxy location>\ssl\host.example.com.pem
:ssl_ca_file:     <smart-proxy location>\ssl\ca.pem

Test and configure smart proxy features

1. Test your configuration by setting :log_level: DEBUG and :log_file: STDOUT in config/settings.yml
2. Open an administrative command prompt and run bundle exec ruby <smart-proxy location>\bin\smart-proxy
3. Configure smart-proxy features like DNS and DHCP
4. Once everything runs well install a Windows service using ruby extra\register-service.rb to register the service Foreman Smart Proxy. Alternatively, use a third party tool like NSSM to create the service.

Caveats: There is an issue with DevKit not finding any ruby version installed. Check that the DevKit and Ruby Installer are both x32 or x64, otherwise add the missing versions manually by editing config.yml.

Puppet hint: If you have Puppet installed on the same host running smart-proxy, you can use Puppet’s Ruby. You only need DevKit. In this case, just add directory containing ruby.exe to your path variable and add it to DevKit settings if necessary by editing DevKit’s config.yml. Also, you might want to use Puppet’s host certificates right away for smart proxy SSL connections. Usually, they can be found in C:\ProgramData\PuppetLabs\puppet\etc\ssl. For example:

:ssl_certificate: C:\ProgramData\PuppetLabs\puppet\etc\ssl\certs\host.example.com.pem
:ssl_private_key: C:\ProgramData\PuppetLabs\puppet\etc\ssl\private_keys\host.example.com.pem
:ssl_ca_file:     C:\ProgramData\PuppetLabs\puppet\etc\ssl\certs\ca.pem

Configuration file

Usually can be found at /etc/foreman-proxy/settings.yml or in the config/settings.yml subdirectory. You can use the settings.yml.example file inside the config directory as a template for your own settings.yml.

Configuration of each subsystem is usually in /etc/foreman-proxy/settings.d/ or in the config/settings.d/ subdirectory. If you don’t plan to use one of the subsystems, please disable them in these configuration files. For more information see Smartproxy Configuration.

Start the daemon

bundle exec bin/smart-proxy

Or if you installed it via a package simply start the foreman-proxy service.

service foreman-proxy start

Add the Smart Proxy to Foreman

• Go to Foreman, under Infrastructure > Smart proxies, click New Proxy
• Type in the Name for your Proxy and the URL of your Proxy, with the port used

For example:

• Name: Puppet-Proxy
• URL: http://puppet.your-domain.com:8443

4.3.2 Smart Proxy Settings

The main configuration for the core Smart Proxy is held in the /etc/foreman-proxy/settings.yml or config/settings.yml file. This includes configuration of ports to listen on, SSL and security settings and logging options.

Each of the modules used in the Smart Proxy have their configuration in the /etc/foreman-proxy/settings.d/ or config/settings.d directory. Modules are enabled or disabled inside their respective configuration files with the :enabled directive, which determines whether the module is available on HTTP, HTTPS, both or is disabled (see below for more details).

YAML start

The first non-comment line of all configuration files must be three dashes.

---

Daemon configuration (settings.yml)

If daemon is present and true then the Smart Proxy will attempt to disconnect itself from the controlling terminal and daemonize itself on startup, writing its pid (process ID) into the specified file.

:daemon: true
:daemon_pid: /var/run/foreman-proxy/foreman-proxy.pid

Logging (settings.yml)

The proxy’s output is captured to the log_file and may be filtered via the usual Unix syslog levels:

• WARN
• DEBUG
• ERROR
• FATAL
• INFO
• UNKNOWN

See Ruby’s Logger class for details.

:log_file: /var/log/foreman-proxy/proxy.log
:log_level: DEBUG

The log_file setting may be set to “STDOUT” which causes log messages to be logged to standard output, for capture by the running process (e.g. systemd with journal). When log_file is set to “SYSLOG”, all messages will be sent to syslog.

A limited number of recent log messages are kept in memory using a ring buffer, which can be exposed in the API and to Foreman by enabling the Logs feature.

The number of all log messages is controlled by the log_buffer setting, and a second buffer of error messages is controlled by the log_buffer_errors setting. The total of the two will directly affect the maximum amount of memory used, which is approximately 500kB in the default configuration of 3,000 recent messages.

:log_buffer: 2000
:log_buffer_errors: 1000

Listening configuration (settings.yaml)

By default the Smart Proxy listens on all interfaces, which can be changed to limit access to a network:

# host to bind ports to (possible values: *, localhost, 0.0.0.0)
:bind_host: ['*']
:bind_host: private.example.com
:bind_host: 192.168.1.10

The Smart Proxy has a number of different modules which can be enabled either for HTTP, for HTTPS or for access on both services. It is highly recommended to enable most only on HTTPS and only enable modules on HTTP when required (e.g. templates) or if no SSL is desired.

The two port options control which TCP port(s) the Smart Proxy will listen on. At least one must be enabled for the proxy to start. It is recommended to only set https_port unless an HTTP-only module is active, which also requires the three ssl_* settings to be set.

:http_port: 8000
:https_port: 8443

Modules are enabled in their per-module configuration file in /etc/foreman-proxy/settings.d/ with the :enabled directive, which can be set to:

• :enabled: false to disable the module entirely
• :enabled: http to listen on HTTP only
• :enabled: https to listen on HTTPS only (recommended)
• :enabled: true to listen on both HTTP and HTTPS if enabled (not recommended)

Security configuration (settings.yml)

The existence of all the three ssl key entries below requires the use of an SSL connection.

NOTE that both client certificates need to be signed by the same CA, which must be in the ssl_ca_file, in order for this to work see SSL for more information

:ssl_certificate: ssl/certs/fqdn.pem
:ssl_ca_file: ssl/certs/ca.pem
:ssl_private_key: ssl/private_keys/fqdn.key

Specific SSL cipher suites can be disabled by using the :ssl_disabled_ciphers: option. For more information on which cipher suites are enabled by default and how to correctly disable specific ones, please see SSL cipher suites.

The TLS versions can be disabled if requiring a specific version. So while insecure SSLv3 and TLS v1.0 are disabled by default, setting the array of :tls_disabled_versions: to include 1.1 will disable this version, too.

This is the list of hosts from which the smart proxy will accept connections. For HTTPS connections, the name must match the common name (CN) within the subject DN and for HTTP connections, it must match the hostname from reverse DNS.

:trusted_hosts:
  - foreman.prod.domain
  - foreman.dev.domain

For HTTPS connections, the name must match the common name (CN) within the subject DN and for HTTP connections, it must match the hostname from reverse DNS. When :forward_verify is enabled (default: true) then the reverse lookup is verified against the forward lookup of the hostname (aka forward-confirmed reverse DNS/FCrDNS).

Some modules may allow connections from all hosts rather than only the trusted_hosts list, particularly if they intend to deal with requests directly from managed hosts rather than only from Foreman.

An empty trusted_hosts list will permit no hosts access:

:trusted_hosts: []

While if the setting is not specified, any host may make requests to the smart proxy, which permits management of any enabled modules and features.

Foreman communication (settings.yml)

Some modules make requests back to Foreman, e.g. when relaying requests from client hosts. The following setting changes the destination URL:

:foreman_url: https://foreman.example.com

And the following settings change the SSL certificates used to authenticate to Foreman and to verify its certificate. In a typical installation, Foreman and the Smart Proxy may both use certificates signed the same certificate authority, so these default to the values of the ssl_* settings defined above.

# SSL settings for client authentication against Foreman. If undefined, the values
# from general SSL options are used instead. Mainly useful when Foreman uses
# different certificates for its web UI and for smart-proxy requests.
:foreman_ssl_ca: /etc/foreman-proxy/ssl/certs/ca.pem
:foreman_ssl_cert: /etc/foreman-proxy/ssl/certs/fqdn.pem
:foreman_ssl_key: /etc/foreman-proxy/ssl/private_keys/fqdn.pem

4.3.3 BMC

Activate the BMC management module within the Smart Proxy instance. This allows users to trigger power management commands through the proxy to controlled hosts using IPMI or similar.

:enabled: https
:bmc_default_provider: freeipmi

Available providers are:

• freeipmi - for IPMI control using the freeipmi implementation
• ipmitool - using the ipmitool implementation
• shell - specialized provider for controlling the proxy server itself (used for Foreman Discovery)
• ssh - simple provider via SSH command shutdown with limited functionality (poweron does not work)

The credentials and addresses used to control hosts are passed from Foreman itself by adding a new network interface with the type set to “BMC” to hosts.

4.3.3.1 SSH BMC

The SSH BMC provider provides a limited level of BMC functionality by running commands over an SSH connection to the host using a trusted SSH key.

It has the following configuration options for authentication, for the remote SSH user and private SSH key:

:bmc_ssh_user: root
:bmc_ssh_key: /usr/share/foreman/.ssh/id_rsa

The following configuration options control the commands executed by the provider on the remote host:

:bmc_ssh_powerstatus: "true"
:bmc_ssh_powercycle: "shutdown -r +1"
:bmc_ssh_poweroff: "shutdown +1"
:bmc_ssh_poweron: "false"

No power on support is possible with this provider.

4.3.4 DHCP

4.3.4.1 dhcp.yml

Activate the DHCP management module within the Smart Proxy instance. This is used to query for available IP addresses (looking at existing leases and reservations), add new and delete existing reservations. It cannot manage subnet declarations, which should be managed by another means (e.g. puppet-dhcp).

The DHCP module is capable of managing the ISC DHCP server, Microsoft Active Directory and Libvirt instances.

Builtin providers are:

• dhcp_isc - ISC DHCP server over OMAPI
• dhcp_libvirt - dnsmasq DHCP via libvirt API
• dhcp_native_ms - Microsoft Active Directory using API

Extra providers are available as plugins and can be installed through packages. See the following pages for more information:

• List of smart proxy plugins
• Plugin installation documentation

To enable the DHCP module and enable a provider, dhcp.yml must contain:

:enabled: https
:use_provider: dhcp_isc

For providers from plugins, check the plugin documentation to determine the exact provider name.

The module manages a DHCP server on the local host by default, but for providers that can be run remotely, the server address can be changed:

:server: 127.0.0.1

Note that if the DHCP server is running remotely, some providers (notably ISC) require that the configuration files must be accessible to the Smart Proxy still. This can be achieved with a network file system, e.g. NFS.

All available subnets will be loaded and can be managed by default, but this can have a performance penalty. If only some subnets are used, specify them as follows in network_address/network_mask notation:

:subnets: [192.168.205.0/255.255.255.128, 192.168.205.128/255.255.255.128]

Each provider has its own configuration file in the same directory with its own settings, e.g. dhcp_isc.yml. This usually needs additional configuration after changing the use_provider setting.

dhcp_isc

The dhcp_isc provider uses a combination of the ISC DHCP server OMAPI management interface and parsing of configuration and lease files. This requires it to be run either on the same host as the DHCP server or to have network filesystem access to these files.

This provider requires the config and leases settings in the dhcp_isc.yml configuration file, which should be set to the location of the DHCP server config and lease files. On a Red Hat or Fedora server use:

:config: /etc/dhcp/dhcpd.conf
:leases: /var/lib/dhcpd/dhcpd.leases

On a Debian or Ubuntu DHCP server, use the following values instead:

:dhcp_config: /etc/dhcp3/dhcpd.conf
:dhcp_leases: /var/lib/dhcp3/dhcpd.leases

If the DHCP server is secured with an “omapi_key”, the following entries must be set with the same values:

:key_name: omapi_key
:key_secret: XXXXXXXX

If the DHCP server is listening on a non-standard OMAPI port (i.e. not 7911), then change this with:

:omapi_port: 7911

For DHCP servers running on a different host, change :server in the main dhcp.yml configuration file.

dhcp_native_ms

The native_ms provider manages reservations in Microsoft Active Directory via its native API.

Possible configuration options in dhcp_native_ms.yml are:

:disable_ddns: true

When disable_ddns is true (default), dynamic DNS updates will be disabled for all hosts that the smart proxy creates. This will slightly slow the host creation process but will ensure that the DHCP server will not create or delete DNS entries on behalf of these clients. It’s preferable to disable this feature at the scope level.

dhcp_libvirt

Provider that manages reservations and leases via dnsmasq through libvirt API. It uses ruby-libvirt gem to connect to the local or remote instance of libvirt daemon.

Possible configuration options in dhcp_libvirt.yml are:

# Libvirt network. Only one network is supported.
:network: default

# Libvirt connection. Make sure proxy effective user have permission to connect.
:url: qemu:///system

When configuring local or remote connections, make sure the foreman-proxy effective user has UNIX permissions to libvirt socket or ssh keys are deployed when using remote connection.

More information about using this provider is in the Libvirt section.

4.3.4.2 ISC DHCP

ISC implementation is based on the OMAPI interface, which means:

• No need for root permissions on your DHCP server
• No need to restart (or “sync”) your DHCP server after every modifications.

Configuration

• dhcpd configuration file: ensure you have the following line in your dhcpd.conf file (somewhere in the top first lines):

omapi-port 7911;

• configure the settings file to point to your dhcpd.conf and dhcpd.leases files (make sure they are readable by the smart-proxy user)
• make sure the omshell command (/usr/bin/omshell) can be executed by the smart-proxy user.
• make sure that /etc/dhcp and /etc/dhcp/dhcpd.conf has group foreman-proxy

Securing the dhcp API

The dhcpd api server will listen to any host. You might need to add a omapi_key to provide basic security.

Example generating a key (on CentOS):

yum install bind
dnssec-keygen -r /dev/urandom -a HMAC-MD5 -b 512 -n HOST omapi_key
cat Komapi_key.+*.private |grep ^Key|cut -d ' ' -f2-

1. Edit your “/etc/dhcpd.conf”:

    omapi-port 7911;
    key omapi_key {
    algorithm HMAC-MD5;
    secret "XXXXXXXXX"; #<-The output from the generated key above.
    };
    omapi-key omapi_key;
    

2. Make sure you also add the omapi_key to your proxy’s dhcp_isc.yml

3. Restart the dhcpd and foreman-proxy services

NOTE: if you don’t see DHCP in Smart Proxies Features, choose “Refresh features” from drop-down menu.

The next step is to set up appropriate Subnets in Foreman from the settings menu.

Sample dhcpd.conf

ddns-update-style interim;
ignore client-updates;
authoritative;
allow booting;
allow bootp;

omapi-port 7911;
#Optional key:
key omapi_key {
        algorithm HMAC-MD5;
        secret "2wgoV3yukKdKMkmOzOn/hIsM97QgLTT4CLVzg9Zv0sWOSe1yxPxArmr7a/xb5DOJTm5e/9zGgtzL9FKna0NWis==";
}
omapi-key omapi_key;

subnet 10.1.1.0 netmask 255.255.255.0 {
# --- default gateway
  option routers      10.1.1.254;
  option subnet-mask  255.255.255.0;

  option domain-name    "domain.com";
  option domain-name-servers  10.1.1.1, 8.8.8.8;
  option log-servers    syslog;
  option ntp-servers    ntp;

  range dynamic-bootp 10.1.1.10 10.1.1.250;
  default-lease-time 21600;
  max-lease-time 43200;

}

4.3.4.3 MS DHCP

It is required that this procedure is executed as an administrator.

It is not required that the smart proxy be on the same host as the MS DHCP server. The smart proxy just needs to be on a Windows host with connectivity to the DHCP server. If this is the case, make sure the smart proxy service runs as a user with sufficient privileges.

Note: Refer to the installation guide for general setup.

1. Edit config/settings.d/dhcp.yml so that it looks a bit like this. :server: can be left commented out if smart proxy runs on the same host.

   Sample config/settings.d/dhcp.yml file

   ---
   # Can be true, false, or http/https to enable just one of the protocols
   :enabled: https
   :use_provider: dhcp_native_ms
   :server: 10.10.10.1
   

2. If needed, you have to create the option 60 on the Windows DHCP (for PXE Boot)

   • Open an administrator command prompt

   • Create the PXE Option using netsh

     C:\Windows\system32>netsh
     netsh>dhcp
     netsh dhcp> server 10.10.10.1
     netsh dhcp server>add optiondef 60 PXEClient String 0 comment= PXE Support
     

4.3.5 DNS

4.3.5.1 dns.yml

Activate the DNS management module within the Smart Proxy instance. This is used to update and remove DNS records from existing DNS zones.

The DNS module can manipulate any DNS server that complies with the ISC Dynamic DNS Update standard and can therefore be used to manage both Microsoft Active Directory and BIND servers. Updates can also be made using GSS-TSIG, see the second section below. Additional providers are available for managing libvirt’s embedded DNS server (dnsmasq) and Microsoft Active Directory using dnscmd, for static DNS records, avoiding scavenging.

Builtin providers are:

• dns_nsupdate - dynamic DNS update using nsupdate
• dns_nsupdate_gss - dynamic DNS update with GSS-TSIG
• dns_libvirt - dnsmasq DNS via libvirt API
• dns_dnscmd - static DNS records in Microsoft Active Directory

Extra providers are available as plugins and can be installed through packages. See the following pages for more information:

• List of smart proxy plugins
• Plugin installation documentation

To enable the DNS module and enable a provider, dns.yml must contain:

:enabled: https
:use_provider: dns_nsupdate

For providers from plugins, check the plugin documentation to determine the exact provider name.

The default TTL of DNS records added by the Smart Proxy is 86400 seconds (one day). This can be changed with the dns_ttl setting:

:dns_ttl: 86400

Each provider has its own configuration file in the same directory with its own settings, e.g. dns_nsupdate.yml. This usually needs additional configuration after changing the use_provider setting.

dns_nsupdate

The dns_nsupdate provider uses the nsupdate command to make dynamic updates to the DNS server records. This works on a wide variety of RFC2136-compliant servers.

DNS servers that support Kerberos authentication, e.g. FreeIPA or Microsoft Active Directory, should use the dns_nsupdate_gss provider instead.

This provider has the following settings in the dns_nsupdate.yml configuration file:

#:dns_key: /etc/rndc.key
:dns_server: localhost

The dns_key specifies a file containing a shared secret used to generate a signature for the update request (TSIG record), thus authenticating the smart proxy to the DNS server.

If neither the dns_key or GSS-TSIG is used then the update request is sent without any signature. Unsigned update requests are considered insecure. Some DNS servers can be configured to accept only signed signatures.

The dns_server option is used if the Smart Proxy is not located on the same physical host as the DNS server. If it is not specified then localhost is presumed.

:dns_key: /etc/foreman-proxy/Kapi.+157+47848.private
:dns_server: dnsserver.site.example.com

dns_nsupdate_gss

For servers that support Kerberos/GSS-TSIG to authenticate DNS updates, the dns_nsupdate_gss provider should be used. This typically applies to FreeIPA and Microsoft Active Directory servers. This is equivalent to the nsupdate -g command.

This provider has the following settings in the dns_nsupdate_gss.yml configuration file:

:dns_tsig_keytab: /usr/share/foreman-proxy/dns.keytab
:dns_tsig_principal: DNS/host.example.com@EXAMPLE.COM

See the section on GSS-TSIG DNS below for steps on setting up the requisite accounts and keytabs with both AD and FreeIPA.

The dns_server option is used if the Smart Proxy is not located on the same physical host as the DNS server. If it is not specified then localhost is presumed.

:dns_server: dnsserver.site.example.com

dns_dnscmd

While the dns_nsupdate provider creates dynamic records in Active Directory, the dns_dnscmd provider uses the dnscmd tool to create static DNS records in AD, which are not affected by scavenging. This requires that the Smart Proxy is installed on a Windows server with dnscmd available.

The dns_server option is used if the Smart Proxy is not located on the same physical host as the DNS server. If it is not specified then localhost is presumed.

:dns_server: dnsserver.site.example.com

dns_libvirt

Provider that manages reservations and leases via dnsmasq through libvirt API. It uses ruby-libvirt gem to connect to the local or remote instance of libvirt daemon.

Possible configuration options in dns_libvirt.yml are:

# Libvirt network. Only one network is supported.
:network: default

# Libvirt connection. Make sure proxy effective user have permission to connect.
:url: qemu:///system

When configuring local or remote connections, make sure the foreman-proxy effective user has UNIX permissions to libvirt socket or ssh keys are deployed when using remote connection.

More information about using this provider is in the Libvirt section.

4.3.5.2 BIND

Bind configuration manipulation is based on nsupdate, which means that in theory could also be used to manipulate other dns servers which support nsupdate (such as Microsoft DNS server).

Configuration

In order to communicate securely with your dns server, you would need a key which will be used by nsupdate and your named daemon using ddns-confgen or dnssec-keygen

example using ddns-confgen

execute ‘ddns-confgen -k foreman -a hmac-md5’ - this should output something like the following:

# To activate this key, place the following in named.conf, and
# in a separate keyfile on the system or systems from which nsupdate
# will be run:
key "foreman" {
        algorithm hmac-md5;
        secret "GGd1oNCxaKsh8HA84sP1Ug==";
};

# Then, in the "zone" statement for each zone you wish to dynamically
# update, place an "update-policy" statement granting update permission
# to this key.  For example, the following statement grants this key
# permission to update any name within the zone:
update-policy {
        grant foreman zonesub ANY;
};

# After the keyfile has been placed, the following command will
# execute nsupdate using this key:
nsupdate -k /path/to/keyfile

You should create a new file (such as /etc/rndc.key or other) and store the key “foreman {…} in it. in the proxy Settings file you should point to this file location - make sure that the proxy have read permissions to this file.

In your named file, you could add the update-policy statement or something like this named example file if you need more fine grained permissions.

4.3.5.3 GSS-TSIG DNS

Both BIND as configured in FreeIPA and Microsoft AD DNS servers can accept DNS updates using GSS-TSIG authentication. This uses Kerberos principals to authenticate to the DNS server. Under Microsoft AD, this is known as “Secure Dynamic Update”.

Pre-requisites

• Kerberos principal in the realm/domain that Smart Proxy can use
• Kerberos keytab for the above principal
• Access to add/delete/modify the required zones in Microsoft DNS. Both forward and reverse lookup.

Microsoft AD configuration

A user has to be created in Active Directory that will be used by the Smart Proxy, e.g. foremanproxy. This will automatically create a service principal, e.g. foremanproxy@EXAMPLE.COM.

Test the Kerberos login with that user on the Smart Proxy using kinit:

kinit foremanproxy@EXAMPLE.COM

This requires that your SRV records in DNS or /etc/krb5.conf file is setup correctly. By default many systems use DNS to locate the Kerberos DC. A KDC can also be statically set in this file. There are dozens of documents on how to do this on the net.

If login works, the keytab file can be created using ktutil. First clear the Kerberos ticket cache:

kdestroy

Now create the keytab file with ktutil:

ktutil: addent -password -p foremanproxy@EXAMPLE.COM -k 1 -e RC4-HMAC
ktutil: wkt dns.keytab
ktutil: q

Once the keytab file has been created, test it using kinit:

kinit foremanproxy@EXAMPLE.COM -k -t dns.keytab

If this works, clear the Kerberos ticket cache once again using kdestroy.

Store the keytab at /etc/foreman-proxy/dns.keytab, ensure permissions are 0600 and the owner is foreman-proxy. If you are using SELinux, do not forget to update the file context.

The DNS zone Dynamic Updates option on the DNS zones can now be set to Secure Only.

Restart the smart proxy service. Next, go to Update the configuration in Foreman.

FreeIPA configuration

A service principal is required for the Smart Proxy, e.g. foremanproxy/proxy.example.com@EXAMPLE.COM.

First of all, create a new principal (FreeIPA service) for Foreman, e.g. ipa service-add foremanproxy/proxy.example.com@EXAMPLE.COM.

Then fetch the keytab, e.g. ipa-getkeytab -p foremanproxy/proxy.example.com@EXAMPLE.COM -s ipa-server.example.com -k /etc/foreman-proxy/dns.keytab.

Store the keytab at /etc/foreman-proxy/dns.keytab, ensure permissions are 0600 and the owner is foreman-proxy.

The ACL on updates to the DNS zone then needs to permit the service principal. In the FreeIPA web UI, under the DNS zone, go to the Settings tab, verify that “Dynamic update” for that zone is set to “True”, and add to the BIND update policy a new grant:

grant foremanproxy\047proxy.example.com@EXAMPLE.COM wildcard * ANY;

Note the \047 is written verbatim, and don’t forget the semicolon. ACLs should be updated for both forward and reverse zones as desired.

Proxy configuration

Update the proxy DNS configuration file (/etc/foreman-proxy/settings.d/dns.yml) with the following setting:

:use_provider: dns_nsupdate_gss

And the DNS GSS configuration file (/etc/foreman-proxy/settings.d/dns_nsupdate_gss.yml) with:

:dns_tsig_keytab: /etc/foreman-proxy/dns.keytab
:dns_tsig_principal: foremanproxy/proxy.example.com@EXAMPLE.COM

Ensure the dns_key setting is not specified, or is commented out.

Restart the smart proxy service. Next, go to Update the configuration in Foreman.

Update the configuration in Foreman

After you have added a DNS smart proxy, you must instruct Foreman to rescan the configuration on each affected smart proxy by using the drop-down menu by its name and selecting “Refresh Features”.

Now, you are allowed to enable this in each subnet (reverse lookup of domain) and domain (forward lookup of domain) that you want this smart proxy to assist. You do this by navigating there and selecting it in the drop-down menu for DNS.

4.3.6 Puppet

Activate the Puppet management module within the Smart Proxy instance. This module has two functions:

• reads the Puppet modules and manifests from the Puppet master, reporting the environments and classes that are declared, used when importing classes into Foreman
• optionally triggers immediate Puppet runs on clients using one of a number of implementations

It should be activated on Puppet masters that have the environments and modules available to import data from. To use the Puppet run functionality, it also needs to be capable of executing puppetrun or equivalent implementation listed in the section below. This works independently of the Puppet CA functionality, which may only be one of many Puppet masters in the environment.

To enable this module, make sure these lines are present in /etc/foreman-proxy/settings.d/puppet.yml:

:enabled: https
:puppet_version: 4.5.0

Replace 4.5.0 with the version of Puppet installed on the Puppet master, this will be used to determine which APIs and commands it supports. If Puppet is later upgraded, this version number should also be changed to match.

Puppet class/environment imports

The proxy generates a list of all Puppet classes and their parameters of the Puppet modules inside the environments declared in puppet.conf. These are imported by Foreman to generate the list of classes, smart class parameters and environments that they belong to. The mechanism used depends on the version of Puppet (specified by :puppet_version):

• On Puppet 4, the smart proxy will use the environments and environment_classes or resource_types API to list classes and parameters. environment_classes API is available and used on Puppet 4.4 or higher, resource_types API is used on 4.0 to 4.3.
• On Puppet 3.2 to 3.8 with directory environments (environmentpath), the smart proxy will use the environments Puppet Master API to list modulepaths and will load the Puppet library to parse manifests to compile a list of classes and parameters.
• Else on Puppet 2 or 3, the smart proxy will parse puppet.conf and load the Puppet library to parse manifests from environment modulepaths to compile a list of classes and parameters.

The sections below describe the configuration options for the different methods.

Puppet 4 configuration options (puppet_proxy_puppet_api.yml)

To get a list of environments, classes and their parameters, the proxy queries the Puppet master on its own API. The URL and settings used for the proxy to Puppet master API query can be controlled with the following settings in /etc/foreman-proxy/settings.d/puppet_proxy_puppet_api.yml:

# URL of the puppet master itself for API requests
#:puppet_url: https://puppet.example.com:8140
#
# SSL certificates used to access the puppet API
#:puppet_ssl_ca: /var/lib/puppet/ssl/certs/ca.pem
#:puppet_ssl_cert: /var/lib/puppet/ssl/certs/puppet.example.com.pem
#:puppet_ssl_key: /var/lib/puppet/ssl/private_keys/puppet.example.com.pem
#
# Smart Proxy api timeout when Puppet's environment classes api is used and classes cache is disabled
#:api_timeout: 30

This method is compatible with both Puppet Server and Puppet masters running under Rack/Passenger, but the Puppet master has to permit these API queries.

The default (older) auth.conf style of configuring the Puppet master is controlled by either /etc/puppetlabs/puppet/auth.conf (AIO) or /etc/puppet/auth.conf, and requires these rules:

path /puppet/v3/environments
method find
allow *

path /puppet/v3/environment_classes
method find
allow *

path /puppet/v3/resource_type
method search
allow *

The HOCON-formatted auth.conf style is at /etc/puppetlabs/puppetserver/conf.d/auth.conf and requires these rules:

{
    match-request: {
        path: "/puppet/v3/environments"
        type: path
        method: get
    }
    allow: "*"
    sort-order: 500
    name: "puppetlabs environments"
},
{
    match-request: {
    path: "/puppet/v3/environment_classes"
       type: path
       method: get
    }
    allow: "*"
    sort-order: 500
    name: "puppetlabs environment classes"
},
{
    match-request: {
        path: "/puppet/v3/resource_type"
        type: path
        method: [get, post]
    }
    allow: "*"
    sort-order: 500
    name: "puppetlabs resource type"
},

Puppet 2 or 3 configuration options (puppet_proxy_legacy.yml)

There are two ways to declare environments within Puppet. Config environments are environments explicitly declared in puppet.conf, either with a single “modulepath” setting (which creates a single “production” environment or may be a wildcard), or with [development] section headers. The proxy will parse puppet.conf in the same manner as Puppet to try and determine the known environments.

More information on configuring them is available in the Puppet environment docs. Since Puppet 3.5, these are deprecated in favor of directory environments.

Directory environments are enabled by adding “environmentpath” to puppet.conf. When the proxy finds this setting, it uses this mode too. The :use_environment_api proxy setting can be used to force this mode on or off, but when unset, it follows the presence of environmentpath (the default). More information on configuring directory environments is available in the Puppet docs.

To get a list of environments and module paths when using directory environments, the proxy queries the Puppet master on its own API. The path to puppet.conf plus the URL and settings used for the proxy to Puppet master API query can be controlled with the following settings in /etc/foreman-proxy/settings.d/puppet_proxy_legacy.yml:

#:puppet_conf: /etc/puppet/puppet.conf
#
# URL of the puppet master itself for API requests
#:puppet_url: https://puppet.example.com:8140
#
# SSL certificates used to access the puppet master API
#:puppet_ssl_ca: /var/lib/puppet/ssl/certs/ca.pem
#:puppet_ssl_cert: /var/lib/puppet/ssl/certs/puppet.example.com.pem
#:puppet_ssl_key: /var/lib/puppet/ssl/private_keys/puppet.example.com.pem

The Puppet master has to permit this API query. Older installations of Puppet that have been upgraded may need a new entry in auth.conf prior to the last ‘deny’ entry:

path /v2.0/environments
method find
allow *

When scanning Puppet manifests, a cache is kept in memory to speed up subsequent import calls. It can be enabled/disabled with the following setting:

# Cache options
:use_cache: true

Puppet run providers

For the optional Puppet run functionality, one of a number of implementations can be chosen in /etc/foreman-proxy/settings.d/puppet.yml.

:use_provider: puppet_proxy_puppetrun

Available providers are:

• puppet_proxy_puppetrun - for puppetrun/kick, deprecated in Puppet 3, not available in Puppet 4, see section below
• puppet_proxy_mcollective - uses mco puppet, see section below
• puppet_proxy_ssh - run puppet over SSH
• puppet_proxy_salt - uses salt puppet.run
• puppet_proxy_customrun - calls a custom command with args

Once a provider is configured, in Foreman itself, enable “puppetrun” under Administer > Settings > Puppet to activate the “Run Puppet” button on individual host pages.

puppetrun (deprecated)

puppet kick (or puppetrun in 2.x) can be used to trigger an immediate Puppet run on a client by connecting to the agent daemon on the client over HTTPS. This functionality is not available in Puppet 4 and was deprecated in Puppet 3, therefore isn’t recommended for new deployments - consider alternatives, e.g. SSH or MCollective.

More information can be found in the puppet kick documentation, specifically the Usage Notes which describe the configuration of the agents to listen and authorize connections.

Its configuration options are in /etc/foreman-proxy/settings.d/puppet_proxy_puppetrun.yml:

# User for execution of puppetrun commands
#:user: peadmin

The :user setting controls which user to sudo to, which on some installations (notably PE) may be different. When unset, it will sudo to root.

sudo access for the proxy is required to run the client - in your sudoers file ensure you have the following lines (use /opt/puppet/bin/puppet for Puppet Enterprise):

Defaults:foreman-proxy !requiretty
foreman-proxy ALL = NOPASSWD: /usr/bin/puppet kick *

If you are using Puppet 2.x, the proxy will use the older puppetrun command instead. The sudoers entry should be:

Defaults:foreman-proxy !requiretty
foreman-proxy ALL = NOPASSWD: /usr/sbin/puppetrun

MCollective

The proxy can trigger Puppet runs using the MCollective “puppet” agent. To enable this, add this line to /etc/foreman-proxy/settings.d/puppet.yml:

:use_provider: puppet_proxy_mcollective

The user that the smart proxy sudos to can be set, notably for PE, by editing /etc/foreman-proxy/settings.d/puppet_proxy_mcollective.yml:

# If you want to override the puppet_user above just for mco commands
:user: peadmin

If :user is not specified, it will sudo to root.

And add a sudoers rule for the user:

Defaults:foreman-proxy !requiretty
foreman-proxy ALL = NOPASSWD: /usr/bin/mco puppet runonce *

SSH

The puppet_proxy_ssh provider uses SSH to connect to the client using SSH keys and run the Puppet agent command directly. It is controlled by the following settings in /etc/foreman-proxy/settings.d/puppet_proxy_ssh.yml:

# the command which will be sent to the host
:command: /usr/bin/puppet agent --onetime --no-usecacheonfailure
#
# whether to use sudo before the ssh command
:use_sudo: false
#
# With which user should the proxy connect
#:user: root
#:keyfile: /etc/foreman-proxy/id_rsa
#
# wait for the command to finish (and capture exit code), or detach process and return 0
#:wait: false

The wait setting controls whether to block on completion of the Puppet command, so the result of the Puppet run can be returned to Foreman, else it’s usually asynchronous. When true, increase proxy_request_timeout under Administer > Settings in Foreman itself to ensure it waits longer for a response, as the Puppet run may take some time to complete.

Here is a howto on enabling the SSH feature.

On the foreman-proxy server, enable the SSH provider and uncomment the user and SSH key settings:

sed -i 's!^#\?\(:use_provider:\).*$!\1 puppet_proxy_ssh!' \
  /etc/foreman-proxy/settings.d/puppet.yml
sed -i 's!^#\?\(:user:\).*$!\1 root!' \
  /etc/foreman-proxy/settings.d/puppet_proxy_ssh.yml
sed -i 's!^#\?\(:keyfile:\).*$!\1 /etc/foreman-proxy/id_rsa!' \
  /etc/foreman-proxy/settings.d/puppet_proxy_ssh.yml

The puppet.yml configuration file should now contain:

:use_provider: puppet_proxy_ssh

And puppet_proxy_ssh.yml should now contain:

:user: root
:keyfile: /etc/foreman-proxy/id_rsa

The new SSH key needs to be generated by running:

ssh-keygen -t rsa -N '' -f /etc/foreman-proxy/id_rsa
chgrp foreman-proxy /etc/foreman-proxy/id_rsa
chmod 640 /etc/foreman-proxy/id_rsa

Then, foreman-proxy should be restarted. If using systemd, run:

systemctl restart foreman-proxy

Else, the good old service command should work:

service foreman-proxy restart

After that, you should make sure the user running foreman-proxy service has a valid home and a writable .ssh subdirectory. Check defined home directory by running

ls -d ~foreman-proxy

It should return /usr/share/foreman-proxy. Now we will create a .ssh subdirectory and set the proper permissions:

mkdir /usr/share/foreman-proxy/.ssh
chown foreman-proxy:foreman-proxy /usr/share/foreman-proxy/.ssh
chmod 700 /usr/share/foreman-proxy/.ssh

An additional step, but not required (security warning) is to disable host keys checking. It means any host allowing this SSH keys to connect will be reachable, without the need to run ssh root@client as foreman-proxy user to accept the host key. The goal here is to allow automatic deployment by only populating the authorized_keys file on new clients.

To enable such behavior, we will create a SSH client configuration file to disable host key checking, and disable storing of known keys:

echo -e "Host *\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null" > /usr/share/foreman-proxy/.ssh/config
chown foreman-proxy:foreman-proxy /usr/share/foreman-proxy/.ssh/config
chmod 600 /usr/share/foreman-proxy/.ssh/config

Finally, copy the foreman public key over the target client:

su - foreman-proxy -s /bin/bash
ssh-copy-id -i /etc/foreman-proxy/id_rsa root@client-addr

Salt

The salt provider uses Salt for remote execution, by executing the puppet module’s run routine. It is controlled by the following settings in /etc/foreman-proxy/settings.d/puppet_proxy_salt.yml:

# Custom salt puppet.run command
# Set :command to 'puppet.run agent no-noop' to run in no-noop mode.
# Default command is puppet.run.
:command: puppet.run

customrun

The customrun provider allows configuration of a script that implements the Puppet run action in any way you require. Set the following configuration in /etc/foreman-proxy/settings.d/puppet_proxy_customrun.yml:

# Set :customrun_cmd to the full path of the script you want to run, instead of /bin/false
:command: /bin/false
# Set :customrun_args to any args you want to pass to your custom script. The hostname of the
# system to run against will be appended after the custom commands.
:command_arguments: -ay -f -s

4.3.7 Puppet CA

Activate the Puppet CA management module within the Smart Proxy instance. This is used to manage the autosign configuration and handle listing, signing and revocation of individual certificates.

This should only be enabled in the Smart Proxy that is hosted on the machine responsible for providing certificates to your puppet clients. You would expect to see a directory /var/lib/puppet/ssl/ca on such a host.

:enabled: https

If your puppet SSL directory is located elsewhere, you’ll need to set ‘ssldir’ as well.

:ssldir: /etc/puppet/ssl

The ‘autosignfile’ setting is used to find autosign.conf:

:autosignfile: /etc/puppet/autosign.conf

The proxy requires write access to the puppet autosign.conf file, which is usually owner and group puppet, and has mode 0644 according to the puppet defaults.

Under a Puppet 4 AIO installation, paths should be set to:

:ssldir: /etc/puppetlabs/puppet/ssl
:autosignfile: /etc/puppetlabs/puppet/autosign.conf

Ensure the foreman-proxy user is added to the puppet group ( e.g. gpasswd -a foreman-proxy puppet or usermod -aG puppet foreman-proxy)

puppet.conf:

[master]
autosign = $confdir/autosign.conf {owner = service, group = service, mode = 664 }

sudo access for the proxy is required - in your sudoers file ensure you allow the “puppet cert” command with NOPASSWD and without requiretty.

Under a Puppet 4 AIO installation, configuration should be:

foreman-proxy ALL = NOPASSWD: /opt/puppetlabs/bin/puppet cert *
Defaults:foreman-proxy !requiretty

Under a non-AIO Puppet installation:

foreman-proxy ALL = NOPASSWD: /usr/bin/puppet cert *
Defaults:foreman-proxy !requiretty

For older versions of Puppet (2.x) with separate commands:

foreman-proxy ALL = NOPASSWD: /usr/sbin/puppetca *
Defaults:foreman-proxy !requiretty

4.3.8 Realm

4.3.8.1 realm.yml

Activate the realm management module within the Smart Proxy instance. This manages Kerberos realms or domains, allowing Foreman to add and remove hosts to enable them to join the realm/domain automatically during provisioning.

:enabled: https
:use_provider: realm_freeipa

Builtin providers are:

• realm_freeipa - host object management in FreeIPA

The configuration for each provider should be in its respective file, i.e: /etc/foreman-proxy/settings.d/realm_freeipa.yml.

The following settings control authentication of the proxy to the realm for management of hosts. In realm_freeipa.yml:

# Authentication for Kerberos-based Realms
:keytab_path: /etc/foreman-proxy/freeipa.keytab
:principal: realm-proxy@EXAMPLE.COM

4.3.8.2 FreeIPA Realm

The FreeIPA implementation of the realm proxy is able to add a host entry to FreeIPA, send the hostgroup name, and request a one-time registration password.

Configuration of FreeIPA

In order to create the realm user and keytab to authenticate to FreeIPA, you can use the included foreman-prepare-realm tool. Your Smart Proxy must be registered to the FreeIPA realm already, and have the ipa-admintools package installed.

Simply provide a user with admin rights in FreeIPA, and a target user to create.

# foreman-prepare-realm admin realm-proxy
Password for admin@EXAMPLE.COM:
---------------------------------------
Added permission "modify host password"
---------------------------------------
  Permission name: modify host password
  Permissions: write
  Attributes: userpassword
  Type: host
[...]
Keytab successfully retrieved and stored in: freeipa.keytab
Realm Proxy User:    realm-proxy
Realm Proxy Keytab:  /root/freeipa.keytab

Configuration of Smart Proxy

Copy the freeipa.keytab created above to /etc/foreman-proxy/freeipa.keytab and set the correct permissions:

    chown foreman-proxy /etc/foreman-proxy/freeipa.keytab
    chmod 600 /etc/foreman-proxy/freeipa.keytab

Then update settings.d/realm_freeipa.yml with the relevant settings.

If you’re using FreeIPA to manage DNS records, and want them to be automatically deleted when the host is deleted in Foreman, set this to true:

:remove_dns: true

Finally, trust the IPA Certificate Authority. Ensure you have the most up-to-date version of the ca-certificates package installed.

cp /etc/ipa/ca.crt /etc/pki/ca-trust/source/anchors/ipa.crt
update-ca-trust enable
update-ca-trust

You will need to disable the DNS proxy for hosts that are provisioned with a realm set, as FreeIPA adds the forward record for you. In order to support adding a reverse lookup record also, you will need to go into the settings for the forward lookup zone on the IPA server and tick Allow PTR sync. This will make sure that FreeIPA creates the PTR records for you.

Using Automember Rules

FreeIPA supports the ability to setup automember rules based on attributes of a system. When using the FreeIPA proxy, the Foreman host group is available as a parameter in FreeIPA known as userclass. Nested host groups are sent as displayed in the Foreman UI, e.g. “Parent/Child/Child”. Note that Foreman does send updates to FreeIPA, however automember rules are only applied at initial add. This will be coming in a future version of FreeIPA.

First, we create a host group in FreeIPA:

# ipa hostgroup-add webservers
Description: web servers
----------------------------
Added hostgroup "webservers"
----------------------------
  Host-group: webservers
  Description: web servers

Define an automember rule:

# ipa automember-add --type=hostgroup webservers
----------------------------------
Added automember rule "webservers"
----------------------------------
Automember Rule: webservers

Create an automember condition based on the userclass attribute:

# ipa automember-add-condition --key=userclass --type=hostgroup --inclusive-regex=^webserver webservers
----------------------------------
Added condition(s) to "webservers"
----------------------------------
  Automember Rule: webservers
  Inclusive Regex: userclass=^webserver
----------------------------
Number of conditions added 1
----------------------------

When a machine in Foreman is in the “webservers” host group, it will automatically be added to the FreeIPA “webservers” host group as well. FreeIPA host groups allow for Host-based access controls (HBAC), sudo policies, etc.

4.3.9 TFTP

tftp.yml

Activate the TFTP management module within the Smart Proxy instance. This is designed to manage files on a TFTP server, e.g. bootloaders for OS installation and PXE menu files.

The tftproot value is directory into which TFTP files are copied and then served from. The TFTP daemon will also be expected to chroot to this location. This component is only supported in a Unix environment.

:enabled: https
:tftproot: /var/lib/tftpboot
:tftp_servername: name of your tftp server (used for next server value in your dhcp reservation) - defaults to the host name of your proxy.

Unattended installation

An essential first step in netbooting a system is preparing the TFTP server with the PXE configuration file and boot images. This document assumes that you have already configured your DHCP infrastructure, either via manual configuration or through the DHCP smart proxy.

Setting Up the Proxy Server Host

Regardless of the filesystem setup is performed, you must also make sure you have the wget utility installed and in the default path. wget is used to download OS specific installation when a given host is enabled for the build process.

Automatic Setup

Foreman includes a TFTP server module that will perform all of the basic setup. It defaults to TFTP root of /var/lib/tftpboot, which may change if necessary. You will still need to provide the basic TFTP load images in your TFTP root directory. For vanilla PXE booting via PXELinux, this includes pxelinux.0, menu.c32, and chain.c32, for PXEGrub this includes grub2/ and grub/ subdirectories.

Manual Setup

The setup is very simple, and may be performed manually if desired.

1. The TFTP root directory must exist (we will use /var/lib/tftpboot in this example).
2. Populate /var/lib/tftpboot with PXE booting prerequisites. These can be taken from syslinux (usually in /usr/share/syslinux on RHEL) . At a minimum, this should include:
   • pxelinux.0
   • menu.c32
   • chain.c32
   • ldlinux.c32 if syslinux provides it
3. Populate the following prerequisites when PXE Grub bootloader is planned. These files can be found in OS distribution repositories, DVD/CD or packages (e.g. grub2-efi on Red Hats which installs into /boot/EFI). Alternatively, these files can be built from modules using grub2-mkimage or grub-mkimage and signed for SecureBoot support.
   • /var/lib/tftpboot/grub2 with grubx64.efi or grubia32.efi
   • /var/lib/tftpboot/grub with bootx64.efi or bootia32.efi
4. Create the directory /var/lib/tftpboot/boot and make it writeable by the foreman proxy user (foreman-proxy, for instance, when installing through a rpm package).
5. Create the directory /var/lib/tftpboot/pxelinux.cfg and make it writeable by the foreman proxy user (foreman-proxy).
6. Make sure /var/lib/tftpboot/grub and /var/lib/tftpboot/grub2 are both writeable by the foreman proxy user (foreman-proxy).
7. Verify SELinux labels when using SELinux.

• Note: if CentOS 7 is used, please make sure to edit the URL under Hosts -> Installation Media, to to exclude the $minor version. For example: http://mirror.centos.org/centos/$major/os/$arch

Setting Up Foreman

In most cases, the default templates should work fine. You do, however, need to make sure that a PXELinux or iPXE template is associated with your hosts. See Unattended Installations for details. The template will be used to define the PXE configuration file when a host is enabled for build.

Workflow

This is a rough outline of the steps triggered on the TFTP smart proxy host when you click on the “Build” link for a host.

1. Call mkdir -p /var/lib/tftpboot/pxelinux.cfg if it does not already exist.
2. Create a host-specific TFTP configuration file in /var/lib/tftpboot/pxelinux.cfg/01-XX-XX-XX-XX-XX-XX, named based off of the MAC address, using the associated PXE template.
3. Call mkdir -p /var/lib/tftpboot/boot if it does not already exist.
4. Download the OS specific kernel and initrd files using wget.
   1. The download URLs are derived from the installation media path, and OS specific log (see app/models/redhat.rb and debian.rb in foreman for examples of the gory details).
   2. The debian.rb file tries to guess if you want Ubuntu or Debian, based on the Name you give to your OS in the UI. If the name does not contain ‘ubuntu’ or ‘debian’, it may default to debian, hence fail to fetch the kernel/initrd.
   3. cd into /var/lib/tftpboot/boot and check that the filesizes are not zero. Check /var/log/foreman-proxy/proxy.log for possible errors.
5. The exact wget command is wget --no-check-certificate -nv -c <src> -O "<destination>"
6. At this point, the TFTP state is ready for the installation process.
7. Once the host has completed installation, the OS specific installation script should inform foreman by retrieving the built URL.
8. The host-specific TFTP configuration file is deleted.
9. The kernel and initrd are not deleted, but left in place for future installs of the same OS and architecture combination. Please note that in the unlikely case that these files are modified, the simplistic freshness check of wget will likely get confused, corrupting the downloaded versions of the files. If this happens, you should simply delete the files and let them be re-downloaded from scratch.

To make sure that you trigger the above workflow make sure you’ve satisfied these requirements:

1. at least 1 host is put in build mode
2. the host is using a subnet with a TFTP proxy

Limitations

At the moment, the proxy is not able to fetch boot files using NFS. As a workaround, expose your installation medium (or use a public mirror) over http/ftp to configure one machine with the require boot files. this would be resolved as part of #992.

Global default templates

You can build PXE default on TFTP proxy from Foreman UI from ‘Provisioning Templates’ page using ‘Build PXE Default’ button. You also have the ability to choose which templates are used for this action. Foreman exposes the following settings in the ‘Provisioning’ group for this purpose: Global default PXEGrub template, Global default PXEGrub2 template and Global default PXELinux template. When settings are empty, Foreman uses default values: PXELinux global default, PXEGrub global default and PXEGrub2 global default.

4.3.10 SSL

The smart proxy can work in SSL mode, where both sides verify and trust each other. Requests from Foreman will only be accepted if the SSL certificate can be verified. Since proxies abstract a high level of control over your infrastructure, the configuration and security of keys and certificates is important.

Using Puppet CA certificates

Since Foreman integrates with Puppet heavily, it is recommended to use the Puppet Certificate Authority (CA) to secure proxy access. See the Security Communciations with SSL section for more advanced installations (multiple or internal CAs).

If the smart proxy host is not managed by Puppet, you will need to generate a certificate - skip forward to the generate section.

When using Puppet’s certificates, the following lines will be required in puppet.conf to relax permissions to the puppet group. The foreman and/or foreman-proxy users should then be added to the puppet group.

[main]
privatekeydir = $ssldir/private_keys { group = service }
hostprivkey = $privatekeydir/$certname.pem { mode = 640 }

Configuring the proxy

Configure the locations to the SSL files in /etc/foreman-proxy/settings.yml, plus the list of trusted Foreman hosts:

:ssl_certificate: /var/lib/puppet/ssl/certs/FQDN.pem
:ssl_ca_file: /var/lib/puppet/ssl/certs/ca.pem
:ssl_private_key: /var/lib/puppet/ssl/private_keys/FQDN.pem

:trusted_hosts:
- foreman.corp.com
#- foreman.dev.domain

Under a Puppet 4 AIO installation, substitute above paths with /etc/puppetlabs/puppet/ssl/.

SSL cipher suites

By default, the smart proxy permits the following SSL cipher suites:

• ECDHE-RSA-AES128-GCM-SHA256
• ECDHE-RSA-AES256-GCM-SHA384
• ECDHE-RSA-AES128-CBC-SHA
• ECDHE-RSA-AES256-CBC-SHA
• AES128-GCM-SHA256
• AES256-GCM-SHA384
• AES128-SHA256
• AES256-SHA256
• AES128-SHA
• AES256-SHA

Please note, the smart proxy uses the OpenSSL suite naming scheme. For more information on suite names please see the OpenSSL docs.

Certain users may require to disable certain cipher suites due to security policies or newly discovered weaknesses. This can be done by using the :ssl_disabled_ciphers: option in /etc/foreman-proxy/settings.yml. For example:

:ssl_disabled_ciphers: ['AES128-SHA','AES256-SHA']

Generating a certificate

To generate a certificate for a proxy host that isn’t managed by Puppet, do the following:

1. Generate a new certificate on your puppetmaster: puppet cert --generate <proxy-FQDN>
2. Copy the certificates and key from the puppetmaster to the smart proxy in /etc/foreman-proxy:
   • /var/lib/puppet/ssl/certs/ca.pem
   • /var/lib/puppet/ssl/certs/proxy-FQDN.pem
   • /var/lib/puppet/ssl/private_keys/proxy-FQDN.pem

Under a Puppet 4 AIO installation, substitute above paths with /etc/puppetlabs/puppet/ssl/.

Follow the configuration section above, however use the /etc/foreman-proxy paths instead of the Puppet defaults.

Configuring Foreman

For Foreman to connect to an SSL-enabled smart proxy, it needs configuring with SSL certificates in the same way. If the Foreman system is managed by Puppet, it will already have these, else certificates can be generated following the above instructions.

The locations of the certificates are managed in the Settings page, under Provisioning with the ssl_ca_file, ssl_certificate and ssl_priv_key settings. By default these will point to the Puppet locations - for manually generated certificates, or non-standard locations, they may have to be changed.

Lastly, when adding the smart proxy in Foreman, ensure the URL begins with https:// rather than http://.

4.3.11 Libvirt

In this chapter, we will describe how to setup DHCP and DNS for use with the libvirt provider for dnsmasq.

This provider is able to change DHCP and DNS settings in libvirt with dnsmasq. The smart proxy directly connects to the libvirt daemon.

The provider is currently limited by the libvirt API which does not provide PTR records creation via the API itself, but dnsmasq automatically creates PTR record for the first A/AAAA entry. Therefore PTR lookups do work in the network, but it is not being created by Foreman orchestration.

The provider also returns active leases on systems with ruby-libvirt gem version 0.6.1 or higher.

Configuration of libvirt

Define the TFTP root first. Edit ‘default’ virtual network and add ‘tftp’, ‘bootp’ and ‘domain’ elements.

<network>
 <name>default</name>
 <uuid>16b7b280-7462-428c-a65c-5753b84c7545</uuid>
 <forward mode='nat'/>
 <bridge name='virbr0' stp='on' delay='0' />
 <domain name="local.lan"/>
 <dns>
 </dns>
 <mac address='52:54:00:a6:01:5d'/>
 <ip address='192.168.122.1' netmask='255.255.255.0'>
   <tftp root='/var/tftproot' />
   <dhcp>
     <range start='192.168.122.2' end='192.168.122.254' />
     <bootp file='pxelinux.0' />
   </dhcp>
 </ip>
</network>

Create a TFTP root directory, make sure it is writeable by the foreman proxy user (foreman-proxy for instance) and accessible to the account dnsmasq is running on (in Fedora this is nobody), set gid flag for newly copied files and copy necessary files to the new TFTP root directory:

mkdir -p /var/tftproot/{boot,pxelinux.cfg}
yum -y install syslinux
cp /usr/share/syslinux/{pxelinux.0,menu.c32,chain.c32} /var/tftproot
chown -R foreman-proxy:nobody /var/tftproot
find /var/tftproot/ -type d | xargs chmod g+s

Open up /etc/libvirt/libvirtd.conf file and configure foreman-proxy user to be able to connect to libvirt daemon:

unix_sock_group = "foreman-proxy"
unix_sock_rw_perms = "0770"

Alternatively (on development setups), you can turn off authentication:

auth_unix_rw = "none"

Configuration of smart-proxy

Configure the Smart Proxy settings under config/ to:

• enable tftp
• set correct tftp boot and set explicit tftp_servername
• enable dns libvirt provider
• enable dhcp libvirt provider
• default settings for both providers are sufficient (network named default on local libvirt daemon instance)

Important configuration values are, in tftp.yml:

:tftp: true
:tftproot: /var/tftproot
:tftp_servername: 192.168.122.1

in dns.yml:

:enabled: true
:use_provider: dns_libvirt

in dhcp.yml:

:enabled: true
:use_provider: dhcp_libvirt

and in dns_libvirt.yml and/or dhcp_libvirt.yml:

:network: default
:url: qemu:///system

Additional steps

Make sure the DNS server is configured with the foreman instance by setting /etc/resolv.conf file or changing this in NetworkManager or dnsmasq configuration. Example:

cat /etc/resolv.conf
nameserver 8.8.8.8
nameserver 8.8.4.4
nameserver 192.168.122.1

Foreman is now configured for libvirt provisioning, this is the recommended setup for git development checkouts.

4.3.12 Templates

In this chapter, we will describe how to setup a Smart Proxy to serve provisioning templates.

The smart proxy is able to proxy template requests from hosts in isolated networks to the Foreman server, when the proxy also handles TFTP.

Ensure the foreman_url in /etc/foreman-proxy/settings.yaml points to your Foreman instance, and that your smart proxy is listening on HTTP by uncommenting http_port. Now configure /etc/foreman-proxy/settings.d/templates.yml:

:enabled: true
:template_url: http://smart-proxy.example.com:8000

Once you’ve completed the above steps, restart the foreman-proxy service and refresh the features on your Foreman server.

The templates feature is used automatically: any host that uses this proxy for TFTP will also use the proxy to retrieve its templates.

4.3.13 Logs

The smart proxy’s logs module provides an API to retrieve recently logged messages and information about failed modules. This will be displayed in Foreman under the Smart Proxy pages when the module is enabled.

The module has no configuration options of its own, and is just enabled by configuring /etc/foreman-proxy/settings.d/logs.yml:

:enabled: https

Once enabled, restart the foreman-proxy service and refresh the features on your Foreman server.

The number of logs is controlled by the main smart proxy logging settings, detailed in Smart Proxy Settings.

4.4 Provisioning

This chapter details the configuration of the required UI components necessary to provision an OS onto a host.

4.4.1 Operating Systems

The Operating Systems page (Hosts -> Operating Systems) details the OSs known to Foreman, and is the central point that the other required components tie into.

Creating an Operating System

Simply click New Operating system on the main page. You will be taken to a screen where you can create the bare essentials of a new OS. Not everything required for a successful provision is on this page (yet) - the remaining components will appear for selection as we create them.

You will need to fill in the first few parts of the form (Name, Major, Minor, Family, and possibly some family-dependent information). In the case of OSs like Fedora, it is fine to leave Minor blank.

If the default Partition Tables & Installation media are suitable, then you can assign them now. If not, return here after each step in this chapter to assign the newly created objects to your Operating System

Auto-created Operating Systems

Foreman does not come with any Operating Systems by default. However, Foreman will detect the Operating System of any host which reports in via Puppet - if the OS of that Host is supported, it will be created (with very basic settings) and assigned to the Host. Thus you may find some OSs already created for you.

4.4.2 Installation Media

The Installation Media represents the web URL from where the installation packages can be retrieved (i.e. the OS mirror). Some OS Media is pre-created for you when Foreman is first installed. Be sure to select the closest or most local mirror and ensure that the OS Family is set correctly. The installer-less repositories published by some Linux distributions (e.g. fedora-updates) won’t work here .

New Installation Media

If your OS of choice does not have a mirror pre-created for you, click the New Medium button to create one. There are a few variables which can be used to pad out the URL. For example:

http://mirror.averse.net/centos/$major.$minor/os/$arch

Be sure to set the Family for the Media

Assign to Operating System

If you have not already done so, return to the Operating System page for your OS and assign the Media to it now.

Use case

Most sites are well-served with having a 1:1 relationship between operating system and installation media. This will obviously work. However, there is a bunch of use cases that make sense to have more than one set of installation media per operating system. The most common case of this is when a site has datacenters in multiple locations and would like to have the systems automatically select the mirror closed to their location. This can be enforced by only allowing the “correct” location for a given set of installation media.

Another possible use case is to use different of installation media for staging of different operating system versions. This is, however significantly more bumpy since this raises a bunch of issues with the version field in the actual operating system and might need other adjustments in the OS deployment process since, for example, the files that are actually used for the tftp boot during network installation are connected to the operating system and not the installation media set. Be prepared to change many templates if you decide doing so.

4.4.3 Provisioning Templates

The Provision Templates is the core of Foreman’s flexibility to deploy the right OS with the right options. There are several types of template, along with a flexible matching system to deliver different templates to different Hosts.

Foreman comes with pre-created templates for the more common OSs, but you will need to review these. All these templates are locked by default, hence they can not be modified. Most of them are customizable through parameters, but if you need some custom functionality, the recommended workflow is to clone the template and edit the clone. You can unlock the pre-created template and edit it directly, but note that any custom change will be overridden on any Foreman update. If you believe your change is worth of including in next Foreman release, please consider sending a patch to community-templates repository which we regularly synchronize with our codebase.

Template Kinds

There are several template kinds:

• PXELinux, PXEGrub, PXEGrub2 - Deployed to the TFTP server to ensure the Host boots the correct installer with the correct kernel options (also referred to as PXE templates)
• Provision - The main unattended installation file; e.g. Kickstart or Preseed
• Finish - A post-install script used to take custom actions after the main provisioning is complete
• user_data - Similar to a Finish script, this can be assigned to hosts built on user_data-capable images (e.g. Openstack, EC2, etc)
• Script - An arbitrary script, not used by default, useful for certain custom tasks
• iPXE - Used in {g,i}PXE environments in place of PXELinux (do not confuse with PXE templates above)

In practice, most environments only make use of the first 3. The Create Host action deploys the PXELinux template to the TFTP server. The PXELinux template directs the host to retrieve the Provision template. The Provision template will direct the installer to retrieve and run the Finish template at the end of the install, and the Finish template will notify Foreman the build is complete just before reboot.

Editing Templates

Unlocked templates can be edited from the Hosts > Provisioning templates menu, or from an existing host page under its Templates tab (which shows the templates in use).

The templates use the ERB (Embedded Ruby) templating language, allowing data from the host in Foreman to be added to the template output and for conditional content. The default templates make heavy use of the ERB feature, adding and changing the template behavior based on parameters, the operating system, or the networking configuration assigned to the host.

There are two general types of ERB syntax in templates. The <%= prefix outputs the value of the following expression into the rendered template, e.g. to output the hostname:

<%= @host.name %>

The <% prefix without the equals sign (=) is a general code block that may contain conditionals, variable assignments, or loops which are not output when rendered. Comments may also be in these blocks, prefixed with #.

<%
a_variable = @host.name
%>
<%= a_variable %>

Other examples of ERB syntax are given on the Help tab of the template editor, and many more examples are available on the Template Writing wiki page. The Help tab also lists available Global methods (functions) provided by Foreman such as foreman_url (the URL for unattended calls to Foreman), and template_name.

The methods available on the provided @host variable are limited by default (when safemode_render is enabled) to prevent exploitation of Foreman through templates. The permitted methods on all types of objects can be found in the Safe mode methods and variables table under the Help tab.

As noted in section 4.1.4 Auditing, changes to the templates are logged as diffs - you can browse the history of changes to the templates from the History tab within the Edit Template page. You can also revert changes here.

Template Association

When editing a Template, you must assign a list of Operating Systems which this Template can be used with. Optionally, you can restrict a template to a list of Hostgroups and/or Environments

When a Host requests a template (e.g. during provisioning), Foreman will select the best match from the available templates of that type, in the following order:

• Host-group and Environment
• Host-group only
• Environment only
• Operating system default

The final entry, Operating System default, can be set by editing the Operating System page.

Associating an Operating System default template

You will need to associate at least one PXE, Provision, and Finish template to your Operating System, and this must be done in two steps. First edit each of the templates, switch to the Association tab, and ensure the appropriate OSs are checked. Then edit the Operating System, switch to the Templates tab, and choose a default template for each template kind.

More than one PXE template can be associated. In this case, all associated PXE templates are deployed to the TFTP server and only one is picked up during provisioning according to the PXE Loader setting (see below).

Templates Details

For image based installs there are two methods to customize and finish the installation. Finish templates and User Data templates:

Finish Templates

Finish templates are available for all hypervisors that support image based installs where the foreman server can reach the newly installed machine via ssh and scp. The script generated from the finish template is copied by the Foreman to the newly installed system via scp using username and password specified with the image. It is then executed by connecting again via ssh, making the script executable and either executing it directly or via sudo if the specified user is not root. Standard output and standard error are logged to a file in the same directory named bootstrap-UUID.log.

User Data Templates

User Data Templates are available for hypervisors that support customization via tools like cloud-init. In this case the installed machine does not need be reachable via ssh by the Foreman server. However, the installed must be able to reach Foreman or a Smart Proxy with the templates feature via http(s) to notify the setup has finished.

PXE Loader

When creating a new Host, the PXE Loader option must be selected in order to pass the correct DHCP filename option to the client. One option out of the following must be chosen:

• PXELinux BIOS (loads pxelinux.0 filename from TFTP)
• PXELinux UEFI (loads pxelinux.efi filename from TFTP)
• PXEGrub UEFI (loads grub/bootx64.efi filename from TFTP)
• PXEGrub UEFI SecureBoot (loads grub/shim.efi filename from TFTP)
• PXEGrub2 UEFI (loads grub2/grubx64.efi filename from TFTP)
• PXEGrub2 UEFI SecureBoot (loads grub2/shim.efi filename from TFTP)
• None - no filename passed (e.g. for HTTP booting via iPXE)

Grub filenames are different for each individual architecture associated with the Host:

• grub/bootia32.efi (for Intel named "i*86" where * can be any character)
• grub/bootx64.efi (for Intel named "x86-64")
• grub2/grubia32.efi (for Intel named "i*86" where * can be any character)
• grub2/grubx64.efi (for Intel named "x86-64")
• grub2/grubaa64.efi (for ARM 64 named either "aa64" or "aarch64")
• grub2/grubppc64.efi (for IBM POWER named "ppc64”)
• grub2/grubppc64le.efi (for IBM POWER Little Endian named "ppc64le")
• grub2/grubXYZ.efi (for arbitrary Architecture named "XYZ")

Foreman installer only installs pxelinux.0 and grub2/grubx64.efi (if grub2 is available). In order to boot systems via other loaders like PXELinux EFI or Grub 1 (legacy), deploy the required bootloader files in the TFTP directory.

Some operating systems use a “shim” loader for SecureBoot (e.g. Red Hat Enterprise Linux and clones). To use SecureBoot with an operating system that does not use a shim chainloader, make a copy of the signed EFI loader named shim.efi or make a symlink in order to do secure boot.

4.4.4 Partition Tables

Partition templates are a subset of normal Provisioning Templates. They are handled separately because it is frequently the case that an admin wants to deploy the same host template (packages, services, etc) with just a different harddisk layout to account for different servers’ capabilities.

Foreman comes with pre-created templates for common Operating Systems, but it is good to edit the template, check it’s content and it’s Family setting. If the Family is wrong, be sure to go back to Operating Systems and associate it with your Operating System.

Per-Host Partition tables

When creating a new Host, you will be given the option to create an individual partition table. This is essentially a ‘one-off’ partition table that is stored with the host and used only for that host. It replaces the choice of Partition Table from the normal list of those associated with the selected OS.

Dynamic Partition tables

Some operating systems allow you to create partition tables via scripts. At the moment Kickstart and AutoYaST based systems can use this feature. Partition templates starting with #Dynamic are interpreted as scripts rather than static partition tables. The Provisioning Template needs to support this feature (search for @dynamic). This enables you to make choices on the fly during provisioning (or re-provisioning).

Kickstart Dynamic Partition tables

Kickstart will run dynamic partition tables as a pre-install bash script using a %pre scriplet. This script needs to create a complete Kickstart partition table in ‘/tmp/diskpart.cfg’. This partition table will then be read by anaconda for the installation by using %include /tmp/diskpart.cfg.

Example Dynamic Partition table:

#Dynamic - this line tells Foreman this is a script rather then a static layout
#This snippets define the swap partition size, it would generate a partition twice the size of the memory if your physical memory is up to 2GB
#or will create a swap partition with your memory size + 2GB.

#get the actual memory installed on the system and divide by 1024 to get it in MB
act_mem=$((`grep MemTotal: /proc/meminfo | sed 's/^MemTotal: *//'|sed 's/ .*//'` / 1024))

#check if the memory is less than 2GB then swap is double the memory else it is memory plus 2 GB
if [ "$act_mem" -gt 2048 ]; then
    vir_mem=$(($act_mem + 2048))
else
    vir_mem=$(($act_mem * 2))
fi

#copy all the HDD partitions to the temp file for execution
cat <<EOF > /tmp/diskpart.cfg
zerombr yes
clearpart --all --initlabel
part swap --size "$vir_mem" 
part /boot --fstype ext3 --size 100 --asprimary
part / --fstype ext3 --size 1024 --grow
EOF

AutoYaST Dynamic Partition tables

AutoYaST will run dynamic partition tables as a pre-install bash script. This script needs to create a AutoYaST XML file in /tmp/profile/modified.xml. The modified.xml file will be read by YaST after your script has finished. An example for getting the same functionality as with Kickstart would be to create your XML partition table in /tmp/diskpart.cfg and sed it together with the original AutoYaST XML like this: sed '/<\/ntp-client>/ r /tmp/diskpart.cfg' /tmp/profile/autoinst.xml > /tmp/profile/modified.xml. Inserting after ntp-client section is just a suggestion that uses the same style the community-templates do. The example uses a simplified version of the AutoYaST LVM Partition table template.

Example Dynamic Partition table:

#Dynamic - this line tells Foreman this is a script rather then a static layout
#This snippets define the swap partition size, it would generate a partition twice the size of the memory if your physical memory is up to 2GB
#or will create a swap partition with your memory size + 2GB.

#get the actual memory installed on the system and divide by 1024 to get it in MB
act_mem=$((`grep MemTotal: /proc/meminfo | sed 's/^MemTotal: *//'|sed 's/ .*//'` / 1024))

#check if the memory is less than 2GB then swap is double the memory else it is memory plus 2 GB
if [ "$act_mem" -gt 2048 ]; then
    vir_mem=$(($act_mem + 2048))
else
    vir_mem=$(($act_mem * 2))
fi

#copy all the HDD partitions to the temp file for execution
cat <<EOF > /tmp/diskpart.cfg
  <partitioning config:type="list">
    <drive>
      <device>/dev/sda</device>
      <initialize config:type="boolean">true</initialize>
      <partitions config:type="list">
        <partition>
          <create config:type="boolean">true</create>
          <filesystem config:type="symbol">ext3</filesystem>
          <format config:type="boolean">true</format>
          <mount>/boot</mount>
          <partition_id config:type="integer">131</partition_id>
          <partition_nr config:type="integer">1</partition_nr>
          <size>1G</size>
          <stripes config:type="integer">1</stripes>
          <stripesize config:type="integer">4</stripesize>
          <subvolumes config:type="list"/>
        </partition>
        <partition>
          <create config:type="boolean">true</create>
          <format config:type="boolean">false</format>
          <lvm_group>vg00</lvm_group>
          <partition_id config:type="integer">142</partition_id>
          <partition_nr config:type="integer">2</partition_nr>
          <size>max</size>
        </partition>
      </partitions>
      <pesize></pesize>
      <type config:type="symbol">CT_DISK</type>
      <use>all</use>
    </drive>
    <drive>
      <device>/dev/vg00</device>
      <initialize config:type="boolean">true</initialize>
      <partitions config:type="list">
        <partition>
          <create config:type="boolean">true</create>
          <filesystem config:type="symbol">ext3</filesystem>
          <format config:type="boolean">true</format>
          <lv_name>root</lv_name>
          <mount>/</mount>
          <size>10G</size>
        </partition>
        <partition>
          <create config:type="boolean">true</create>
          <filesystem config:type="symbol">swap</filesystem>
          <format config:type="boolean">true</format>
          <lv_name>swap</lv_name>
          <mount>swap</mount>
          <size>${vir_mem}M</size>
        </partition>
      </partitions>
      <type config:type="symbol">CT_LVM</type>
      <use>all</use>
    </drive>
  </partitioning>
EOF
sed '/<\/ntp-client>/ r /tmp/diskpart.cfg' /tmp/profile/autoinst.xml > /tmp/profile/modified.xml

Configuration

A partition table entry represents either

• An explicit layout for the partitions of your hard drive(s). E.G.

  zerombr
  clearpart --all --initlabel
  part /boot --fstype ext3 --size=100 --asprimary
  part /     --fstype ext3 --size=1024 --grow
  part swap  --recommended
  

• A script to dynamically calculate the desired sizes. E.G.

#Dynamic - The below code is to manage the swap size

#get the actual memory installed on the system and divide by 1024 to get it in MB
usable_ram=$((`awk '$1 ~ /^MemTotal/ {printf "%d\n", $2 / 1024}' /proc/meminfo`))

#check if the memory is less than 2GB then swap is double the memory else it is maximum 24 G for really inactive stuff.
if [ "$usable_ram" -le 2048 ]; then
  swap_size=$(($usable_ram * 2))
else
  swap_size=$(($usable_ram + 2048))
fi
if [ $swap_size -gt 24576 ] ; then
  swap_size=24576
fi

#copy all the HDD partitions to the temp file for execution
cat << EOF > /tmp/diskpart.cfg
zerombr
clearpart --all --initlabel
part swap --size 250 --maxsize "$swap_size" --grow
part /boot --fstype ext3 --size 100 --asprimary
part / --fstype ext3 --size 8192 --maxsize 12288 --grow
part /tmp2 --size 250 --fstype ext3 --grow
EOF

The inclusion of the keyword string #Dynamic at the start of a line lets Foreman know that this is not an explicit disk layout and must treated as a shell script, executed prior to the install process and that the explicit partition table will be found at /tmp/diskpart.cfg during the build process.

The dynamic partitioning style is currently only available for the Red Hat family of operating systems, all others must provide an explicit list of partitions and sizes.

You may also associate one or more operating systems with this partition table or alternatively set this up later on the Operating systems page.

4.4.5 Architectures

Architectures are simple objects, usually created by Foreman automatically when Hosts check in via Puppet. However, none are created by default, so you may need to create them if you’re not using Foreman for reporting.

Creating a new Architecture

Simply click New Architecture to create a new one. This should match the Facter fact :architecture e.g. “x86_64”. If you’ve already created some Operating Systems, you can associate the Architecture with the OS now; if not, the list of Architectures will be present when you create an OS.

4.4.6 Workflow

Foreman performs a number of orchestration steps when performing unattended installation or provisioning, which vary depending on the integration options chosen - e.g. use of compute resources, configuration management tool and provisioning method (network/PXE/image).

4.4.6.1 Example: PXE booting an instance with Salt configuration

The following example uses:

• Compute resource (oVirt), also applicable to Libvirt and VMware
• Network (PXE) provisioning with DHCP and TFTP orchestration
• DNS orchestration
• Salt configuration management, also applicable to Puppet

Steps

1. On the New Host page, the default VM configuration is shown and compute profiles can be applied.
2. An unused IP address is requested from the DHCP smart proxy associated with the subnet.
3. The IP address field is filled in on the New Host page.
4. n/a
5. The New Host page is submitted.
6. Foreman contacts the compute resource to create the virtual machine.
7. The compute resource creates a virtual machine on a hypervisor.
8. The VM’s MAC address is returned from the compute resource and stored on the host.
9. A reservation is created on the DHCP smart proxy associated with the subnet.
10. DNS records are set up:
    1. A forward DNS record is created on the smart proxy associated with the domain.
    2. A reverse DNS record is created on the DNS smart proxy associated with the subnet.
11. A PXELinux menu is created for the host in the TFTP smart proxy associated with the subnet.
12. Foreman contacts the compute resource to power on the VM.
13. The compute resource powers up the virtual machine.
14. The host requests a DHCP lease from the DHCP server.
15. The DHCP lease response is returned with TFTP options (next-server, filename) set.
16. The host requests the bootloader and menu from the TFTP server.
17. The PXELinux menu and OS installer for the host is returned over TFTP.
18. The installer requests the “provision” template/script from Foreman.
19. Foreman renders the template and returns the resulting kickstart/preseed to the host.
20. Autosigning configuration for Salt (or Puppet) is added on the Salt or Puppet CA smart proxy.
21. The installer notifies Foreman of a successful build in the postinstall script.
22. The PXELinux menu is reverted to a “local boot” template.
23. The host requests its configuration from Salt or Puppet.
24. The host receives appropriate configuration using data defined in Foreman.
25. Configuration reports and facts are sent from Salt or Puppet to Foreman and stored.

4.4.6.2 Example: Cloud-based provisioning via cloud-init

This example shows workflow of cloud-based provisioning on IaaS (e.g. OpenStack) via cloud-init which configures Puppet agent to finish off the configuration.

4.4.6.3 Example: Cloud-based provisioning via ssh

This example shows workflow of cloud-based provisioning on IaaS (e.g. OpenStack) via ssh finish template for Puppet agent configuration.

4.4.6.4 Example: Image-based provisioning via ssh

This example shows workflow of image-based provisioning on virtualization (e.g. oVirt, RHEV or VMware via ssh finish template for Puppet agent configuration.

4.4.6.5 Example: Anaconda PXE-based provisioning

This example shows workflow of PXE booting into Anaconda installer with Puppet agent configuration.

4.4.7 Networking

Foreman can store information about the networking setup of a host that it’s provisioning, which can be used to configure the virtual machine, assign the correct IP addresses and network resources, then configure the OS correctly during the provisioning process.

This section details the options available for network interfaces in the New Host form and how they’re used.

Host interface options

• Primary: each host must have a single primary interface, which is used as the name of the host itself. This should be considered the main hostname and would usually also carry the default route.
• Provision: the interface on which provisioning of the operating system should be carried out. This will be used for PXE (if applicable) with TFTP menus, running SSH finish scripts etc. This may be different to the primary interface and should have access to Foreman and other provisioning systems.
• Managed: whether Foreman orchestrates creation of DNS entries, DHCP reservations and configuration of the interface during provisioning. Unmanaged interfaces would be used for informational purposes only.

A simple, single-homed host would have one network interface with a DNS name set matching the hostname, then managed, primary and provision flags all ticked. This would create one interface with DNS and DHCP records (if configured) over which the OS would be set up.

A dual or multi-homed host could have one interface with primary enabled (“host.example.com”) and another network with provision enabled (“host-build.example.com”). If both are also managed, Foreman will create DNS and DHCP records for both, but on the provision interface, the next-server/filename options for PXE will also be set. A TFTP (PXELinux) menu would also be created for the provision interface’s MAC address so the host can PXE boot on that physical interface, while its hostname would be assigned from the primary interface.

Virtual machines and interfaces

When Foreman deploys a host onto a compute resource, it creates a new interface on the VM for each interface specified when creating the host.

After creation, Foreman reads back the network information and matches the created interfaces to the list of interfaces given for the host and stores the assigned MAC and IP addresses (depending on the compute resource type) in its database. It then continues with orchestration, creating DNS and DHCP records etc. for the addresses retrieved from the new VM. Once orchestration of these is complete, it powers up the VM.

This design alleviates the need to supply MAC addresses for hosts being created on compute resources.

Subnet options

Subnets are defined in Foreman under Infrastructure > Subnets, and have a few options that affect how hosts are provisioned.

• IPAM: DHCP will use a DHCP-enabled smart proxy, checking for assigned leases and reservations and suggesting a new IP from the range. Internal DB will use Foreman’s list of already-assigned IPs and doesn’t rely on a DHCP smart proxy. None disables auto-suggestion of IP addresses.
• Boot mode: during OS provisioning, the template will configure the interface with either a static IP address or to use DHCP depending on the value of this setting.

Various combinations of the IPAM and Boot mode settings make sense, but the most common are DHCP (IPAM) with DHCP (Boot mode) and Internal DB with Static.

Use within provisioning templates and Puppet

Provisioning templates (such as kickstart, preseed or finish scripts) can make use of the interfaces data stored in Foreman for the host to configure the network.

A snippet (“kickstart_networking_setup”) is supplied by default in Foreman for kickstart-based OSes, which configures all managed network interfaces after the main OS installation is complete. This can be used in the %post kickstart section. No template is currently available for preseed-based OSes (ticket)

A hash of interfaces data is also made available to Puppet via a global ENC parameter called “foreman_interfaces”. This can be used to fully configure the network from a Puppet run.

4.4.8 OS Specific Notes

This chapter contains information about provisioning specific operating systems.

4.4.8.1 FreeBSD

As the FreeBSD installer itself does not support a kickstart-like pulling of a response file, a custom mfsBSD image with zfsinstall is used. Prebuilt images are available for download to be placed into the boot directory of your TFTP server.

However, these images can also be built from scratch as described below:

Building an installer image

This is an example how to build an image for FreeBSD 10.2-RELEASE. For other releases, simply replace the version accordingly.

# fetch the necessary FreeBSD components
mkdir /tmp/basefiles
cd /tmp/basefiles
fetch ftp://ftp.freebsd.org/FreeBSD/releases/amd64/10.2-RELEASE/base.txz
fetch ftp://ftp.freebsd.org/FreeBSD/releases/amd64/10.2-RELEASE/kernel.txz

# clone the git repository
git clone https://github.com/theforeman/mfsbsd.git
cd mfsbsd

# build the image (you need to be root for this to succeed)
make BASE=/tmp/basefiles RELEASE=10.2-RELEASE ARCH=amd64

# copy the image into the TFTP server directory
cp mfsbsd-10.2-RELEASE-amd64.img /tftpboot/boot/FreeBSD-x86_64-10.2-mfs.img

4.4.8.2 SLES

The installation media URL has to contain the contents of the first SLES DVD, it’s easiest to loopback mount the ISO image on a webserver. For Puppet, the systemsmanagement:puppet repository on OBS is used.

SLES 11

For Puppet, in addition to systemsmanagement:puppet also the devel:languages:ruby:backports repository on OBS and the SLES SDK DVD is used. The placeholder in the AutoYaST SLES template has to be updated with the actual SDK URL.

4.4.8.3 Windows

Provisioning Windows is a two step process. The first step, creating Installation Media, is not discussed here. It includes getting the WIM files, updates and drivers and boot files ready. The necessary boot files are are later downloaded by automatically by the smart proxy.

Tasks break down

• Change / add a new Architecture and OS
• Edit provision templates
• Add installation media
• Edit partition table
• Add parameters
• Link provisioning templates to OS

Architecture and OS

In Hosts > Architectures add a new architecture:

• Name: x64

Add a new OS in Hosts > Operating systems if needed. If you already have Windows hosts and with Puppet installed, the correct OS and architecture will have been auto created already. This example covers Windows 8.1 / Windows Server 2012R2.

• Name: windows
• Major: 6
• Minor: 3
• OS family: windows
• Description: Windows8
• Root password hash: Base64
• Architectures: x64

Take special care to set Root password hash = Base64. The templates do not render correctly if this is set otherwise. Changing the encoding does not apply to existing hosts.

Note: Foreman’s Safe Mode does prevent using the password directly. Thus, the string AdminPassword needs to be appended to your password when adding a new host. Eg P@55w0rd would become P@55w0rdAdminPassword. This can be automated however by replacing the password part with a ruby function and disabling safe mode render.

Add provision templates

Head to Hosts > Provisioning Templates and edit the templates starting with WAIK to meet your needs. Make sure to get the latest version of the WAIK templates from the community templates project. Assign each of those templates to your Windows OS (does not apply to snippets). The naming of the templates is a suggestion and up to you. This does not apply to snippets! There, the name is important.

Note: You can find more info about Foreman Template Writing in the Wiki.

Required templates

WAIK Finish

• Name: WAIK Finish
• Kind: Finish

WAIK unattend.xml

• Name: WAIK unattend.xml
• Kind: Provision

WAIK peSetup.cmd

• Name: WAIK peSetup.cmd
• Kind: Script

Note: To get the download folders nicely, the wget commands in this template might need tweaking. This could especially be necessary if you intend to use the extraFinishCommands snippet. Eg, --cut-dirs=3 would cut the first three directories form the download path when saving locally. This way http://winmirror.domain.com/pub/win81x64/extras/puppet.msi will be stripped of pub/win81x64/extras and download to puppet.msi.

WAIK PXELinux

• Name: WAIK PXELinux
• Kind: PXE Linux

Optional templates

WAIK joinDomain.ps1

• Name: WAIK joinDomain.ps1
• Kind: User Data

WAIK local users

• Name: WAIK local users
• Kind: Snippet

Note: This snippet creates extra users in the unattended stage. This may be very useful for debugging early stages of your deployment; since you can find yourself locked out of the newly provisioned host.

Microsoft does not really care about password security in unattend.xml files; so it does not really matter if you use <PlainText>true</PlainText> or not. If you prefer the encoded form, you need to append the string Password to your user password and encode it to Base64. The following ruby function is an example, it creates the encoded from of P@55w0rd:

Base64.encode64(Encoding::Converter.new("UTF-8", "UTF-16LE",:undef => nil).convert("P@55w0rd"+"Password")).delete!("\n").chomp

WAIK extraFinishCommands

• Name: WAIK extraFinishCommands
• Kind: Snippet

Note: The commands here are executed at the last stage just before finishing host building. Make sure they get executed in a synchronous way (eg. do not run in background like msiexec). Otherwise the following reboot might kill them.

WAIK OU from host group

• Name: WAIK OU from host group
• Kind: Snippet

Note: This snippet may be used to generate the computer OU from the host’s host group and domain.

Example: Given a host example in domain ad.corp.com and in host group servers/windows/databases. The snippet generates the OU path: OU=databases,OU=windows,OU=servers,DC=ad,DC=corp,DC=com. Optionally, set the host parameter computerOuSuffix to add some arbitrary OU at the end.

Add installation media

For each of your Windows versions add a new installation medium pointing to the root of the folder containing boot and sources Eg, http://winmirror.domain.com/pub/win81x64. Assign them to your operating system.

Modify partition table

The default partition table is a simple diskpart.exe script. It will wipe Disk 0

Define templates

Link all the created templates as well as the installation media and partition table to the OS:

1. Head to your OS, then provisioning
2. Select the template from each kind from the drop down list
3. In partition tables, select WAIK default
4. In installation media, check the appropriate installation media added above.

Add Parameters

To render the templates correctly, some parameters need to be added. They need to be available as global/host parameters. Most of them make the most sense as parameter on the OS. Most parameters are not required and have defaults. For the most up to date description see the template itself.

Important parameters

Required

• windowsLicenseKey: Valid Windows license key or generic KMS key
• windowsLicenseOwner: Legal owner of the Windows license key
• wimImageName: WIM image to install from a multi image install.wim file

Note: The correct value for wimImageName depends on your install.wim. The provisioning will fail an incorrect value is supplied for a multi image WIM file and gets silently ignored if the image contains one image only.

Optional The following parameters are only applied if they exist. Some, like domainJoinAccount and domainJoinAccountPasswd require each other.

• systemLocale: en-US
• systemUILanguage: en-US
• systemTimeZone: Pacific Standard Time - see MS TimeZone Naming
• localAdminAccountDisabled: false - will keep the local administrator account disabled (default windows)
• ntpSever: time.windows.com,other.time.server - ntp server to use
• domainJoinAccount: administrator@domain.com - use this account to join the computer to a domain
• domainJoinAccountPasswd: Pa55w@rd - Password for the domain join account
• computerOU: OU=Computers,CN=domain,CN=com - Place the computer account in specified Organizational Unit
• computerOuSuffix: Used if computerOU is not present to generate the computer OU from host group and host domain. computerOU takes precedence! Note, the OU must still be manually created in active directory
• computerDomain: domain.com - domain to join

Troubleshooting

Templates

The templates most likely need a lot of testing to work. This is not covered here; though some hints how to start. You should proceed in this order:

1. Get your templates to render correctly. Create random Bare Metal host in the desired host group for this purpose and make extensive use of Foreman’s template preview feature.
2. Continue tesing with VMs to test netbooting and basic installation
3. Debug peSetup.cmd by pausing it at the send (remove the comment from ::PAUSE). Then, use Ctrl-C to cancel the script altogether. This way you can debug the rendered peSetup.cmd quite nicely in WinPE (eg, notepad peSetup.cmd)
4. The WAIK Finish template uses sDelete.exe to remove all rendered commands from the provided host. Comment out all sDelete commands to debug finish scripts.
5. Use a manual installed host to test rendered snippets like WAIK extraFinishCommands directly.
6. Examine C:\foreman.log. - the output left from the finish script. Also, comment out the clean up stage in the finish script to examine and test the rendered scripts directly.

Netbooting

Sometimes wimboot seems not to be able to boot our winPE.wim. Symptoms range from black screens to kernel panics (aka BSOD). These problems seem to be more likely on older hardware.

In this case a workaround can be to simply use any other bootable media like USB thumb drives and CD-ROMs. The process is relatively simple:

1. Use a common tool Media Creator to create a bootable medium like a USB stick or ISO image. Since we do not use the image downloaded by the tool, the only important choice is architecture.
2. In the image or the USB drive, replace sources/boot.wim with the version from the installation media.
3. Boot from the medium.

4.5 Command Line Interface

The framework used for implementation of command line client for foreman provides many features common for modern CLI applications. The task of managing Foreman from command line is quite complex so the commands have to be organized in more levels of subcommands. There is help available for each level to make it easy to use. Some other features for greater comfort are option validation, logging and customizable output formatting.

4.5.1 Usage Examples

Basic help and list of supported commands:

$ hammer -h
Usage:
    hammer [OPTIONS] SUBCOMMAND [ARG] ...

Parameters:
    SUBCOMMAND                    subcommand
    [ARG] ...                     subcommand arguments

Subcommands:
    architecture                  Manipulate architectures.
    auth                          Foreman connection login/logout.
    compute-resource              Manipulate compute resources.
    domain                        Manipulate domains.
    environment                   Manipulate environments.
    fact                          Search facts.
    filter                        Manage permission filters.
    global-parameter              Manipulate global parameters.
    host                          Manipulate hosts.
    hostgroup                     Manipulate hostgroups.
    location                      Manipulate locations.
    medium                        Manipulate installation media.
    model                         Manipulate hardware models.
    organization                  Manipulate organizations.
    os                            Manipulate operating system.
    partition-table               Manipulate partition tables.
    proxy                         Manipulate smart proxies.
    puppet-class                  Search puppet modules.
    report                        Browse and read reports.
    role                          Manage user roles.
    sc-param                      Manipulate smart class parameters.
    shell                         Interactive shell
    subnet                        Manipulate subnets.
    template                      Manipulate config templates.
    user                          Manipulate users.
    user-group                    Manage user groups.

Options:
    -v, --verbose                 be verbose
    -c, --config CFG_FILE         path to custom config file
    -u, --username USERNAME       username to access the remote system
    -p, --password PASSWORD       password to access the remote system
    --version                     show version
    --show-ids                    Show ids of associated resources
    --csv                         Output as CSV (same as --adapter=csv)
    --output ADAPTER              Set output format. One of [base, table, silent, csv]
    --csv-separator SEPARATOR     Character to separate the values
    -P, --ask-pass                Ask for password
    --autocomplete LINE           Get list of possible endings
    -h, --help                    print help

First level command help:

$ hammer architecture -h
Usage:
    hammer architecture [OPTIONS] SUBCOMMAND [ARG] ...

Parameters:
    SUBCOMMAND                    subcommand
    [ARG] ...                     subcommand arguments

Subcommands:
    list                          List all architectures.
    info                          Show an architecture.
    create                        Create an architecture.
    delete                        Delete an architecture.
    update                        Update an architecture.
    add_operatingsystem           Associate a resource
    remove_operatingsystem        Disassociate a resource

Options:
    -h, --help                    print help

Second level command help:

$ hammer architecture create -h
Usage:
    hammer architecture create [OPTIONS]

Options:
    --name NAME
    --operatingsystem-ids OPERATINGSYSTEM_IDS
                                  Operatingsystem ID’s
    -h, --help                    print help

4.5.2 Success Story

There was a set of common commands identified as necessary for basic Foreman management, we called it “success story” and track the progress of its implementation. The commands could also serve as a basic hammer cookbook.

The goal is to provision bare metal host on a clean install of Foreman. The following steps are necessary:

• create smart proxy

hammer proxy create --name myproxy --url https://proxy.my.net:8443

• create architecture

hammer architecture create --name x86_64

• create new subnet

hammer subnet create --name "My Net" --network "192.168.122.0" --mask "255.255.255.0" --gateway "192.168.122.1" --dns-primary "192.168.122.1"

• import existing subnet from a proxy

  missing, see #3355

• create new domain

hammer domain create --name "my.net" --fullname "My network"

• associate domain with proxy

hammer domain update --id 1 --dns-id 1

• associate subnet with domain

hammer subnet update --id 1 --domain-ids 1

• associate subnet with proxy (DHCP, TFTP, DNS)

hammer subnet update --id 1 --dhcp-id 1 --tftp-id 1 --dns-id 1

• create new partition table

hammer partition_table create --name "Redhat test" --file /tmp/rh_test.txt

• create new OS

hammer os create --name RHEL --major 6 --minor 4

• create new template

hammer template create --name "kickstart mynet" --type provision --file /tmp/ks.txt

• edit existing pre-defined template

hammer template dump --id 4 > /tmp/ks.txt
    vim /tmp/ks.txt
hammer template update --id 4 --file /tmp/ks.txt

• associate applicable OS with pre-defined template

hammer template update --id 1 --operatingsystem-ids 1

Listing associated OS’s is still missing - see #3360

• associate OS with architecture

hammer os update --id 1 --architecture-ids 1

• associate OS with part table

hammer os update --id 1 --ptable-ids 1

• associate OS with install media

hammer os update --id 1 --medium-ids 1

• associate OS with install provision and pxelinux templates

  Missing, needs investigation, may be related to #3360

• create libvirt compute resource

hammer compute_resource create --name libvirt --url "qemu:///system" --provider Libvirt

• import puppet classes

  missing - see #3035

• and finally create a bare metal host entry

  works with some options, needs improvements - see #3063

4.6 Email Management

Foreman is also able to send out a variety of email notifications either on an event, or summary messages on a regular schedule. Plugins are also able to extend this with their own summaries and notifications.

To send email requires a configured SMTP server or local MTA (e.g. sendmail), which is set up in Adminster > Settings > Email as per Configuration Options.

Scheduled emails are sent through rake tasks (reports:daily, reports:weekly, reports:monthly) run from cronjobs, which are configured in /etc/cron.d/foreman.

4.6.1 Email Preferences

Users

Email messages are sent to individual users registered to Foreman, to the email address configured on the account if present. Users can edit the email address by clicking on their name in the top-right hand corner of the web page and selecting My account.

To change which message subscriptions are received by an individual user, the Mail Preferences tab under the user account lists all available message types and the frequency at which each message should be received. A global checkbox to disable all email messages from Foreman is also available.

Event-based notifications can either be enabled or disabled, and these are sent from Foreman at the same time as the event occurring. Scheduled notifications can be sent either daily, weekly or monthly.

Hosts

Notifications relating to hosts can be disabled on a per-host basis, useful when errors are expected. On the host’s Additional Information tab, untick Enabled to disable notifications and remove the host from reports. Enabling and disabling notifications can also be done from the host list by using the tickboxes and selecting Enable/Disable Notifications from the Select Action dropdown menu.

Event notifications for a host are sent to the host’s registered owner. This is selected on the Additional Information tab of the host, and may be either an individual user or a user group. When set to a user group, all group members who are subscribed to the email type will receive a message.

4.6.2 Account Notifications

New account welcome email

When the send_welcome_email setting is enabled (Configuration Options), new account holders will receive an email providing their username and a link to Foreman.

4.6.3 Host Notifications

Build complete

When a host has completed its build process, either notifying Foreman of completion via a request at the end of its unattended installation or after Foreman has run a script remotely, this email notification will be sent to owners of the host.

Puppet error state

When a Puppet report is received that puts the host into a red error state, a corresponding email notification is sent to owners of the host.

4.6.4 Scheduled Emails

Audit summary

A regular summary email of all changes to objects in Foreman that triggered audit events (see Auditing), including the user that made the change, the time of the change and a link to further details.

Puppet summary

A regular overview of all hosts that a user has access to, and their Puppet status. This includes the number of Puppet events over the reporting period, such as applied, skipped and failed resources.

4.7 Managing Ansible

Please refer to the foreman_ansible plugin documentation.

4.8 Managing Chef

Please refer to the foreman_chef plugin documentation.

4.9 Managing Salt

Please refer to the foreman_salt plugin documentation.

4.10 Monitoring

To monitor your infrastructure, host statuses are useful. In Foreman each host has a global status that indicates which hosts need attention. Each host also has sub-statuses that represents status of a particular feature. With any change of a sub-status, the global status is recalculated and the result is determined by statuses of all sub-statuses.

4.10.1 Global status

The global status represents the overall status of a particular host. The status can have one of three possible values - OK, Warning or Error.

OK means that no errors were reported by any sub-status. It is represented with the color green.

Warning suggests that user should verify the status, while no error was detected, some sub-status raised a warning. A good example would be that there are no Puppet reports for the host even though the host is configured to send Puppet reports. Therefore it is highlighted with the color yellow.

The last possible value is Error which indicates that some sub-status reports a failure. This could for example mean that Puppet run contains some failed resources. Obviously it is something that should be fixed and is user is alerted by the color red.

You can find global status on hosts overview page displayed as a small icon next to host name with corresponding color. Hovering over the icon renders a tooltip with sub-status information to quickly find out more details.

4.10.2 Sub-statuses

A sub-status monitors only a part of host capabilities. Currently Foreman ships only two - build and configuration sub-statuses. Not all sub-statuses are relevant for all hosts, therefore configuration is only considered if host is using some configuration management system, e.g. has some Puppet proxy associated. Build sub-status is relevant for managed hosts and when Foreman is run in unattended mode.

You can see a global host status with all sub-statuses on the host detail page, in the properties table. Note that there can be more sub-statuses added by plugins.

Each sub-status can define own set of possible values that are mapped to three global status values. Build sub-status has two possible values - pending and built that are both mapped to global OK value. Configuration status is more complicated and its possible values and mappings are described in table below.

4.10.3 Searching by statuses

You can search hosts by global status. Some examples can be found below:

• search for all hosts that are OK

  global_status = ok

• search for all hosts that deserves some attention

  global_status = error or global_status = warning

To search hosts based on configuration status you can search by last report metrics like this:

• find hosts that have at least one pending resource

  status.pending > 0

• find hosts that restarted some service during last puppet run

  status.restarted > 0

• find hosts that have an interesting last Puppet run (something happened)

  status.interesting = true

5. Advanced Foreman

5.1 API

API v2 is the default, stable and recommended version for Foreman 1.17. API v1 is also available, but future versions of Foreman will eventually deprecate and remove it.

This section documents the JSON API conventions for the Foreman API v2 and Katello API v2. To explicitly select the API version, see Section 5.1.6.

5.1.1 CRUD Request Examples

The following examples show the basic CRUD operations (Create, Read, Update, Delete) using the JSON API.

Show a Collection of Objects

Get of a collection of domains: GET /api/domains

Send a HTTP GET request. No JSON data hash is required.

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" https://foreman.example.com/api/domains

This returns a collection JSON response. The format for a collection response is described in Section 5.1.2.

Show a Single Object

Get a single domain: GET /api/domains/:id or GET /api/domains/:name

Send a HTTP GET request with the object’s unique identifier, either :id or :name. No JSON data hash is required.

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" \
    https://foreman.example.com/api/domains/42
# or
$ curl -k -u admin:changeme -H "Accept: version=2,application/json" \
    https://foreman.example.com/api/domains/foo

This returns a single object in JSON format. The format for a single object response is described in Section 5.1.3.

Create an Object

Create a new domain: POST /api/domains

Send a HTTP POST request with a JSON data hash containing the required fields to create the object. In this example, a domain is being created.

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" -H "Content-Type: application/json" \
    -X POST -d '{ "name":"foo.bar.com","fullname":"foo.bar.com description" }' \
    https://foreman.example.com/api/domains

This returns the newly created object in JSON format, with the same attributes as in the show/GET call. The format for a single object response is described in Section 5.1.3.

The HTTP response code of the create call will be 201, if created successfully.

Update an Object

Update a domain: PUT /api/domains/:id or PUT /api/domains/:name

Send a HTTP PUT request with the object’s unique identifier, either :id or :name, plus a JSON data hash containing only the data to be updated. In this example, only the domain name is being updated.

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" -H "Content-Type: application/json" \
    -X PUT -d '{ "name": "a new name" }' https://foreman.example.com/api/domains/12
# or
$ curl -k -u admin:changeme -H "Accept: version=2,application/json" -H "Content-Type: application/json" \
    -X PUT -d '{ "name": "a new name" }' https://foreman.example.com/api/domains/foo

This returns the newly updated object in JSON format. The format for a single object response is described in Section 5.1.3.

Delete an Object

Delete a domain: DELETE /api/domains/:id or DELETE /api/domains/:name

Send a HTTP DELETE request with the object’s unique identifier, either :id or :name. No JSON data hash is required.

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" -X DELETE \
    https://foreman.example.com/api/domains/17
# or
$ curl -k -u admin:changeme -H "Accept: version=2,application/json" -X DELETE \
    https://foreman.example.com/api/domains/foo

This returns the deleted object in JSON format. The format for a single object response is described in Section 5.1.3.

5.1.2 JSON Response Format for Collections

Collections are a list of objects (i.e. hosts, domains, etc). The format for a collection JSON response consists of a results root node and metadata fields total, subtotal, page, per_page. Note: for Katello objects, the metadata includes limit, offset instead of page, per_page.

Below is an example of the format for a collection JSON response for a list of domains: GET /api/domains

{
    "total": 3,
    "subtotal": 3,
    "page": 1,
    "per_page": 20,
    "search": null,
    "sort": {
        "by": null,
        "order": null
    },
    "results": [
        {
            "id": 23,
            "name": "qa.lab.example.com",
            "fullname": "QA",
            "dns_id": 10,
            "created_at": "2013-08-13T09:02:31Z",
            "updated_at": "2013-08-13T09:02:31Z"
        },
        {
            "id": 25,
            "name": "sat.lab.example.com",
            "fullname": "SATLAB",
            "dns_id": 8,
            "created_at": "2013-08-13T08:32:48Z",
            "updated_at": "2013-08-14T07:04:03Z"
        },
        {
            "id": 32,
            "name": "hr.lab.example.com",
            "fullname": "HR",
            "dns_id": 8,
            "created_at": "2013-08-16T08:32:48Z",
            "updated_at": "2013-08-16T07:04:03Z"
        }
    ]
}

The response metadata fields are described below:

• total - total number of objects without any search parameters
• subtotal - number of objects returned with given search parameters (if there is no search, then subtotal equals total)
• page (Foreman only) - page number
• per_page (Foreman only) - maximum number of objects returned per page
• limit - (Katello only) specified number of objects to return in collection response
• offset - (Katello only) number of objects skipped before beginning to return collection.
• search - search string (based on scoped_scoped syntax)
• sort
  • by - the field that the collection is sorted by
  • order - sort order, either ASC for ascending or DESC for descending
• results - collection of objects. See Section 5.1.4 for how to change the root name from ‘results’ to something else.

5.1.3 JSON Response Format for Single Objects

Single object JSON responses are used to show a single object. The object’s unique identifier :id or :name is required in the GET request. Note that :name may not always be used as a unique identifier, but :id can always be used. The format for a single object JSON response consists of only the object’s attributes. There is no root node and no metadata by default. See Section 5.1.4 for how to add a root name.

Below is an example of the format for a single object JSON response: GET /api/domains/23 or GET /api/domains/qa.lab.example.com

{
    "id": 23,
    "name": "qa.lab.example.com",
    "fullname": "QA",
    "dns_id": 10,
    "created_at": "2013-08-13T09:02:31Z",
    "updated_at": "2013-08-13T09:02:31Z"
}

5.1.4 Customize JSON Responses

Customize Root Node for Collections

The default root node name for collections is results but can be changed.

To change the root node name per API request, pass root_name= as a URL parameter. See example below:

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" \
    https://foreman.example.com/api/domains?root_name=data

Customize Root Node for Single Object

There is no root node as the default for single object JSON responses, but it can be added.

To change the object’s root node name per API request, pass object_name= as a URL parameter. See example below:

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" \
    https://foreman.example.com/api/domains/23?object_name=record

Customize Partial Response Attributes

Currently, there is no option to change or customize which attributes are returned for collections or single objects. In the future, customized partial responses such as fields=field1,field2,field3 or fields=all may be implemented (#3019). Similarly, there is currently no option to specify child nodes in an API call or to remove child nodes if they are returned by default.

Custom Number of Objects in Collection Per Response

Foreman paginates all collections in the JSON response. The number of objects returned per request is defined in Administer > Settings > General > entries_per_page. The default is 20. Thus, if there are 27 objects in a collection, only 20 will be returned for the default page=1.

To view the next page, pass page= as a URL parameter. See example below:

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" \
    https://foreman.example.com/api/domains?page=2

The example above will show the remaining 7 objects in our example of 27 objects in the collection.

To increase or decrease the number of objects per response, pass per_page= as a URL parameter. See example below:

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" \
    https://foreman.example.com/api/domains?per_page=1000

This will return all the objects in one request since 27 is less than the per_page parameter set to 1000.

Custom Search of Collections Per Response

Foreman uses the scoped_search library for searching and filtering which allows all query search parameters to be specified in one string. The syntax is described in the Searching section, and matches exactly the syntax used for the web UI search boxes. This allows you use of the auto-completer and to test a query in the UI before reusing it in the API.

To filter results of a collection, pass search= as a URL parameter, ensuring that it is fully URL-escaped to prevent search operators being misinterpreted as URL separators. See example below:

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" \
    https://foreman.example.com/api/domains?search=name%3Dexample.com

The number of objects returned will be shown in the subtotal metadata field, and the query string will be shown in the search metadata field.

Custom Sort of Collections Per Response

Custom sort order per collection can be specified by passing order= as a URL parameter. See example below:

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" \
    https://foreman.example.com/api/domains?order=name+DESC

The default sort order is ascending (ASC) if only a field name is passed. The sort parameters will be shown in sort by and order metadata fields.

5.1.5 Nested API routes

The goal is to implement nested routes for all objects as an alternative to filtering collections.

For example, rather then filtering subnets by a specified domain using a search string

$ GET /api/subnets?search=name%3Dqa.lab.example.com

the alternative nested route below returns the same result as the above.

$ GET /api/domains/qa.lab.example.com/subnets

All actions will be accessible in the nested route as in the main route.

5.1.6 API Versioning

The default API version is v2 for Foreman 1.17, however explicitly requesting the version is recommended. Both API v1 and v2 are currently shipped.

There are two methods of selecting an API version:

1. In the header, pass Accept: application/json,version=2

2. In the URL, pass /v2/ such as GET /api/v2/hosts

Similarly, v1 can still be used by passing Accept: application/json,version=1 in the header or api/v1/ in the URL.

5.1.7 Handling Associations

Updating and creating associations are done in a few different ways in the API depending on the type of association.

One-to-One and One-to-Many

To update a one-to-one or a one-to-many association, simply set the name or id on the object. For example, to set a host group for a host, simply set the hostgroup_name or hostgroup_id of the host.

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" \
    -H "Content-Type: application/json" -X POST \
    -d '{ "hostgroup_name": "telerin" }' \
    https://foreman.example.com/api/hosts/celeborn.firstage

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" \
    -H "Content-Type: application/json" -X POST \
    -d '{ "hostgroup_id": 42 }' \
    https://foreman.example.com/api/hosts/celeborn.firstage

Many-to-One and Many-to-Many

To update an association for an object that contains a collection of other objects, there are a few options. First you can set the names or ids:

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" \
    -H "Content-Type: application/json" -X POST \
    -d '{ "host_names": ["enel.first", "celeborni.first", "elwe.first"] }' \
    https://foreman.example.com/api/hostgroups/telerin

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" \
    -H "Content-Type: application/json" -X POST \
    -d '{ "host_ids": [4, 5, 6] }' \
    https://foreman.example.com/api/hostgroups/telerin

This will set the host group’s hosts to enel, celeborn, and elwe (or 4, 5, 6) and only those.

Alternatively, you can pass in a set of objects:

$ curl -k -u admin:changeme -H "Accept: version=2,application/json" \
    -H "Content-Type: application/json" -X POST \
    -d '{ "domains": [{ "name": "earendil", "id": 1}, { "name": "turgon", "id": 3 }] }' \
    https://foreman.example.com/api/subnets/iluvatar

This would set the domains for the subnet to be earendil and turgon. If another domain for example belonged to the subnet before the request, it would be removed.

5.1.8 Authentication

The API requires authentication for all endpoints, typically using HTTP Basic authentication. Requests with credentials are authenticated against the users stored in Foreman.

HTTP Basic authentication

HTTP Basic authentication (RFC 2617) is supported by a wide range of API and web clients and works by specifying a Base64-encoded username and password in an Authorization header. For example, these common clients can access the API with the following arguments:

• curl -u admin:changeme, or curl -u admin (interactive prompt)
• wget --user=admin --password=changeme

Every call to the API will require authentication, unless the client supports sessions (see below). Some clients may also support storing credentials in ~/.netrc or similar for more privacy.

No confidentiality is provided with this method, so it is very important to use HTTPS when connecting to Foreman to prevent the plain-text credentials from being obtained. (Note that when require_ssl is enabled, access to the API will only be allowed over HTTPS.)

Session support

When authenticating to the API, a new server-side session will be created on each request and the response will contain a cookie containing a session ID. If this cookie is stored by the client, it can be used on subsequent requests so the credentials are only passed over the connection once.

A basic authenticated request to the status API returns the following Set-Cookie header, containing a _session_id cookie:

> GET /api/v2/status HTTP/1.1
> Authorization: Basic YWRtaW46Y2hhbmdlbWU=
> Host: foreman.example.com
> Accept: */*

< HTTP/1.1 200 OK
< ...
< Set-Cookie: _session_id=572ca37e8c5845b900cc58d45d6e1e34; path=/; secure; HttpOnly

When supplying this on subsequent requests, they will use the same account:

> GET /api/v2/status HTTP/1.1
> Host: foreman.example.com
> Accept: */*
> Cookie: _session_id=572ca37e8c5845b900cc58d45d6e1e34

< HTTP/1.1 200 OK
< ...

Command-line clients may support cookie jars for automatic storage of cookies, e.g. curl -c ~/.foreman_cookies -b ~/.foreman_cookies will automatically store and use cookies.

5.1.9 Using OAuth

Alternatively to basic authentication, limited OAuth 1.0 authentication is supported in the API.

Configuration of OAuth in Foreman

OAuth must be enabled in Foreman settings. In Administer > Settings > Authentication, search for OAuth active configuration and set it to Yes. Then set OAuth consumer key to some string. This will be a token used by all OAuth clients.

If you want all API requests made using OAuth to be authorized as built-in anonymous admin user keep OAuth map users set to No. If you want to specify the user under which the request is made, change this configuration option to Yes. This allows client to send FOREMAN-USER header with the login of existing Foreman user. Please note that this header is not signed in OAuth request so can be forged. Anyone with valid consumer key can impersonate any Foreman user.

Request example

Usually some OAuth client library is used to generate the request. An example of curl command can be found here to better understand how it works

$ curl 'https://foreman.example.com/api/architectures' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json,version=2' \
  -H 'FOREMAN-USER: ares' \
  -H 'Authorization: OAuth oauth_version="1.0",oauth_consumer_key="secretkey",oauth_signature_method="hmac-sha1",oauth_timestamp=1321473112,oauth_signature=Il8hR8/ogj/XVuOqMPB9qNjSy6E='

In example above we list architectures using OAuth for authentication. We try to do the request under user with login ares, if mapping is enabled on Foreman side, the result will only include architectures, that user ares can see. Note that we constructed the signature manually, this should change with any oauth_timestamp change. Also it reflects every parameter, HTTP method and URI change. Therefore we recommend using some OAuth client library that will construct all OAuth parameters.

5.1.10 Using Apipie-Bindings

The following examples show how to do basic API operations using apipie-bindings.

Show a Collection of Objects

Get of a collection of domains: GET /api/domains

Call the index function of the domains resource.

#!/usr/bin/env ruby

require 'apipie-bindings'

url = 'https://foreman.example.com/'
username = 'admin'
password = 'changeme'
api = ApipieBindings::API.new({:uri => url, :username => username, :password => password, :api_version => '2'})
domains = api.resource(:domains).call(:index)

domains['results'].each do |domain|
  puts domain
end

$ ruby domains.rb
{"fullname"=>"", "dns_id"=>1, "created_at"=>"2016-05-06 08:46:20 UTC", "updated_at"=>"2016-11-24 11:49:06 UTC", "id"=>1, "name"=>"example.com"}

Show a Single Object

Get a single domain: GET /api/domains/:id or GET /api/domains/:name

Call the show function of the domains resource with the object’s unique identifier :id or :name.

#!/usr/bin/env ruby

require 'apipie-bindings'

url = 'https://foreman.example.com/'
username = 'admin'
password = 'changeme'
api = ApipieBindings::API.new({:uri => url, :username => username, :password => password, :api_version => '2'})
puts api.resource(:domains).call(:show, {:id => 1})
puts api.resource(:domains).call(:show, {:id => 'example.com'})

Create an Object

Create a new domain: POST /api/domains

Call the create function of the domains resource with a JSON data hash containing the required fields to create the object. In this example, a domain is being created.

#!/usr/bin/env ruby

require 'apipie-bindings'

url = 'https://foreman.example.com/'
username = 'admin'
password = 'changeme'
api = ApipieBindings::API.new({:uri => url, :username => username, :password => password, :api_version => '2'})
api.resource(:domains).call(:create, {:domain => {:name => "foo.example.com", :fullname => "foo.example.com"}})

Update an Object

Update a domain: PUT /api/domains/:id or PUT /api/domains/:name

Call the update function of the domains resource with the object’s unique identifier, either :id or :name, plus a JSON data hash containing only the data to be updated. In this example, only the domain name is being updated.

#!/usr/bin/env ruby

require 'apipie-bindings'

url = 'https://foreman.example.com/'
username = 'admin'
password = 'changeme'
api = ApipieBindings::API.new({:uri => url, :username => username, :password => password, :api_version => '2'})
api.resource(:domains).call(:update, {:id => 3, :domain => {:name => "foo.example.com", :fullname => "foo.example.com"}})

Delete an Object

Delete a domain: DELETE /api/domains/:id or DELETE /api/domains/:name

Call the destroy function of the domains resource with the object’s unique identifier, either :id or :name.

#!/usr/bin/env ruby

require 'apipie-bindings'

url = 'https://foreman.example.com/'
username = 'admin'
password = 'changeme'
api = ApipieBindings::API.new({:uri => url, :username => username, :password => password, :api_version => '2'})
api.resource(:domains).call(:destroy, {:id => 3})

5.2 Compute Resources

Foreman supports creating and managing hosts on a number of virtualization and cloud services - referred to as “compute resources” - as well as bare metal hosts.

The capabilities vary between implementations, depending on how the compute resource provider deploys new hosts and what features are available to manage currently running hosts. Some providers are able to support unattended installation using PXE, while others are image-based. Some providers have graphical consoles that Foreman interfaces to, and most have power management features. A summary of all providers and their support features is given below, and more detailed sections follow with specific notes.