301 Moved Permanently

This is the last posting I do on this blog.
I decided to switch to a self hosted wordpress installation for blogging. I hope to see you soon on my new blog site.
There is already a new blog post about setting up SUSE Manager.

Cobbler on openSUSE

Preamble

What is cobbler?

To answer that, I want to quote from the homepage of Cobbler


Cobbler is a Linux installation server that allows for rapid setup of network installation environments. It glues together and automates many associated Linux tasks so you do not have to hop between lots of various commands and applications when rolling out new systems, and, in some cases, changing existing ones.

With a simple series of commands, network installs can be configured for PXE, reinstallations, media-based net-installs, and virtualized installs (supporting Xen, qemu, KVM, and some variants of VMware). Cobbler uses a helper program called 'koan' (which interacts with Cobbler) for reinstallation and virtualization support.

This artice describes the installation and setup of Cobbler on openSUSE 11.4 and the basic steps of using it.

Preparation

Install the required software

First of all, you need it install an openSUSE 11.4 with the cobbler+cobbler-web package installed of course :)

Most people will have those repos already after the installation but anyway, if you don't have the default repos do:

zypper ar http://download.opensuse.org/distribution/11.4/repo/oss/ oss
zypper ar http://download.opensuse.org/update/11.4/ updates

There is cobbler v2.0.10 on openSUSE 11.4 (oss repo) but that version does not have all fixes and functionality you want. So you have to use the latest cobbler version you can find in this repo:

zypper ar http://download.opensuse.org/repositories/systemsmanagement/openSUSE_11.4/ cobbler
zypper ref -s

Make sure you have the latest mod_python module update installed. It has to be at least apache2-mod_python-3.3.1-161 because the 11.4 goldmaster version of mod_python has a bug and does not work with any python script. We released an online update for mod_python on 11.4 on openSUSE 11.4 in the beginning of march 2011, so maybe you already have it installed. In any case, do:

zypper up apache2-mod_python

Now we install cobbler:

zypper install cobbler cobbler-web atftp

That should install version 2.1.0 or later. We are done with the package installation and have to do some configuration now.

Basic configuration of Cobbler

If you want to use the cobbler web ui, you have to authenticate and there are multiple ways cobbler can authenticate a user. In the file /etc/cobbler/modules.conf we set the authentication module to authn_configfile. That will read the account information from /etc/cobbler/users.digest:

sed -i 's/module = authn_denyall/module = authn_configfile/' \
  /etc/cobbler/modules.conf

There are more auth modules like for ldap-authentication and they are all documented in /etc/cobbler/modules.conf

Now we change the server settings in /etc/cobbler/settings. Search for "server: 127.0.0.1" in that file and replace it with the ip address of your cobbler server. That address will end later in the autoyast=..... and install=.... parameter.

YOUR_IP="....."
sed -i "s/^server: 127.0.0.1/server: $YOUR_IP/" \
  /etc/cobbler/settings

We are done with the basic configuration now and it's time start cobbler for the first time:

insserv apache2
insserv cobblerd
rcapache2 restart
rccobblerd restart

First Steps

Look around

take your webbrowser and visit http://localhost/cobbler_web. You'll get a login mask where you can login with user: cobbler, password: cobbler. That's the default configuration in /etc/cobbler/users.digest and we did not change that so far. If you want to add your own accounts, you can do that by:

htdigest2 /etc/cobbler/users.digest "Cobbler" $USER

After the login, your webbrowser presents you a webpage like this:


We are ready to work with cobbler now.

Importing a Distro

To do installations, we need an installation source. So let us import one into Cobbler and we use the openSUSE 11.4 DVD for that. Please mount the DVD to /mnt and choose "Import DVD" on the left side of the webui. Fill out the form like you see on the screenshot:


If you press the "run" button, an import job is being scheduled. You can see the job by clicking on "Events" (third link from the top in "Configuration" on the left side). Your import job has the state "running" but that will change to "complete" in a while, when cobbler has copied the complete DVD to the harddisk.

When Cobbler has finished the import, two new distros are created like you can see on the Distros page


Distro is another word for installation source but why two? Because the DVD contains two bootable kernels. One for xen guests and one for a normal physical machine. You can see that one distro got a "xen" in the name and that's the installation source you can use to install virtual xen machines.
As you can see, an install=... parameter has already been added by Cobbler so the distro is ready for installations. You can add own kernel parameter if you wish but for this tutorial it's not needed.

Adding an Autoinstallation Profile

Profiles in Cobbler are connected to a distribution.

If you click on Profiles you can see that cobbler has created two empty autoinstallation profiles already - one for each distro. We will ignore those and create our own one by chosing "Create new profile" and click on "go". Copy your AutoYaST XML file to /var/lib/cobbler/kickstarts/ (or any other path you like) and fill out the form like I did:


You can see that I selected the non-xen version of our distros. If you want to install a virtual xen machine, you have to choose the other distro with the "xen" in the name.

If you press "save" you'll get back to the profile overview page where you can see three profiles now, including the one you just created.

On the right side of each profile is a link "view kickstart" and if you press on that, you'll see your XML file (cobbler has added some scripts but you don't have to care about that now).

Install a virtual machine

We have a distro and an AutoYaST XML file now and that's enough to create a virtual machine. I'll not explain here how to setup kvm or xen but I assume you have a physical machine configured for running virtual machines. I use kvm and need to to some little changes in the test_profile for that. I change the virt-type to qemu and the Virt Bridge from xenbr0 tp br0 because that's how the bridge is called on my virtual host platform.

You have to install koan from the same repo as cobbler on your virtual host platform (cobbler itself is not needed to be installed) and run:

koan --server=$your_cobbler_server.example.net \
 --virt \
 --profile=test_profile \
 --virt-name=virt_host1

This will trigger the creation and autoinstallation of a virtual machine called "virt_host1". If you start virt-manager, you can see that the machine is installing the openSUSE 11.4 with the AutoYaST XML file.

Bare Metal Installation

If you want to do bare metal installations via PXE boot, you need to acticate the tftp server on the Cobbler machine:

rcatftpd start
insserv atftpd

For the bare metal installation we have to create a system record in Cobbler, so let's got to Systems -> create new system and configure the fields as I di in the screenshot:


When you press save you will be redirected to the systems-list and you can see your system there. If you switch to the console, you can see that a PXE boot config file was created in /srv/tftpboot/pxelinux.cfg/01-yo:r:mac:add:rr:ess so it'll start an installation next time it boots.

If you set the option pxe_just_once in /etc/cobbler/settings to "1", autoyast will turn off the PXE boot before it reboots at the end of the 1st stage, so you don't have endless PXE boot loops. You can switch on/off the netboot option for a system in the web ui of cobbler.

What else?

There is a lot to discover in cobbler. For example:

• You can create Snippets which are a kind of include files but way more powerful. They can be referenced by $SNIPPET(name_of_snippet) then in the AutoYaST XML file


• You can let Cobbler manage your dhcp server, so all systems get a dhcp record automatically (very useful, I use it here in my testlab)
• You can let Cobbler manage your dns server, so all systems get a dns record automatically (very useful too but I don't use it here)
• You can create boot ISO images so you can do installations without network
• You can register existing systems by calling cobbler-register to add a system-record based on the running system automatically (devel:tools:python-ethtool is needed for that on the client)


• You can let koan reinstall a running machine. It'll copy kernel/initrd to the harddisk and rewrites the grub config. With the next reboot, a new system is getting installed on the machine
• cobbler has a command line client tool called cobbler that lets you do everything you can do in the web ui via command line too
• ...

As you can see, cobbler offers a lot more than I can cover in one blog.

If you have questions or look for more information go to:

• Cobbler Homepage
• Mailinglist Archive
• my git repo on github (my patches for SUSE support are upstream too in version >2.1.0 )
• devel:tools repo for python-ethtool

Of course you can send me an eMail too.


have fun, Uwe

SUSE Manager Autoinstall Screenshots

as you might know, we published the SUSE Manager yesterday.

It was fun to work on that project and I learned really a lot.
Anyway, I read many announcement texts in the press and from Novell but I did not see any pictures. People love pictures. So here they come (most screenshots about autoinstallation - I guess you know why ;) ).

Overview page:

AutoYaST/Kickstart Profiles overview:

Edit an AutoYaST profile:

Overview of your systems (machines and virtual machines):

Reinstall a physical machine with AutoYaST/Kickstart:

Install a virtual machine on a physical host with AutoYaST:

Configure bare metal installations (assign an AutoYaST XML file to an ip address range):

that's it for now!

Have fun - Uwe

AutoYaST and Image Creation/Installation

In openSUSE 11.3 I enhanced the AutoYaST image installation feature a bit and I added a nice useable UI to configure it.
It's still quite beta because I was not able to test it the way I should have but I think that if you are interested in that topic, it's worth a look already.
Let's start immediately and create an AutoYaST image with the UI.

Step 1 - make the openSUSE Installation Source available

taylor:~ # mount /dev/dvd /mnt

I simply mount the openSUSE 11.3 DVD here to the /mnt directory.

Step 2 - start the AutoYaST UI

taylor:~ # yast2 autoyast

When the UI is started, go to the software selection configuration. You'll be asked where the installation source is you want to use.
As you can see in the screenshot, I set the installation source to the mounted DVD by "file:///mnt/".

Hint: You can't create 64bit images on a 32bit system but 32bit images on a 64bit system

Step 3 - select your software for the image

I selected the base pattern and the kernel I want (don't forget the kernel. Linux is no fun without a kernel).




Select okay to return to the main UI when you have selected all the software you want to have in the image.
You probably want to configure anything non image related now, like root password, partitioning, ...

Step 4 - Create the Image

Choose "Image Deployment" in the AutoYaST UI and configure the location of the image. That's where AutoYaST can find it during the installation process.
I want to do a network installation and I'll put the image on my webserver and so I type the http address here.
As you can see in the screenshot, you can also create an ISO. Then you don't have to specify a name and the location, because a DVD ISO will be created that contains the image and the autoyast XML file plus everything you need to do an installation from the ISO by booting it.
Creating an ISO is probably easier in the beginning if you want to play with this feature.

Step 5 - Choose the location to store the image

I'll store it to /srv/www/htdocs/ because that's the document root of my webserver. Like mentioned in step 4, I'll fetch the image from my webserver.



If you create an ISO, you have to choose where to save the ISO now.

Step 6 (optional) - make changes to the image

AutoYaST will now create the image and you can do changes to it when all patterns and packages are installed. You can see that in the screenshot.
So if you want to put any files into the image, you can copy them now to the place shown in the popup, which is a directory that contains the system that will later be dumped to you harddisk. You can even chroot into that directory and let scripts run if you want.
This step is optional, so if you don't want to add files or anything, you can simply press "ok" here to compress the image.

Now you are done with the image creation. Let's take a look at what we have in the XML file now:

...
  <software>
    <image>
      <image_location>http://taylor.suse.de/</image_location>
      <image_name>my_image</image_name>
      <run_kickoff config:type="boolean">true</run_kickoff>
      <script_location>file:///usr/lib/YaST2/bin//fetch_image.sh</script_location>
      <script_params config:type="list">
        <listentry>http://taylor.suse.de//my_image.tar.gz</listentry>
      </script_params>
    </image>
    <instsource>file:///mnt/</instsource>
    <packages config:type="list">
      <package>kernel-desktop</package>
    </packages>
    <patterns config:type="list">
      <pattern>apparmor_opt</pattern>
      <pattern>base</pattern>
      <pattern>yast2_install_wf</pattern>
      <pattern>sw_management</pattern>
      <pattern>apparmor</pattern>
    </patterns>
...

The XML file is prepared for an installation of the image. The fetch_image.sh is part of AutoYaST since openSUSE 11.3 and will fetch and unpack the image after the partitioning is done. That's why we can use "file:///" here, because autoyast will find it as it's part of the autoyast2-installation RPM.
You can replace that script by anything you want that can fetch and unpack the image. So if you have a complete different image type, you can write your own script to fetch and unpack the image. Of course if you have a different kind of image, you can't use the AutoYaST UI anymore to create it. AutoYaST will always create a tar.gz file that contains the installed system.

Installing images via AutoYaST has three major benefits compared to the RPM installation step:

• multiple times faster


• post-configuration can be done in the image instead of after the installation


• very easy switching between image installation and RPM installation (with rules, even dynamically when starting the installation)

You can install images without losing any of the AutoYaST functionality. Ask-dialogs, all kind of scripts (even pre-scripts), complex partitioning and so on is still possible because only the slow RPM installation step is skipped and replaced by putting an image to the harddisk while you have all kinds of flexibility of how that image is created.

As mentioned in the beginning of this article, I had not enough time to test this through but if you are interested in it feel free to play with it and report feedback to me if anything fails or can be enhanced :)

Have fun!

ciao, Uwe

AutoYaST goes Twitter

well, it's not that it can read XML files from a twitter feed now ;) but after this blog, starting from today, AutoYaST goes one further step to web 2.0 and has it's own twitter account.
It's not for great hints or enhanced support or so. It's just a little look behind the curtain of my life with AutoYaST ... for those who care ;)
And it seems you can't do any bigger project these days without something like a blog, twitter or facebook or ... so ... here it is.

ciao, Uwe

AutoYaST and it's Tools

"Al: Didn't you study the manual at all?
Tim: A real man doesn't need a manual. "


Because my last article was for the experts, this is more for the AutoYaST-newbies again.
It's not about AutoYaST itself but about the tools that orbit around it.

• ayast_setup
  This tool is part of the autoyast2 RPM.
  Bubli wrote an excellent tutorial already so I'll just sum things up a little here.
  With ayast_setup you can reconfigure an already installed and running system without reinstalling it. All you need is an AutoYaST XML file and then call:
  

  yast2 ayast_setup setup filename=/tmp/my.xml

  
  The XML file will be read by AutoYaST and applied to your currently running system except (and that's good - you want it that way, believe me) all 1st stage configuration like partitioning, bootloader, pre- and chrootscripts and software selection. So if you use a XML file with partitioning information, your system will not get repartitioned.
  If you want to install packages, you have to use the <post-packages> section because the normal <software> section is evaluated during 1st stage only too.
  


• yast2 clone_system
  This tool is also part of the autoyast2 RPM.
  This is kind of the opposite of ayast_setup. At the end of the normal installation, you have the option to clone the freshly installed system for AutoYaST but if you missed that opportunity, you can do that later in the running system with the clone_system client.
  Since openSUSE 11.3 you can call it like this:
  

  yast2 clone_system help

  
  and you'll get a list of "known modules" that can be cloned like this:
  

  known modules:
    bootloader dns-server
    firewall host
    http-server inetd
    iscsi-client kerberos
    lan ldap mail nfs
    ...

  
  and you can clone them like this:
  

  yast2 clone_system modules clone=lan,partitioning,software

  
  This will create an AutoYaST XML file in /root/autoinst.xml with the networking, partitioning and software section. If you don't specify any modules to clone, the same XML file will be created like with the "Clone this System" checkbox at the end of a normal installation.
  The "modules" parameter is new on openSUSE 11.3 and will not work on SLES11. On SLES11 you can only do the default cloning by calling

  yast2 clone_system

  
  Some customers are even doing remote-cloning with screen and ssh like this:
  

  ssh --batch root@myhost "screen -D -m yast clone_system &"

  
  


• ayast_probe
  This tool is part of the autoyast2-installation RPM.
  It's not very useful in a running system but can become handy during the installation.
  When you use a rules.xml file, you can do things like (pseudocode) "if installed_product == 'SLES11' and memsize > 1024 use big_software.xml". The real syntax can be found
  here in the documentation.
  The question is, what are reasonable values of installed_product or memsize?
  

  /usr/lib/YaST2/bin/y2base ayast_probe ncurses

  shows you.
  
  


• yast2 autoyast
  This tool is part of the autoyast2 RPM.
  If you start to look at AutoYaST for the first time, you'll probably get in touch with this tool immediately. It's the AutoYaST userinterface to create a XML file. Every module like networking, samba-client, partitioning, software configuration and so on has a UI here so you don't have to deal with XML code.
  But the AutoYaST UI can do more. You can clone single modules like you can do with the clone_system client by just pressing a single button (I painted that red in the screenshot, where the nfs-client is selected and would be cloned).
  You can apply the config of one module to the currently running system like with ayast_setup but with just a single button press again (green in the screenshot for the nfs-client config).
  And last but not least you can check your XML file for syntax errors.
  It's nearly impossible to create an invalid XML file with the UI and if you are not sure about certain XML parameters, the UI can be of a big help for you.
  
  

While the tools above were all 100% AutoYaST related and are only useful in combination with something autoyastish, the following tools are used by people who never heared of AutoYaST too.

• xsltproc
  xsltproc is part of the libxslt RPM.
  This tool is used to apply XSLT files on a XML file.
  "What???!" - well, you only have to know that AutoYaST uses a XSLT file to merge multiple XML files into one big file. So when you use AutoYaST Rules or AutoYaST Classes your single XML files will be merged to a big one with the help of /usr/share/autoinstall/xslt/merge.xslt.
  It can be very useful to test the merging before you start the actual installation to make sure everything comes out as you planned it:
  

  /usr/bin/xsltproc --novalid \
    --param replace "'false'" \
    --param dontmerge1 "'package'" \
    --param with "'a.xml'" \
    --output out.xml \
    /usr/share/autoinstall/xslt/merge.xslt \
    b.xml
  

  
  a.xml and b.xml will be merged to out.xml. So take a look at out.xml to see if the merging led to the expected result.
  The dontmerge parameter is explained here. In short, you need that parameter if you want to merge two XML files that have the same sections. Like two XML files both with a <software> section.
  


• create_update_source.sh and createrepo
  create_update_source.sh is part of the inst-source-utils RPM, createrepo is part of the createrepo RPM
  Both tools are for creating RPM repositories. If you create such a repo and add the location to your XML file, AutoYaST can install the RPMs from that repo too. So if you want to add own RPMs to your installation, don't change the actual SUSE Installation source, but create a new repo and add your RPM's there.
  How to do that is explained here.
  Even if you only want to install RPM's that are on openSUSE or SLES, you can use these tools. You can create a repo with all online-updates that were released and AutoYaST will install those updates then instead of the original RPM's.

I'd say those are the most important tools around AutoYaST - have fun using them for "more power" ;-)

ciao, Uwe

Interactive AutoYaST Rules

This article describes a new feature of AutoYaST in openSUSE 11.3 (a backport for 11.2 is available too).

The rules.xml file describes how AutoYaST creates the complete XML profile out of single XML snippet files. Like for example "if the main memory is more than 2GB, use kde.xml, else use windowmaker.xml", "if you find more than one harddisk, use lvm.xml, else use simple_partitioning.xml" ... and so on ...
That's available since ages in AutoYaST what's new now is, that you can present all XML snippets you have to the users so they can select which ones to use on their own.
Confused but curious? Then read on ....

A while ago I read a feature request on the AutoYaST mailinglist that it would be cool to have the <rules> visible and selectable during installation and I was thinking: "Sounds like a good idea. So why not?". So here it is.
People love pictures so lets start with a screenshot:

On the screenshot you can see a dialog where you can choose between two rules where the first one is pre-selected and the second one is greyed out because it conflicts with the first one (I'll explain later what that conflict means).
Lets take a look at the interesting part of the rules.xml file now to see how to create such a dialog.

<rules config:type="list">
...
<rule>
  <custom1>
      <script>
<![CDATA[
echo -n 100
]]>
    </script>
    <match>100</match>
    <match_type>exact</match_type>
  </custom1>
  <result>
    <profile>rules/kde.xml</profile>
    <continue config:type="boolean">true</continue>
  </result>
  <dialog>
    <element config:type="integer">0</element>
    <question>KDE Desktop</question>
    <title>Desktop Selection</title>
    <conflicts config:type="list">
      <element config:type="integer">1</element>
    </conflicts>
    <dialog_nr config:type="integer">0</dialog_nr>
    <timeout config:type="integer">30</timeout>
  </dialog>
</rule>
<rule>
  <custom1>
      <script>
<![CDATA[
echo -n 100
]]>
      </script>
     <match>101</match>
     <match_type>exact</match_type>
  </custom1>
  <result>
    <profile>rules/gnome.xml</profile>
    <continue config:type="boolean">true</continue>
  </result>
  <dialog>
    <element config:type="integer">1</element>
    <dialog_nr config:type="integer">0</dialog_nr>
    <question>Gnome Desktop</question>
    <conflicts config:type="list">
      <element config:type="integer">0</element>
    </conflicts>
  </dialog>
</rule>
<rule>
...
</rule>
...
</rules>

That's not a complete rules.xml but only the interesting part with the two rules that created the dialog you see in the screenshot.
Here comes the description of the single XML elements:

• <dialog_nr config:type="integer">0</dialog_nr>
  
  That's a uniq id for the a popup dialog. All rules with the same dialog_nr are presented on the same popup dialog. This element is optional and the default for a missing dialog_nr is always "0". If you have one popup only anyway, you don't need to specify the dialog_nr.
  


• <element config:type="integer">0</element>
  
  element needs to be a uniq id. Even if you have more than one dialog, you must not use the same id twice like an element-id "1" on dialog 1 and and element-id "1" on dialog 2 too. That's different than with <ask> dialogs, where you can have the same <element> id on multiple dialogs. I'll explain later why this is.
  


• <title>Software Selection</title>
  
  that's a text string at the top of the dialog. A caption for that dialog if you want so and it's optional. If you configure multiple titles for one dialog, the last one is used.
  


• <question>KDE Software Selection</question>
  
  That's the text string behind the checkbox. If you don't configure a question, the name of the XML file that is triggered by this rule will be shown instead (in our example above, that would be "rules/kde.xml" and "rules/gnome.xml"). If you know what's in kde.xml and gnome.xml you maybe don't need the <question> but I'd prefer a short explaining text over just a filename.
  


• <timeout config:type=""integer">30</timeout>
  
  if you don't answer the dialog in "timeout" seconds, AutoYaST will continue.
  As soon as you touch one checkbox or button, the timeout stops for this popup and you have to press "okay" to continue. The next dialog can have a new timeout then.
  


• <conflicts config:type="list">
  
  here you can specify a list of element id's that conflict with this XML file. If you select this XML file, the conflicting one(s) will be deselected and disabled. Be careful what you configure here, you can create deadlocks. An element can conflict with another element on a different dialog and that's the reason why element id's have to be uniq.
  

Whether a question is pre-selected or not is defined by the result of the rule. In my example above, I configured a rule that matches always (100 = 100) and a rule that matches never (101 = 100). Without dialogs, such a rule file would not make a lot of sense because gnome.xml would never be used but here I use them to do the default pre-selection.
So again, if a rule matches, the file (kde.xml in our example) is pre-selected.
That means, if you add dialogs to your already existing rules.xml that has no dialogs yet and you set a timeout, your autoinstallation process will be the same like before, except that you'll see a popup dialog for "timeout" seconds. The rules work like before and you can see which ones have matched in the popups.

To sum things up:

All dialog configuration is done in the rules.xml file. Each <rule> can have one <dialog> which represents one checkbox on a dialog popup. You can have multiple dialog popups and you can have multiple rules (checkboxes) on one dialog popup.


This feature is officially implemented for openSUSE 11.3 but since openSUSE 11.2 is brand new and out for just a few days now, I made a driverupdate_11.2.tar.bz2 to make the feature available for openSUSE 11.2 too. If you want to try it, download and extract the file and copy the "driverupdate" file to the root of your installation source to get that feature on openSUSE 11.2.
Feedback is appreciated.

Thanx to Stacey Murphy and Martin Vogt for the idea.


have fun creating your own rules ;)
Uwe