news aggregator

Luke Faraone: Unstandardized standards are the worst: sendmail

Planet Ubuntu - Thu, 2014-01-02 05:25
Implementing software to replace legacy systems is always a challenge, especially when you're dealing with a system with as much legacy as sendmail, which was first introduced as delivermail in 1979.ref

Each UNIX vendor, it seems, rewrote or heavily customized sendmail. This has lead to sometimes conflicting implementations.
Case in point: -tNormally, you invoke sendmail(8) with a series of arguments indicating the subject of a message, the recipients, etc. When invoked this way, the command expects a message on standard input, waits for EOF, and then sends your message along.

However, sometimes you don't want to have to fiddle with command-line parameters; you've already written a perfectly fine message with headers.

-t is generally passed to sendmail when you want to build a message envelope from an already-formatted message, with headers, etc. For example, if you had a file foo.txt with a body like this:
From: Luke Faraone
To: John Smith
Subject: Hello, world!

Hi there.
you could send the message with a simple invocation of cat foo.txt | sendmail -t. The system would take care of ensuring a Message-id was appended if appropriate, and queue the message to be sent. However, it is when you do slightly more complex invocations of sendmail that things get ambiguous.

It turns out that implementations differ on what exactly it means when you use -t in combination with naming destination addresses after the arguments to sendmail. exim4's documentation describes the situation in greater detail:
extract_addresses_remove_   argumentsUse: mainType: booleanDefault: trueAccording to some Sendmail documentation (Sun, IRIX, HP-UX), if any addresses are present on the command line when the -t option is used to build an envelope from a message’s To:, Cc: and Bcc: headers, the command line addresses are removed from the recipients list. This is also how Smail behaves. However, other Sendmail documentation (the O’Reilly book) states that command line addresses are added to those obtained from the header lines. When extract_addresses_remove_arguments is true (the default), Exim subtracts argument headers. If it is set false, Exim adds rather than removes argument addresses.
Thus, there's basically no mechanism for a program to know which behaviour to expect. God forbid two programs are installed on a system that expect different behaviours!

It appears that the default behaviour of Ruby is the opposite of what exim4 (Debian's default mail client) expects. This has resulted in numerous bug reports. Some replies suggest changing exim4's defaults, while others advocate overriding ActionMailer and friends to use sendmail -i instead, without -t.

That said, its not really clear who's wrong here; at no point does there appear to have been a definitive specification for sendmail, and as such we can hope for defined behaviour by common custom at best, and a sea of incompatibility bugs at worst. Amusingly, POSIX standards have nothing to say on this subject of sendmail at all; it defines that a mailx command must exist, but says that its sending mode may be implementation-specific.

As Matthew Garrett writes, there's not enough gin in the world.

Eric Hammond: Default ssh Usernames For Connecting To EC2 Instances

Planet Ubuntu - Thu, 2014-01-02 05:03

Each AMI publisher on EC2 decides what user (or users) should have ssh access enabled by default and what ssh credentials should allow you to gain access as that user.

For the second part, most AMIs allow you to ssh in to the system with the ssh keypair you specified at launch time. This is so common, users often assume that it is built in to EC2 even though it must be enabled by each AMI provider.

Unfortunately, there is no standard ssh username that is used to access EC2 instances across operating systems, distros, and AMI providers.

Here are some of the ssh usernames that I am aware of at this time:

OS/Distro Official AMI
ssh Username Legacy / Community / Other AMI
ssh Usernames Amazon Linux ec2-user Ubuntu ubuntu root Debian admin root RHEL 6.4 and later ec2-user RHEL 6.3 and earlier root Fedora ec2-user root Centos root SUSE root BitNami bitnami TurnKey root NanoStack ubuntu FreeBSD ec2-user OmniOS root

Even though the above list will get you in to most official AMIs, there may still be situations where you aren’t quite sure how the AMI was built or what user should be used for ssh.

If you know you have the correct ssh key but don’t know the username, this code can be used to try a number of possibilities, showing which one(s) worked:

host=<IP_ADDRESS> keyfile=<SSH_KEY_FILE.pem> for user in root ec2-user ubuntu admin bitnami do if timeout 5 ssh -i $keyfile $user@$host true 2>/dev/null; then echo "ssh -i $keyfile $user@$host" fi done

Some AMIs are configured so that an ssh to root@ will output a message informing you the correct user to use and then close the connection. For example,

$ ssh root@<UBUNTUHOST> Please login as the user "ubuntu" rather than the user "root".

When you ssh to a username other than root, the provided user generally has passwordless sudo access to run commands as the root user. You can use sudo, ssh, and rsync with EC2 hosts in this configuration.

If you know of other common ssh usernames from popular AMI publishers, please add notes in the comments with a link to the appropriate documentation.

Original article:

Stephan Adig: It never rains in Southern California...

Planet Ubuntu - Wed, 2014-01-01 19:00

… totally not true

but what is true…

is that I am here for one month already and I am totally excited.

As mentioned in one of my last posts, I am working now for Sony Europe, especially for a company which was aquired by Sony. We are working closely with the Sony Playstation Team.

And Guys, it’s a blast.

I would really like to write something about what we are doing, and especially how we are doing it, but sadly I would kick my own ass out of this adventure, so I won’t reveal anything.

What I can say is this:

This month was full of new experiences. A very different approach to our discipline. And somehow I am feeling at home.

Our people here are very enthusiastic about their product, you can see that every day. The proudness, focus and knowledge is special here.

The work environment is more than awesome. Yes, StartUp feeling, for sure, but that is not it.

People are discussing new ideas, and how we can approach challenges in a very different way. Different from the traditional SysAdmin approach.

Being an SRE here means, not only knowing your system and being able to fix stuff in the SysAdmin way, but also to improve the overall quality of the system, which also means, that we are coding a lot of tools by ourselves to improve our work, reporting quality drops to other departments and debugging issues in third party software.

Furthermore, the bond between SRE and Development/Engineering is very tight. Means, whichever bug SRE finds, in OS, third party software stacks or in-house developed software, we will fix them ourselves, or we will report our findings directly to Development/Engineering (of course via bugtracker :))

The answers are coming fast, and the bugfixes, too. Faster than I was expecting.

That brings me to my next surprise. I never saw so many people working with Linux on a Workstation. This is really surprising. Don’t get me wrong, you’ll find here a lot of different computers and OSes, mostly Apple MacBooks and other types of Laptops, mostly dual booting, but at least every SRE and Developer has a Workstation with Linux running on it. Pretty awesome.

Yes, the vast amount of OpenSource Software here is incredible and surprising.

Anyways, I am so excited, and I am proud to work on this project. It will be a success, I have no doubts.

One last statement, what we are doing here, is revolutionary. It will change the way of todays Gaming experience, believe me. I am already dogfooding and even when I am more a casual gamer, I am impressed about the quality.

So, when you are into Gaming, look out for announcements from Sony in 2014.


And there is still time to do some other things. Like fixing Python code for the Python Sphinx Contrib Project.

While working on a Python Project inhouse, I needed to use the sphinxcontrib-httpdomain module, sadly it wasn’t Python3 compatible.

Until 2 days ago :)

I worked on changing this, but without looking at some Python Helpers, which would have made the work more easy. After filing the pull-request, Upstream said thanks, but I should have a look at python-six, a Python Library which makes the transition a lot more easier than manual coding.

I did that, and ported the fixes to python-six and commited the changes and updated the pull-request. Upstream merged after 5 minutes, and my changes will be in the next release of sphinxcontrib-httpdomain.

Well, this is really special. We are using OpenSSH with Roumen Petrovs X.509 Patch. Sadly, this patch is not applied to the OpenSSH packages of most distributions. Neither Ubuntu, Debian or Fedora are carrying this patch in their repos.

So I am working on a sane solution for this and resolving this bug in Launchpad.

And to make things even more smooth for our Friends from Fedora, I am working on an RPM package for the same OpenSSH package as well.

Charles Profitt: Ubuntu 13.10 Reboots Instead of Shutting Down

Planet Ubuntu - Wed, 2014-01-01 16:32

The last few days I was struggling to find the causes of a peculiar issue. Ubuntu 13.10, given the conditions detailed below, would reboot instead of shutting down.

  • Suspend computer by closing lid
  • Resume computer by opening lid
  • Select shut down from the menu

The expected behavior is that the laptop would shutdown, but what actually happened is a shutdown followed within two seconds by a reboot. I posted a question on Askubuntu asking what logs I should look at and I posted on the Ubuntu forums looking for any advice at all. I continued my own trouble shooting and found the culprit was the newly acquired Anker® Uspeed USB 3.0 7-Port Hub + 5V 2A Charging Port.

Two different computers had the issue with the hub attached. I tested on a Lenovo T530 and X230. The Anker uses the VIA VL812 chipset and my research on that shows other manufacturers advertising that they have updated firmware (v8581) on their devices. I was unable to determine what firmware the Anker was using and their website did not offer any firmware upgrades. There is also a VIA VL812-B2 chipset that Anker and other manufacturers are advertising as well.

Ubuntu users looking for a USB 3.0 hub may want to avoid the Anker 68UPHUB-B8U and potentially the VIA VL812 chipset.

St&eacute;phane Graber: LXC 1.0: Security features [6/10]

Planet Ubuntu - Wed, 2014-01-01 14:32

This is post 6 out of 10 in the LXC 1.0 blog post series.

When talking about container security most people either consider containers as inherently insecure or inherently secure. The reality isn’t so black and white and LXC supports a variety of technologies to mitigate most security concerns.

One thing to clarify right from the start is that you won’t hear any of the LXC maintainers tell you that LXC is secure so long as you use privileged containers. However, at least in Ubuntu, our default containers ship with what we think is a pretty good configuration of both the cgroup access and an extensive apparmor profile which prevents all attacks that we are aware of.

Below I’ll be covering the various technologies LXC supports to let you restrict what a container may do. Just keep in mind that unless you are using unprivileged containers, you shouldn’t give root access to a container to someone whom you’d mind having root access to your host.


The first security feature which was added to LXC was Linux capabilities support. With that feature you can set a list of capabilities that you want LXC to drop before starting the container or a full list of capabilities to retain (all others will be dropped).

The two relevant configurations options are:

  • lxc.cap.drop
  • lxc.cap.keep

Both are lists of capability names as listed in capabilities(7).

This may sound like a great way to make containers safe and for very specific cases it may be, however if running a system container, you’ll soon notice that dropping sys_admin and net_admin isn’t very practical and short of dropping those, you won’t make your container much safer (as root in the container will be able to re-grant itself any dropped capability).

In Ubuntu we use lxc.cap.drop to drop sys_module, mac_admin, mac_override, sys_time which prevent some known problems at container boot time.

Control groups

Control groups are interesting because they achieve multiple things which while interconnected are still pretty different:

  • Resource bean counting
  • Resource quotas
  • Access restrictions

The first two aren’t really security related, though resource quotas will let you avoid some obvious DoS of the host (by setting memory, cpu and I/O limits).

The last is mostly about the devices cgroup which lets you define which character and block devices a container may access and what it can do with them (you can restrict creation, read access and write access for each major/minor combination).

In LXC, configuring cgroups is done with the “lxc.cgroup.*” options which can roughly be defined as: lxc.cgroup.<controller>.<key> = <value>

For example to set a memory limit on p1 you’d add the following to its configuration:

lxc.cgroup.memory.limit_in_bytes = 134217728

This will set a memory limit of 128MB (the value is in bytes) and will be the equivalent to writing that same value to /sys/fs/cgroup/memory/lxc/p1/memory.limit_in_bytes

Most LXC templates only set a few devices controller entries by default:

# Default cgroup limits lxc.cgroup.devices.deny = a ## Allow any mknod (but not using the node) lxc.cgroup.devices.allow = c *:* m lxc.cgroup.devices.allow = b *:* m ## /dev/null and zero lxc.cgroup.devices.allow = c 1:3 rwm lxc.cgroup.devices.allow = c 1:5 rwm ## consoles lxc.cgroup.devices.allow = c 5:0 rwm lxc.cgroup.devices.allow = c 5:1 rwm ## /dev/{,u}random lxc.cgroup.devices.allow = c 1:8 rwm lxc.cgroup.devices.allow = c 1:9 rwm ## /dev/pts/* lxc.cgroup.devices.allow = c 5:2 rwm lxc.cgroup.devices.allow = c 136:* rwm ## rtc lxc.cgroup.devices.allow = c 254:0 rm ## fuse lxc.cgroup.devices.allow = c 10:229 rwm ## tun lxc.cgroup.devices.allow = c 10:200 rwm ## full lxc.cgroup.devices.allow = c 1:7 rwm ## hpet lxc.cgroup.devices.allow = c 10:228 rwm ## kvm lxc.cgroup.devices.allow = c 10:232 rwm

This configuration allows the container (usually udev) to create any device it wishes (that’s the wildcard “m” above) but block everything else (the “a” deny entry) unless it’s listed in one of the allow entries below. This covers everything a container will typically need to function.

You will find reasonably up to date documentation about the available controllers, control files and supported values at:


A little while back we added Apparmor profiles support to LXC.
The Apparmor support is rather simple, there’s one configuration option “lxc.aa_profile” which sets what apparmor profile to use for the container.

LXC will then setup the container and ask apparmor to switch it to that profile right before starting the container. Ubuntu’s LXC profile is rather complex as it aims to prevent any of the known ways of escaping a container or cause harm to the host.

As things are today, Ubuntu ships with 3 apparmor profiles meaning that the supported values for lxc.aa_profile are:

  • lxc-container-default (default value if lxc.aa_profile isn’t set)
  • lxc-container-default-with-nesting (same as default but allows some needed bits for nested containers)
  • lxc-container-default-with-mounting (same as default but allows mounting ext*, xfs and btrfs file systems).
  • unconfined (a special value which will disable apparmor support for the container)

You can also define your own by copying one of the ones in /etc/apparmor.d/lxc/, adding the bits you want, giving it a unique name, then reloading apparmor with “sudo /etc/init.d/apparmor reload” and finally setting lxc.aa_profile to the new profile’s name.


The SELinux support is very similar to Apparmor’s. An SELinux context can be set using “lxc.se_context”.

An example would be:

lxc.se_context = unconfined_u:unconfined_r:lxc_t:s0-s0:c0.c1023

Similarly to Apparmor, LXC will switch to the new SELinux context right before starting init in the container. As far as I know, no distributions are setting a default SELinux context at this time, however most distributions build LXC with SELinux support (including Ubuntu, should someone choose to boot their host with SELinux rather than Apparmor).


Seccomp is a fairly recent kernel mechanism which allows for filtering of system calls.
As a user you can write a seccomp policy file and set it using “lxc.seccomp” in the container’s configuration. As always, this policy will only be applied to the running container and will allow or reject syscalls with a pre-defined return value.

An example (though limited and useless) of a seccomp policy file would be:

1 whitelist 103

Which would only allow syscall #103 (syslog) in the container and reject everything else.

Note that seccomp is a rather low level feature and only useful for some very specific use cases. All syscalls have to be referred by their ID instead of their name and those may change between architectures. Also, as things are today, if your host is 64bit and you load a seccomp policy file, all 32bit syscalls will be rejected. We’d need per-personality seccomp profiles to solve that but it’s not been a high priority so far.

User namespace

And last but not least, what’s probably the only way of making a container actually safe. LXC now has support for user namespaces. I’ll go into more details on how to use that feature in a later blog post but simply put, LXC is no longer running as root so even if an attacker manages to escape the container, he’d find himself having the privileges of a regular user on the host.

All this is achieved by assigning ranges of uids and gids to existing users. Those users on the host will then be allowed to clone a new user namespace in which all uids/gids are mapped to uids/gids that are part of the user’s range.

This obviously means that you need to allocate a rather silly amount of uids and gids to each user who’ll be using LXC in that way. In a perfect world, you’d allocate 65536 uids and gids per container and per user. As this would likely exhaust the whole uid/gid range rather quickly on some systems, I tend to go with “just” 65536 uids and gids per user that’ll use LXC and then have the same range shared by all containers.

Anyway, that’s enough details about user namespaces for now. I’ll cover how to actually set that up and use those unprivileged containers in the next post.


Subscribe to Free Software Magazine aggregator