1 (edited by muzso 2012-07-06 16:04:13)

Topic: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

My environment is Ubuntu 10.10 with MySecureShell 1.25 (installed from the http://mysecureshell.free.fr/repository … hp/ubuntu/ repository).

The server has apparmor installed (since it is installed by default), but I suspect that apparmor is not at fault here. I've tested with apparmor disabled (invoke-rc.d apparmor stop) and the same error occured when connecting to the server via SSH+SFTP+MySecureShell.

The complete error message from /var/log/sftp-server.log is:

2012-07-06 14:28:13 [17501][backup][myserver.example.com]Couldn't chroot to '/home/backup': Operation not permitted

sftp-verif doesn't find any problems:

################################################################################
            MySecureShell Verification Tool
################################################################################

### Verifing file existance ###

/bin/MySecureShell                                                       [ OK ]
/usr/bin/sftp-who                                                        [ OK ]
/usr/bin/sftp-kill                                                       [ OK ]
/usr/bin/sftp-state                                                      [ OK ]
/usr/bin/sftp-admin                                                      [ OK ]
/usr/bin/sftp-verif                                                      [ OK ]
/usr/bin/sftp-user                                                       [ OK ]


### Verifing rights ###

Verifing file rights of /etc/ssh/sftp_config                             [ OK ]
Verifing file rights of /usr/bin/sftp-who                                [ OK ]
Verifing file rights of /usr/bin/sftp-verif                              [ OK ]
Verifing file rights of /usr/bin/sftp-user                               [ OK ]
Verifing file rights of /usr/bin/sftp-kill                               [ OK ]
Verifing file rights of /usr/bin/sftp-state                              [ OK ]
Verifing file rights of /usr/bin/sftp-admin                              [ OK ]
Verifing file rights of /bin/MySecureShell                               [ OK ]


### Verifing rotation logs ###

Rotation logs have been found                                            [ OK ]


### Verifing server status ###

Verifing server status (ONLINE)                                          [ OK ]


### Verifing server dependencies ###

Show only error(s) :


### Verifing server configuration ###

Show only error(s) :
Trying user: root
Checking user : backup

### All tests dones ###

If checking manually, MySecureShell's permissions seem to be alright:

$ ls -al /bin/MySecureShell
-rwsr-xr-x 1 root root 71496 2012-07-06 16:27 /bin/MySecureShell

I've downloaded the package source, modified SftpServer/SftpServer.c to log the UID (via getuid()) and EUID (via geteuid()) and unfortunately the following is logged:

2012-07-06 14:28:13 [17501] muzso debug, UID = 112, EUID = 112

Looking at the processes (while the SFTP client is connected) shows this:

$ ps faux
(...)
root      1124  0.0  0.1   5544  1032 ?        Ss   Jun29   0:00 /usr/sbin/sshd -D
(...)
root     23557  0.0  0.3  12672  3396 ?        Ss   16:51   0:00  \_ sshd: backup [priv]
backup  23570  0.0  0.1  12672  1676 ?        S    16:51   0:00      \_ sshd: backup@notty 
backup  23571  0.0  0.2   7060  3056 ?        SLs  16:51   0:00          \_ MySecureShell -c /usr/lib/openssh/sftp-server

The above of course doesn't show that anything would be wrong. In case of a Debian Squeeze server of mine VirtualChroot works just fine and the processes look quite the same:

root      9624  0.0  0.0  49176  1136 ?        Ss   Jun30   0:00 /usr/sbin/sshd
root     27171  0.1  0.0  72552  3284 ?        Ss   16:56   0:00  \_ sshd: backup [priv]
backup     27189  0.0  0.0  72552  1616 ?        S    16:56   0:00  |   \_ sshd: backup@notty
backup     27190  0.0  0.0  30800  1932 ?        Ss   16:56   0:00  |       \_ MySecureShell -c /usr/lib/openssh/sftp-server

So something prevents the setuid bit put on /bin/MySecureShell to actually take effect on my Ubuntu server.
This might be apparmor, but I don't know how/why. When I stop apparmor (invoke-rc.d apparmor stop), all profiles are unloaded and (afaik) apparmor should not mess with any privileges anymore. My knowledge of apparmor is pretty slim, so it's still possible that apparmor is somehow interfering with MySecureShell. I'll try what happens if I completely remove the apparmor package and reboot. I'll come back with my findings.

In the meantime ... do you've any idea what might cause this problem ... apart from apparmor?

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

Just tried something. I've created an apparmor profile for /bin/MySecureShell and the result (/etc/apparmor.d/bin.MySecureShell) looks like this:

#include <tunables/global>

/bin/MySecureShell flags=(complain) {
  #include <abstractions/base>

  capability setuid,
  capability sys_chroot,

  /etc/host.conf r,
  /etc/hosts r,
  /etc/nsswitch.conf r,
  /etc/resolv.conf r,
  /etc/ssh/sftp_config r,

}

As you can see, I permit both setuid and chroot, but it didn't help. Next comes complete removal of apparmor (and a reboot). Let's see if it helps. (Although I might not be able to do this right now, because there're still plenty of people in the office and the Ubuntu server in question is our primary fileserver. :-) )

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

Hi,

It's very strange because it works on my ubuntu with apparmor... perhaps it's a sshd config ?

4 (edited by muzso 2012-07-06 17:47:50)

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

teka wrote:

It's very strange because it works on my ubuntu with apparmor... perhaps it's a sshd config ?

Probably not. I've tested (a few seconds ago) with the very same /etc/ssh/sshd_config that works on my Debian Squeeze server. Although you could be onto something here. On the Debian server (where VirtualChroot works) the openssh-server package is 1:5.5p1-6+squeeze2, on the Ubuntu (Maverick) server it's 1:5.9p1-5ubuntu1. It might be worth to check whether or not something related to this problem was changed in OpenSSH.

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

No, it was not the sshd version either. I've noticed that the openssh-server package in Maverick (in the maverick-updates repo) is only 1:5.5p1-4ubuntu6 and I had 1:5.9p1-5ubuntu1 installed (and not from any currently configured repo). I guess I tried a newer openssh-server package sometime in the past and just forgot about it (this VirtualChroot problem haunts me for quite some time ... I just wasn't too determined to get to the bottom of it).

The point is: I've reverted back to the repo default package (1:5.5p1-4ubuntu6) and the problem is still there. The one last thing I might try (before I try the apparmor removal+reboot next week) is to install the openssh-server package from Debian Squeeze. I know this is a weird idea, but since VirtualChroot works on my Squeeze server ... it's worth a try.

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

Noop, that was not it either. I've installed the very same openssh-server package on the Ubuntu server that runs on my Debian server and the problem still persists. :-(

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

I've modified main.c to print the UID+EUID too. And if I log in to the Ubuntu server with a normal user and execute

/bin/MySecureShell --version

, it prints an UID=10000 and EUID=0. So the setuid bit on /bin/MySecureShell works indeed. The only problem is that when sshd executes MySecureShell, the EUID is not kept "0".

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

Works at home with openssh "1:5.9p1-5ubuntu1" ...

Really i don't know why it doesn't work sad sad sad

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

I'm tracking it down right now. :-) It's somehow MySecureShell's "fault", that I'm already sure of.
I'm adding extra logs of UID and EUID throughout the code to see where is EUID set to UID.
One thing I already now: when MySecureShell is spawned by sshd, the EUID is still 0.
But when the chroot() call gets executed, EUID is already equal to UID.
So EUID is set to the value of UID somewhere in between.

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

Can you paste your configuration of MySecureShell ?

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

Sorry for the delay. I had a 5.2 km swim today :-) and had to go to bed.
I've found the problem yesterday around midnight (GMT+2 :-) ).
My Ubuntu server uses LDAP authentication for users and uses the libnss_ldap library for fetching a user list from the LDAP server.

The EUID is changed by libnss_ldap (or another lib that this one depends on). It happens in SftpServer/GetUsersInfos.c in the init_usersinfos() function, when you call getpwent() (which at some point calls a function in libnss_ldap). The EUID is only changed, when the for() loop reaches the user that I logged in with (through SFTP). The EUID is set (from 0 to the user's UID) at the next call (after the one that returned the SFTP user's record) to getpwent().

I'll track down the exact library and function that does this and see if it can be disabled via configuration.

All in all: the failure of VirtualChroot is by no way MySecureShell's fault. :-)
However it might be helpful if init_usersinfos() would watch the EUID and log if it changes during the call to getpwent().
Moreover you could check the EUID right before the call to chroot() too and log if it's not zero (in case there're other libs out there that can interfere with MySecureShell).

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

No problem, gratz smile


Huuu very interesting in function "getpwent()"... but i'm happy in MySecureShell v1.30 (currently in progress) we don't use the function ! big_smile

I think it's not possible to fix bug in MySecureShell 1.25... but you can use "StayAtHome" to bypass the problem smile

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

It's actually not just getpwent(). If you call a system function, you cannot know what is going to be executed. In case of getpwent() all kinds of NSS libraries can be executed (depends on the configuration). But MySecureShell calls tons of system functions and any one of them could potentially call user-defined, custom libraries (like in case of getpwent() it calls something from libnss_ldap on my server).

So if you make the assumption that MySecureShell is running with root privileges (just because /bin/MySecureShell belongs to root and it's setuid bit is set), then you make a mistake. The root privilege (ie. EUID) can be changed during runtime by any library that MySecureShell calls ... even libraries that are invoked by system functions.

So if I may suggest for the development of v1.30 ... if the user enables VirtualChroot in MySecureShell's config, then you should check the EUID repeatedly in various parts (functions) of MySecureShell so that if EUID changes (due to an external library) you can easily narrow down what may have caused it.

The getpwent() problem I've just found is just an example, but there might be a lot more system calls where external libraries can get in the way of chroot().

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

But it doesn't make sense to modify EUID for a library... however MSS 1.30 is using less functions smile

I'll add some check to prevents strange error wink

Thank you for your help !

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

I guess there's some reason for that lib to mess with EUID, but certainly it's a bit strange. I had no time yet to track the specific lib and function down, but I'll do it in the next few days (I'm curious too :-) ).

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

The problem is not present anymore in MySecureShell v1.30. I've compiled it from source and simply switching from v1.25 to v1.30 solved my problem.

P.S.: I had no time to investigate further on the issue with the libnss_ldap library (as I wrote earlier I would), but since v1.30 does not use the given system call in which libnss_ldap dropped root privileges, there's not much need to do any further digging (apart from personal curiosity ... but mine had moved on to different areas).

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

Ok, thank you for the report smile

Re: VirtualChroot fails, Couldn't chroot to ...: Operation not permitted

This is just a problem of administrative rights to this permission. The program needs to be given permission to access this root. The best way to work with this program is to give it full admin power.







--------------------------------
CORBIN GRAVELY
microsoft security help