UPX and SELinux                        March 6, 2006When a program that has been compressed by UPX is run, the decompressormust create and write new memory pages of executable instructions.SELinux (Security Enhanced Linux) directly controls the conditionsunder which generating and/or executing new instructions is allowed,so the configuration settings of SELinux affect the running of programsthat have been compressed by UPX.In SELinux "strict enforcing" mode (the most restrictive), generatingnew instructions at runtime is not allowed at all: any page withPROT_EXEC permission must be mapped from a file in a mounted filesystemthat has 'x' [eXecute] permission, and the generation of such files isalso tightly controlled.  A program that was compressed by UPX will notrun in SELinux strict enforcing mode.  Attempts will fail with exitcode 127, and a record will be added to the history file/var/log/audit/audit.log.In "targeted enforcing" mode, SELinux pays close attention mostly todesignated processes that run with elevated privileges: web server,print server, login server, etc.  Ordinary user excutables receivemuch less scrutiny.  However, one of the eventual goals of SELinux is toeradicate runtime generation of instructions because of the possibilityfor exploitation by malware (virus, trojan, key logger, privilegeelevation exploit, etc.)  Thus targeted enforcing mode notices andlogs the use of "execmem" capability that is used by a program whichwas compressed by UPX.  In keeping with the goal of eventual prohibition,SELinux ordinarily would deny execmem.  However, most current SELinuxsystems, including Fedora Core 5 [set for release March 15, 2006],override this with "allow_exemem=1" in /etc/selinux/targeted/booleans.Thus a program compressed by UPX will run in the default installedconfiguration (targeted enforcing, allow_execmem=1) of SELinux underFedora Core 5.  Each invocation will add a few lines to the log file/var/log/audit/audit.log, one line for each use of execmem.  If theSELinux policy becomes more restrictive in the future, then a specialSELinux class or other mechanism must be created for compressed programs,or else UPX-compressed executables will not run then.In its "permissive" modes, SELinux just logs the potential problems,but otherwise does not interfere.  A program compressed by UPX will runin any permissive mode.