sudoedit.com!

Just some guy on the internet.


authors: [“Luke Rawlins”] date: 2017-07-03 draft: false title: How to RTFM (Read the *#&@ Manual) description: “How to read man pages and find help without searching the internet on Linux and Unix systems.” url: /read-the-manual/ tags: – documentation – Linux – troubleshooting – rtfm


Finding help with Linux

If you hang out in enough Linux forums asking questions sooner or later someone will tell you to read the manual (presumably they think this will help you). Fortunately, over the last few years “rtfm” has ceased being the default answer to most questions from new users. All things considered, the Linux world has become more user friendly even if the man pages haven't.

Learning to read and understand man pages will allow you to separate yourself from new, and even some intermediate, users. Plus, if you plan to take any Linux exams you will need to know how to find and read man pages. There is simply no way that most of us can memorize all of the command and configuration options you need to pass a Red Hat or Linux Foundation exam.

The Help Option

Nearly every command you use at the command line has a help option. I like to use help when I am fairly comfortable with a command but need a quick reference to see specific options. In many cases --help will get the information you are looking for.

Let's look at a good example, the lvextend command.

lvextend --help

     WARNING: Running as a non-root user. Functionality may be unavailable.
    /run/lvm/lvmetad.socket: connect failed: Permission denied
      WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
    lvextend: Add space to a logical volume
    lvextend
    [-A|--autobackup y|n]
    [--alloc AllocationPolicy]
    [--commandprofile ProfileName]
    [-d|--debug]
    [-f|--force]
    [-h|--help]
    [-i|--stripes Stripes [-I|--stripesize StripeSize]]
     {-l|--extents [+]LogicalExtentsNumber[%{VG|LV|PVS|FREE|ORIGIN}] |
    -L|--size [+]LogicalVolumeSize[bBsSkKmMgGtTpPeE]}         --poolmetadatasize [+]MetadataVolumeSize[bBsSkKmMgG]}
     [-m|--mirrors Mirrors]
     [--nosync]
    [--use-policies]
     [-n|--nofsck]
    [--noudevsync]
    [-r|--resizefs]
    [-t|--test]
    [--type VolumeType]
    [-v|--verbose]
    [--version]
     LogicalVolume[Path] [ PhysicalVolumePath... ]


The first thing we see might be exactly what we needed to know. “WARNING: Running as a non-root user. Functionality may be unavailable.” There probably isn't much you can do with this command if you don't have root access.

As you can see, the help gives us a great summary of the command options. For example: does lvextend use -l or -L to specify size? Who can remember such things? And why should you have to when the computer will remember them for you?

This illustrates exactly why you might want to consider getting comfortable using --help. I use the lvextend command all the time, but occasionally I can't remember if I want -l or -L to extend a volume by 1 gigabyte. It's far more time consuming to search the web for an answer to this type of question than it is to append --help to the end of the command.

Apropos

What if you needed to extend a logical volume but you didn't know about the lvextend command?

This is where apropos comes in handy.

Let's say all you know is that you need to make a volume bigger but have no idea how to do that. If you know that you can search man pages with apropos, then you should be able to solve this problem on your own, albeit with a bit of extra effort.

apropos logical volume

    alarm (2)            - set an alarm clock for delivery of a signal
    cryptsetup (8)       - manage plain dm-crypt and LUKS encrypted volumes
    dmsetup (8)          - low level logical volume management
    lvchange (8)         - change attributes of a logical volume
    lvconvert (8)        - convert a logical volume from linear to mirror or snapshot
    lvcreate (8)         - create a logical volume in an existing volume group
    lvdisplay (8)        - display attributes of a logical volume
    lvextend (8)         - extend the size of a logical volume
    lvm-lvpoll (8)       - Internal command used by lvmpolld to complete some Logical Volume operations.
    lvm2-activation-generator (8) - generator for systemd units to activate LVM2 volumes on boot
    lvmchange (8)        - change attributes of the logical volume manager
    lvreduce (8)         - reduce the size of a logical volume
    lvremove (8)         - remove a logical volume
    lvrename (8)         - rename a logical volume
    lvresize (8)         - resize a logical volume
    ....
    .....
    ....


I cut off some of the output for the sake of brevity, however, as you can see searching for logical volume with apropos will return a lot of results.

What you are seeing in the first column of this output is the name of a man page that will contain the information you need in order use a command.

The second column is a short description of what you will find in the man page. As you scan the results hopefully lvextend will catch your eye along with the many other lvm commands, so that you can learn all you need to learn about managing logical volumes.

Another thing to notice is that I ran apropos with two keywords logical and volume.

How did I know that I could pass two keywords into apropos? I read the man page... and now so will you.

How To Read Man Pages

Knowing how to read a man page is just as important as knowing how to find a man page.

In the last example I knew that apropos could take more than one keyword as input because I can read and understand the man page. Let's look at the man page for apropos and see if we can demystify some of the archaic documentation.

{{< highlight “hl_lines=7” >}} man apropos

APROPOS(1) Manual pager utils APROPOS(1) NAME apropos – search the manual page names and descriptions SYNOPSIS apropos [-dalv?V] [-e|-w|-r] [-s list] [-m system[,...]] [-M path] [-L locale] [-C file] keyword ... DESCRIPTION ... ... OPTIONS ... ... ... EXIT STATUS ... ... ... ENVIRONMENT ... ... ... FILES /usr/share/man/index.(bt|db|dir|pag) A traditional global index database cache. /var/cache/man/index.(bt|db|dir|pag) An FHS compliant global index database cache. /usr/share/man/.../whatis A traditional whatis text database. SEE ALSO man(1), whatis(1), mandb(8) AUTHOR Wilf. (G.Wilford@ee.surrey.ac.uk). Fabrizio Polacco (fpolacco@debian.org). Colin Watson (cjwatson@debian.org). 2015-11-06 APROPOS(1 man(1), whatis(1), mandb(8)

{{< /highlight >}}

I have obviously redacted quite a bit of this document so if you are following along in your terminal don't worry. There are just a few things I want to point out. First is the structure of the page.

NAME

The first section in every man page is the name of the program along with a short summary. The name and summary should look familiar if you found the page through an apropos search.

SYNOPSIS

This is the point at which most people start to let their eyes glaze over, and begin imagining how much fun it would be to poke yourself in that eye with a fork. Don't let this section intimidate you, this is where some of the best information can be found.

apropos [-dalv?V] [-e|-w|-r] [-s list] [-m system[,...]] [-M path] [-L locale] [-C file] keyword ...


Let's take a look at what each piece of this line means. * First it shows the command to run apropos pretty straight forward.

  • Next, ignore the letters and symbols inside the []brackets. What is important here is that they are inside [] brackets, which means that they are optional. You can use them, but you don't have to.

  • Some of the commands have special options of there own. Typically options separated with a | (pipe) indicate “or”. As in you can use [-e or -w or -r] but not all of them. In this case -s takes some kind of list, and -m system[ has some other optional stuff and things]. If you want to know what those optional things are you need to read about them in the options sections. Right now all we care about is that these commands have some special modifier and they are optional.

  • Next we come to keyword notice that this is not in brackets. Like apropos this value is required. If you just type apropos without a keyword the command will output an error. Anything that does not appear inside [] brackets is a required value.

  • I discovered that you can use multiple keywords because of the ... (three dots) that comes immediately after keyword. Whenever you see ... in any man page it means that you can specify multiple values for that option.

OPTIONS

This is the section that will explain what each option does. The options are those optional things in the [] brackets.

EXIT STATUS

Just like it say's, this section tells you what the possible exit codes will be after you run the command, and hopefully what they mean. Usually useful information if you are using the command in a script.

ENVIRONMENT

Details about the system environment that effect the operation of the command. Not all man pages will have this section.

FILES

Any relevant files that you may need or find useful will be listed here. Often times you will see the name and location for configuration files listed here.

Additionally, though not listed in this man page sometimes you will have a section labeled EXAMPLES that will show practical examples of how a command can be used, and why you might specify particular options. The man page for lvextend has several examples listed that we would find useful.

Man pages are also searchable with vim commands. If you want to skip through the doc to find out what the -C option does you can do that by typing /-C. Slash / is shorthand for search and -C is what we want to find.

You can also move the bottom of the page with SHIFT + G or to the top by typing gg.

To exit the man page type q.

Documentation

The /usr/share/doc directory contains all the documentation files that come with a particular package. Sometimes these files contain example configuration files. For instance the apache2 or httpd package (depending on disto) will put example configuration files in this directory. These examples might help you solve a problem on an exam or help you quickly set up a proof of concept without all the internet research.

There is an enormous amount of help available for Linux. Much of it is available directly on the system you are using. If you spend some time to get comfortable reading the documentation and playing with some of the options (in a safe environment not production) you will quickly find out that you don't always need a search engine.

In my opinion the ability to find and use information in the man pages quickly is probably the biggest difference between a really good Linux Administrator and a great one.

Now, the next time someone tells you to RTFM you will at least have a better idea of how to read the f.... friendly manual.


authors: [“Luke Rawlins”] date: 2017-06-12 22:12:25+00:00 draft: false title: RPM package queries description: “How to query RPM packages to find helpful information about installed software on a Linux system.” url: /rpm-package-query/ tags: – CentOS – OpenSUSE – rtfm – package management – rpm


This post is just a quick walk-through of some basic commands to help you find information about rpm packages.

These commands will work for any rpm based distribution (Red Hat, Centos, Suse, Mageia).

Debian based distributions like Ubuntu or Mint use dpkg instead of rpm and I'll cover those in a different post.

Query the rpm database

You can query the rpm database to find a particular installed package using the -q option.

With rpm -q you must also pass a package name. For example, to find out what version of the httpd server we have installed we can use rpm -q httpd/

rpm -q httpd
httpd-2.4.6-45.el7.centos.4.x86_64


List all installed packages

To get a quick list of every installed package on an rpm based Linux distribution you can use -qa.

In this case you do not need to pass any specific package into the command. Often times you will use this to find a package when you are not sure of the exact name. For example, you might grep for ruby to find all the installed ruby libraries.

Running rpm -qa | grep ruby will produce output similar to this on a Centos 7 server.

rpm -qa | grep ruby
    rubygem-json-1.7.7-29.el7.x86_64
    ruby-libs-2.0.0.648-29.el7.x86_64
    rubygems-2.0.14.1-29.el7.noarch
    ruby-irb-2.0.0.648-29.el7.noarch
    rubygem-bigdecimal-1.2.0-29.el7.x86_64
    ruby-2.0.0.648-29.el7.x86_64
    rubygem-rdoc-4.0.0-29.el7.noarch
    rubygem-io-console-0.4.2-29.el7.x86_64
    rubygem-psych-2.0.0-29.el7.x86_64


Find associated files

Using rpm -ql and rpm -qc will help you to locate files associated with a particular package.

This is a great tool to help you find your way around a newly installed application. For example, how could you find out that the configuration file for Apache can be found at /etc/httpd/conf/httpd.conf, without searching the interwebs?

rpm -qc httpd will list all of the configuration files.

rpm -qc httpd
    /etc/httpd/conf.d/autoindex.conf
    /etc/httpd/conf.d/userdir.conf
    /etc/httpd/conf.d/welcome.conf
    /etc/httpd/conf.modules.d/00-base.conf
    /etc/httpd/conf.modules.d/00-dav.conf
    /etc/httpd/conf.modules.d/00-lua.conf
    /etc/httpd/conf.modules.d/00-mpm.conf
    /etc/httpd/conf.modules.d/00-proxy.conf
    /etc/httpd/conf.modules.d/00-systemd.conf
    /etc/httpd/conf.modules.d/01-cgi.conf
    /etc/httpd/conf/httpd.conf
    /etc/httpd/conf/magic
    /etc/logrotate.d/httpd
    /etc/sysconfig/htcacheclean
    /etc/sysconfig/httpd


Similarly running rpm -ql will give you a list not only of configuration files but also every file that was installed on your server with its location.

Identify vendors

In some cases, you may need more information about a package. Who is the vendor? When was it installed? etc... Getting this type of information with rpm packages is easy.

rpm -qi httpd

    Name        : httpd
    Version     : 2.4.6
    Release     : 45.el7.centos.4
    Architecture: x86_64
    Install Date: Mon 12 Jun 2017 09:37:04 PM UTC
    Group       : System Environment/Daemons
    Size        : 9823677
    License     : ASL 2.0
    Signature   : RSA/SHA256, Thu 13 Apr 2017 01:04:44 AM UTC, Key ID 24c6a8a7f4a80eb5
    Source RPM  : httpd-2.4.6-45.el7.centos.4.src.rpm
    Build Date  : Wed 12 Apr 2017 09:05:23 PM UTC
    Build Host  : c1bm.rdu2.centos.org
    Relocations : (not relocatable)
    Packager    : CentOS BuildSystem <http://bugs.centos.org>
    Vendor      : CentOS
    URL         : http://httpd.apache.org/
    Summary     : Apache HTTP Server
    Description :
    The Apache HTTP Server is a powerful, efficient, and extensible
    web server.


Finding documentation

You can also quickly find where documentation for a package can be found on your system using rpm -qd.

rpm -qd httpd
    /usr/share/doc/httpd-2.4.6/ABOUT_APACHE
    /usr/share/doc/httpd-2.4.6/CHANGES
    /usr/share/doc/httpd-2.4.6/LICENSE
    /usr/share/doc/httpd-2.4.6/NOTICE
    /usr/share/doc/httpd-2.4.6/README
    /usr/share/doc/httpd-2.4.6/VERSIONING
    /usr/share/doc/httpd-2.4.6/httpd-dav.conf
    /usr/share/doc/httpd-2.4.6/httpd-default.conf
    /usr/share/doc/httpd-2.4.6/httpd-info.conf
    /usr/share/doc/httpd-2.4.6/httpd-languages.conf
    /usr/share/doc/httpd-2.4.6/httpd-manual.conf
    /usr/share/doc/httpd-2.4.6/httpd-mpm.conf
    /usr/share/doc/httpd-2.4.6/httpd-multilang-errordoc.conf
    /usr/share/doc/httpd-2.4.6/httpd-vhosts.conf
    /usr/share/doc/httpd-2.4.6/proxy-html.conf
    /usr/share/man/man8/apachectl.8.gz
    /usr/share/man/man8/fcgistarter.8.gz
    /usr/share/man/man8/htcacheclean.8.gz
    /usr/share/man/man8/httpd.8.gz
    /usr/share/man/man8/rotatelogs.8.gz
    /usr/share/man/man8/suexec.8.gz


This documentation will include listing man pages that might be available. As well as example configuration files. In this case you can see that /usr/share/doc/httpd-2.4.6/httpd-vhosts.conf is an example of a virtual host file.... Maybe something that would come in handy as a template for virtual hosts you might have to set up.

As you can see there is quite a lot of information that can be extracted from the rpm database. In my humble opinion, this is one of the big advantages that rpm has over dpkg (though you can get this information from dpkg, it's just not as straight forward), rpm makes it easy to query the database and quickly find the information you need.

All of this really only scratches the surface of what you can do as well. There are many ways to modify these commands to help you discover information about the packages you have installed on your system. I encourage you to read the full man page for rpm if you are interested in learning more in-depth capabilities for rpm packages.


authors: [“Luke Rawlins”] date: 2017-04-13 draft: false title: openSUSE patch vs update description: “The difference between patches and updates on an openSUSE system.” url: /opensuse-patch-vs-update/ tags: – Linux – OpenSUSE – patching – updates – zypper


openSUSE Logo

If you dig into the man pages for zypper, you will notice that zypper provides three distinct options for keeping your openSUSE system up-to-date; update (up), patch, and dist-upgrade (dup).

If you aren't familiar with zypper see my previous post managing packages with zypper for more information.

In this post I will attempt to demonstrate the differences between each option and suggest when you may want to consider using each. In particular, I will try to explain the difference between a simple update and a patch, with emphasis on how to gather detailed information on particular patches.

Update

According to the man page, using zypper update (or zypper up) will; “Update installed packages with newer versions, where possible.” As long as updating the package will not cause a change in vendor (see dist-upgrade), using zypper up will update every installed package to the newest version that is available in the repositories that are enabled on your system.

zypper update is a safe and reliable way to update any openSUSE or SUSE Enterprise Linux system, without worrying about major version changes.

To see a list of all available updates use zypper lu.

zypper lu
    Loading repository data...
    Reading installed packages...
    S | Repository | Name | Current Version | Available Version | Arch
    --+------------------------+---------------------+-----------------+-------------------+-------
    v | Main Update Repository | gimp | 2.8.18-1.4 | 2.8.18-2.3.1 | x86_64
    v | Main Update Repository | gimp-help-browser | 2.8.18-1.4 | 2.8.18-2.3.1 | x86_64
    v | Main Update Repository | gimp-lang | 2.8.18-1.4 | 2.8.18-2.3.1 | noarch
    v | Main Update Repository | gimp-plugins-python | 2.8.18-1.4 | 2.8.18-2.3.1 | x86_64
    v | Main Update Repository | libgimp-2_0-0 | 2.8.18-1.4 | 2.8.18-2.3.1 | x86_64
    v | Main Update Repository | libgimpui-2_0-0 | 2.8.18-1.4 | 2.8.18-2.3.1 | x86_64


For most users this is all you will probably ever need to know about keeping your openSUSE system updated.

If you are using openSUSE as your desktop OS you may notice that running zypper up will show more updates than you see through the GUI. This is because by default Yast Online Update (which is where the gui tools in gnome, and kde retrieve update information) only shows official software patches.

Patches

While patches in openSUSE are similar to updates in that they will install updated packages. Patches are meant for specific bug fixes and security fixes for software that comes packaged by openSUSE and is maintained in the Main Updates repository. A single patch might include several package updates to mitigate a specific security vulnerability or bug fix.

In many cases installing patches will not fully update your system. A package with an updated version number that doesn't match a “fix” for a bug or security flaw will not be included in a patch. Installing “Patches” rather than “Updates”, strictly speaking, is probably only necessary for production environments that can only handle minimal changes to installed packages, while still maintaining a security.

However, if you want to ensure that you have a stable and undisturbed desktop experience, there is certainly nothing wrong with limiting your updates to patches.

One of the great things about working with patches is the vast amount of information that is available for them that can be accessed straight from the command line. For example, if we wanted to know which cve's (Common Vulnerabilities and Exposures) are currently affecting our system we could find out by using the list-patches (lp) option with zypper like this.

zypper lp --cve
Issue | No. | Patch | Category | Severity | Interactive | Status | Summary  
------+---------------+-------------------+----------+----------+-------------+--------+-------------------------  
cve | CVE-2007-3126 | openSUSE-2017-462 | security | moderate | --- | needed | Security update for gimp


This command gives us a lot of good information that can be useful for explaining the reasons that a patch is necessary. In this case, we see the cve number, the name of the patch (openSUSE-2017-462), its category (security), the severity (moderate), whether or not the patch is needed, and a quick summary (Security update for gimp).

We can take this a step further and pull down all the details of what packages will change if we patch this particular cve, by calling the info option with zypper and passing in the name of the patch we want to look at in this case openSUSE-2017-462.

zypper info openSUSE-2017-462


You can also send this command more specifically (but with identical output) as.

zypper info --type patch openSUSE-2017-462


When I'm writing a script I would tend to use the more verbose method, just so that the next person looking at it (or me 6 months later) will have a better chance of understanding what I was doing.

Either way you choose to run one of those commands you will receive output similar to this:

Information about patch openSUSE-2017-462:

    Repository : Main Update Repository
    Name : openSUSE-2017-462
    Version : 1
    Arch : noarch
    Vendor : maint-coord@suse.de
    Status : needed
    Category : security
    Severity : moderate
    Created On : Wed 12 Apr 2017 05:15:35 AM EDT
    Interactive : ---
    Summary : Security update for gimp
    Description :
    This update for gimp fixes the following issues:

    This security issue was fixed:

    CVE-2007-3126: Context-dependent attackers were able to cause a denial of service via an ICO file with an InfoHeader containing a Height of zero<br>(bsc#1032241).

    These non-security issues were fixed:

    bsc#1025717: Prefer lcms2 over lcms1 if both are available
    bgo#593576: Preven crash in PDF Import filter when importing large image PDF or specifying high resolution
    Provides : patch:openSUSE-2017-462 = 1

    Conflicts : gimp.i586 &lt; 2.8.18-2.3.1 gimp.src 2.8.18-2.3.1 gimp-debuginfo.i586 2.8.18-2.3.1 gimp-debugsource.i586 2.8.18-2.3.1 gimp-devel.i586 2.8.18-2.3.1 gimp-devel-debuginfo.i586 &lt; 2.8.18-2.3.1 gimp-help-browser.i586 2.8.18-2.3.1
    ...
    ...
    This update for gimp fixes the following issues:
    This security issue was fixed:

    CVE-2007-3126: Context-dependent attackers were able to cause a denial of service via an ICO file with an InfoHeader containing a Height of zero<br>(bsc#1032241)

    These non-security issues were fixed:
    bsc#1025717: Prefer lcms2 over lcms1 if both are available
    bgo#593576: Preven crash in PDF Import filter when importing large image PDF or specifying high resolution
    Provides : patch:openSUSE-2017-462 = 1
    Conflicts : [36]
    gimp.i586 < 2.8.18-2.3.1
    gimp.src < 2.8.18-2.3.1
    gimp-debuginfo.i586 < 2.8.18-2.3.1
    gimp-debugsource.i586 < 2.8.18-2.3.1
    gimp-devel.i586 < 2.8.18-2.3.1
    gimp-devel-debuginfo.i586 < 2.8.18-2.3.1
    gimp-help-browser.i586 < 2.8.18-2.3.1
    gimp-help-browser-debuginfo.i586 < 2.8.18-2.3.1
    ...


As you can see this particular command string gives us plenty of details about the patch.

Everything from basic information about the repository it will be coming from, to the date it was implemented, and a more full description including how the vulnerability may affect you.

We can see both the security issues and non-security issues that are fixed.

One of the potential points of confusion in the output comes in the last section Conflicts :. The word “Conflicts” might seem to suggest that some of the currently installed packages conflict with updates that are needed to patch the vulnerability. This is not the way you should understand conflicts in this context.

From the zypper man pages: “A released patch conflicts with the affected/vulnerable versions of a collection of packages. As long as any of these affected/vulnerable versions are installed, the conflict triggers and the patch is classified as needed, optional or as unwanted if the patch is locked.” In proper context, the conflict is a trigger to let us and the system know that an updated package is available to fix the vulnerability. So, in fact, the conflict section can be read like a list of packages that are going to be updated, in order to remove the conflict.

To install all available patches you can simply run

sudo zypper patch


or to install patches to resolve a specific cve

sudo zypper install patch:openSUSE-2017-462


You can also list all patches that are categorized as “security” fixes.

zypper lp --category security


There are other ways to find patches that are available by listing bugzilla reports and severity, but I will let you find those on your own after you do a little more reading, and have become familiar with the commands listed here.

Distribution Upgrade (dup)

A distribution upgrade, performed by running: sudo zypper dup , will upgrade (or downgrade) all installed packages to the latest version listed in every enabled repository, and as such, it should be used with caution.

The upgrade operation will be performed regardless of vendor or repository and is often used when you want to replace an official package with one from a 3rd party repository, such as packman.

On a production system or system that needs to be kept in a stable state you will not want to use the dist-upgrade option except in certain situations. Most likely one of these:

  1. Upgrading from one openSUSE stable release to another i.e. (Leap 42.1 to Leap 42.2)
  2. Switching packages from one vender repo to another i.e. (openSUSE to packman)
  3. You are using Tumbleweed and want to keep your system up-to-date.
    1. Ostensibly you could still use zypper up.
    2. I have found some documentation that would suggest using zypper dup with special switches whenever you update under Tumbelweed (sudo zypper dup --no-allow-vendor-change) https://en.opensuse.org/Portal:Tumbleweed

I very rarely use the dist-upgrade option on my own systems, though I hear it is more common with the rolling release (Tumbleweed).

I would highly recommend spending some time reading the zypper man pages if you are planning to do any serious work with zypper.

As I've said before zypper is a great (probably the best) package manager that is available in any Linux distribution. It is a powerful tool that makes managing packages easier and faster than anything else that I've worked with.


authors: [“Luke Rawlins”] date: 2017-02-21 draft: false title: Working with files in Linux – File Attributes description: “Whereas, permissions and ACL's deal with user and group access to a file, attributes are properties of a file that regulate how the operating system interacts with a given file.” url: /linux-file-attributes/ tags: – file permissions – filesystem – Linux – permissions


In the previous two posts, we've looked at file permissions and access control lists.

Today let's take look at file attributes. Whereas, permissions and ACL's deal with user and group access to a file, attributes are properties of a file that regulate how the operating system interacts with a given file.

There are 15 file attributes: append only (a), no atime updates (A), compressed (c), no copy on write (C), no dump (d), synchronous directory updates (D), extent format (e), immutable (i), data journalling (j), project hierarchy (P), secure deletion (s), synchronous updates (S), no tail-merging (t), top of directory hierarchy (T), and undeletable (u).

Even though there are 15 attributes it's been my experience that (i) immutable, and (a) append only attributes seem to be the only ones that work consistently across filesystems.

If you are using an ext2, ext3, or ext4 file system then c,s, and u will not be honored by your file systems.

The secure delete (s), and undeletable (u) attributes would definitely be a nice-to-have feature in the future. If you are using btrfs (likely on a SUSE or OpenSUSE based system) then (c) compression will be respected along with a few others (feel free to experiment).

The only attribute flag that I want to look at today is (i) immutable. The reason I have singled this attribute out is to demonstrate the power you have with this flag, and because it's the only one I use on any regular basis.

List attributes with lsattr.

Let's make a file and add some content to it:

luke@Tumbleweed01:~/Documents> touch file1
luke@Tumbleweed01:~/Documents> echo "Some awesome file content" >> file1
luke@Tumbleweed01:~/Documents> cat file1
    Some awesome file content
luke@Tumbleweed01:~/Documents>


What attributes does this file have upon creation?

luke@Tumbleweed01:~/Documents> lsattr file1
------------------- file1
luke@Tumbleweed01:~/Documents>


You will notice that newly created files have no attributes (at least if you are using openSUSE with btrfs like I am).

If you are using Ubuntu you will very likely see the e flag which means that your file system is using extents for mapping the file to disk.

Making a file immutable with chattr

What does it mean to make a file “immutable”?

Immutable means that the file cannot be changed in any way. You can't delete it, you can't add new data, or remove old data, or even change its name. When you add the i attribute you have chiseled the file into stone. Not even root can change a file that has been marked immutable (at least not without removing the flag first).

Attribute flags can be added and removed using the chattr command with +/-and the attribute.

sudo chattr +i file1


Now try to add another line to this file.

luke@Tumbleweed01:~/Documents> echo "Even more awesome content" >> file1
bash: file1: Operation not permitted
luke@Tumbleweed01:~/Documents>

What about as root? Still not permitted right?

Can you delete it?

luke@Tumbleweed01:~/Documents> rm file1
rm: cannot remove 'file1': Operation not permitted
luke@Tumbleweed01:~/Documents> sudo rm file1
rm: cannot remove 'file1': Operation not permitted
luke@Tumbleweed01:~/Documents>


Notice the message output Operation not permitted even when using sudo.

Our POSIX permissions have not changed if you check:

luke@Tumbleweed01:~/Documents> ls -l file1
-rw-r--r-- 1 luke users 26 Feb 21 10:49 file1


You can see that my user should have read and write access to this file. The file attribute tells the filesystem that the contents of this file are frozen and cannot change.

Remove the immutable flag by replacing +i with -i:

luke@Tumbleweed01:~/Documents> sudo chattr -i file1e


What is the use case for this?

I see using these attributes as only a single layer to defend files that I want to protect against accidental deletion or to prevent them from being altered.

For example, I use a media server called Plex to stream photos and movies to other devices in my home. I use the immutable flag to prevent files from being accidentally deleted or maliciously deleted in the event that the wrong person was to gain access to my Plex server.

File attributes should be seen as a way to further layer your file access scheme, and not as a replacement for best practices.


authors: [“Luke Rawlins”] date: 2017-02-06 draft: false title: Working with Linux Files – Access Control Lists url: /linux-acl-access-control-list/ description: “Using ACL's we can add a list of users, or groups, or both that have permissions to a file that are separate from the standard owner/group in basic POSIX permissions.” tags: – access control list – acl – file permissions – Linux – permissions


In the last post, we looked at basic file permissions. The ideas covered in that post are probably enough to get you through a large portion of the real world scenario's that you will encounter. There are some special cases, however. One of them being access control lists (ACL) which I will discuss in this post.

Access Control List – ACL

As we saw in part 1 every file on a Linux system has an owner and a group associated with it, each of which has separate permissions. But what if a user or group needed read permissions to a file that they do not own? You could modify permissions to allow all users access to a file with chmod o+r  but this is not ideal on a system with multiple users where data needs to be kept confidential. Instead of going this route, which is insecure, we can add a list of users, or groups, or both that have permissions to a file that are separate from the standard owner/group in basic POSIX permissions.

Using ACL's

So let's look at a simple example. Batman and the finance group need to be able to read and write a payroll document, Superman and Robin need to be able to write, but not read, the payroll document (we don't want them to know that Batman makes more money than they do).

First, let's set up our environment:

sudo useradd batman
sudo useradd superman
sudo useradd robin
sudo groupadd finance
sudo usermod -aG finance batman

Set a password for each user. I'm just going to use the username for each as the password to make it easy for this example. It should go without saying that this wouldn't be an acceptable password policy for real world use.

    sudo passwd batman
    New password:
    BAD PASSWORD: it is based on a dictionary word
    BAD PASSWORD: is too simple

    sudo passwd superman
    New password:
    BAD PASSWORD: it is based on a dictionary word
    BAD PASSWORD: is too simple

    sudo passwd robin
    New password:
    BAD PASSWORD: it is based on a dictionary word
    BAD PASSWORD: is too simple

Switch to the user batman and create a file called jlpayroll.txt in the /tmp directory.

    su - batman
    cd /tmp
    newgrp finance
    touch jlpayroll.txt
    chmod 660 jlpayroll.txt
    ls -l jlpayroll.txt
    -rw-rw---- 1 batman finance 0 Feb  5 09:32 jlpayroll.txt

At this point, batman can read and write to the jlpayroll.txt file. Now add batman's pay to this file.

{{< highlight bash >}} echo “Batman Week One: 2,000,000” >> jlpayroll.txt {{< /highlight >}} Cat out the file to see what's in it.

    cat jlpayroll.txt
    Batman Week One: 2,000,000

Now we need to add ACL's to allow superman and robin to add their payroll information, while also ensuring that they can't read the file. Batman can do that with the setfacl command that he keeps in his utility belt.

    setfacl -m u:superman:w -m u:robin:w jlpayroll.txt

Take a look at this command for a minute. First, notice that you can add multiple users at the same time each separated by the -m option. In the setfacl command -m means modify.

Next, notice the format that is used to identify users. The setfacl command expects 3 values separated by colons:

type:name:permission – the type can be “u” for user or “g” for group, the name is the group or username, and the permissions are the standard read, write, and execute permissions that we looked at previously.

So how does this affect the way permissions are displayed?

ls -l jlpayroll.txt -rw-rw----+ 1 batman finance 27 Feb 5 09:43 jlpayroll.txt

The change is subtle but it's there. Notice the "+" after "-rw-rw----". That + sign indicates that permissions have been altered by access control lists.

How to display the ACL's on a file.

Access control list's can be displayed with the getfacl command.

    getfacl jlpayroll.txt
    # file: jlpayroll.txt
    # owner: batman
    # group: finance
    user::rw-
    user:superman:-w-
    user:robin:-w-
    group::rw-
    mask::rw-
    other::---

The output here is a detailed look at exactly who has what level of access to this file. It lists the file name, owner, and group. Then each user and group have individual access level's displayed.

Testing the ACL's

Switch to the user superman and make sure he can write to the jlpayroll.txt file.

    su - superman
    cd /tmp
    id
    uid=1007(superman) gid=100(users) groups=100(users)
    ls -l jlpayroll.txt
    -rw-rw----+ 1 batman finance 27 Feb  5 09:43 jlpayroll.txt

We can see that superman is not in the finance group and he is not the owner of this file. Normally he wouldn't have any access to this file at all. But with the ACL we set up earlier he should be able to write to this file but not read it.

    cat jlpayroll.txt
    cat: jlpayroll.txt: Permission denied

As you can see superman cannot read the jlpayroll.txt file. Can he write to it?

    echo "Superman Week One: 400" >> jlpayroll.txt

As long as that command doesn't return an error then it should've worked. Let's do the same thing for robin

    su - robin
    cd /tmp
    echo "Robin Week One: 4,000" >> jlpayroll.txt

Since payroll is done for the week lets make sure that batman can see each entry. If you've been following along you can get back to batman using the exit command twice.

    cat jlpayroll.txt
    Batman Week One: 2,000,000
    Superman Week One: 400
    Robin Week One: 4,000

Access control lists allow an administrator to exercise fine-grained control over files and directories that would not be possible using only POSIX permissions. This was a pretty simple example but it should give you an idea about how ACL's work, how to set them, and how to discover what ACL's are set on a file.

 Clean up

When you are finished with this example make sure you clean up your system to get rid of the users and group we made at the beginning of the tutorial. Switch back to your normal user account with sudo privileges.

    sudo userdel batman
    sudo userdel superman
    sudo userdel robin
    sudo groupdel finance
    sudo rm /tmp/jlpayroll.txt


authors: [“Luke Rawlins”] date: 2017-01-26 draft: false title: Working with files in Linux – Permissions description: “Every file in Linux has three primary permissions settings (read, write, execute) that apply to three elements (owner, group, others).” url: /linux-file-permissions/ tags: – chmod – chown – file permissions – Linux – posix – posix permissions


Over the next few posts I'll be covering three basic elements of files in Linux:

The ls command

Every file in Linux has three primary permissions settings (read, write, execute) that apply to three elements (owner, group, others).

File permissions can be viewed on the command line using the ls command.

{{< highlight “hl_lines=3” >}} [luke@testserver stuff]$ ls -l total 0 -rwxrw-r-x 1 luke admins 0 Jun 21 19:44 file1 {{< /highlight >}}

Looking at the output from ls -l, from left to right we can break the output into several groups as shown below. Each group separated by parenthesis.

[ (–) (rwxrw-r-x) (1)  (luke admins)  (0) (Jun 21 19:44)  (file1) ]

Let's look at each block separately.

- The leading dash tells us that this is a normal file. You may also see d (directory), l (link), or b (block device) and a few others. but -,d, and l will cover 90+ % of the files you will come into contact with.

Continuing down the line we have rwxrw-r-x this should be viewed as three sets of permissions (r) read, (w) write, and (x) execute. Each of the 3 permissions are applied to the owner, group, and others (others is everyone else on the system).

In this example, the permissions read as follows

Owner: rwx (read, write, execute)

Group: rw- (read, and write). The - indicates that the group does not have execute permissions.

Other: r-x (read, and execute) In this case others can read and execute but not write to the file, as indicated by the - in the “w” place.

1 Continuing to the right we see the number 1.

This number represents the link count for a file. This file has 1 link which is to itself, if we created a link (shortcut) to this file from another location then the link count would be 2 and would increase by one for each additional link.

The next two entries luke admins represent the file owner (luke) and the group that has permission to the file (admins).

0 The number after the group listing is the file size in bytes. In this case, it's 0. To see the file size in human readable form use ls -lh .

Jun 21 19:44 This section shows the date and time that the file was last modified.

file1 Last we see the file name.

Changing file ownership

Files in Unix-like operating systems belong to a single user (the owner) and a group. Only the root user can change the ownership of a file or directory.

To change ownership of a file use the chown command like this:

chown <user>:<group> file


Here are a few examples:

Change user and group of a file:

sudo chown superman:justiceleague goodguy.file


Change only the user: 

sudo chown superman goodguy.file


Change only the group:

sudo chown :justiceleague goodguy.file


Changing Permissions

File permissions are changed with the chmod command. Permissions can be modified using two different formats, numerical and symbolic.

File permissions in Linux are coded symbolically (as letters)

  • r – read
  • w – write
  • x – execute

And Numerically:

  • 4 – read
  • 2 – write
  • 1 – execute

Each applies to the user(owner), group, and other.

For example, if a file has the following permissions

{{< highlight “hl_lines=2” >}} ls -lh myawesomefile.txt -rw-r—r— 1 luke users 9.3M Jan 21 21:43 myawesomefile.txt

{{< /highlight >}}


  • User – read, write
  • Group – read
  • Others -read

If we wanted to change this so that users can execute this file we can make this change in one of two ways.

Symbolically

chmod u+x myawesomefile.txt


In this example u= user and x = execute.

You can remove the execute permission by changing the + to a -.

chmod u-x myawesomefile.txt


Changing permissions symbolically uses (ugoa) user, group, other, all

Give group write permission on myawesomefile.txt.

chmod g+x myawesomefile.txt


Give all users execute permission.

chmod a+x myawesomefile.txt

Numerically 

Changing permissions numerically is intimidating for new Linux users but it shouldn't be. If you can add up to seven then you should be fine.

Let's make a new file called USMC.OORAH and then display the permissions.

{{< highlight “hl_lines=2” >}} touch USMC.OORAH; ls -lh USMC.OORAH -rw-r—r— 1 luke users 0 Jan 25 19:50 USMC.OORAH

{{< /highlight >}}

We can see permissions are rw-r--r--(User read/write, Group read only, Others read only)

These permissions can be expressed numerically as 644 which is admittedly much higher than seven. However, this number is not six hundred forty-four.

It is six, four, four.

Numerically file permissions are always Read + Write + Execute = numerical permission 1. Read is equal to the number 4. 2. Write is equal to the number 2. 3. Execute is equal to the number 1.

Our file has numerical permissions of six, four, four – and here is how that breaks down:

  • User = 6 (read/write because read (4) + write (2) + execute (0) = 6)
  • Group = 4 (read only because read (4) + write (0) + execute (0) = 4)
  • Others = 4 (read only because read (4) + write (0) + execute (0) = 4)
  • Remember read = 4, write =2, and execute = 1 and each applies to users groups and others.
    • user permissions add up to 6.
    • group permissions add up to 4.
    • others permissions add up to 4.
    • Which is generally expressed as just 644.

Lets change permissions on our file to allow the group to write to the file, while keeping everything else the same. We need to add 2 only to the group portion of our permissions.

{{< highlight “hl_lines=3” >}} chmod 664 USMC.OORAH ls -lh USMC.OORAH -rw-rw-r— 1 luke users    0 Jan 25 19:50 USMC.OORAH

{{< /highlight >}}

What if we wanted to allow others to execute this file? Currently, others can only read which is represented by the number 4 execute is represented by the number 1. 4+1=5 so we will want to change permissions to 665.

{{< highlight “hl_lines=3” >}} chmod 665 USMC.OORAH ls -lh USMC.OORAH -rw-rw-r-x 1 luke users 0 Jan 25 19:50 USMC.OORAH

{{< /highlight >}}

Full permission, read write and execute, is represented by the number 7 because 4+2+1 is 7. So to give the user read write and execute on our file:

{{< highlight “hl_lines=3” >}} chmod 765 USMC.OORAH ls -lh USMC.OORAH -rwxrw-r-x 1 luke users 0 Jan 25 19:50 USMC.OORAH {{< /highlight >}}

Of course, you can take permissions away by subtracting. To change the file back to its original permissions of rw-r--r--

{{< highlight “hl_lines=3” >}} chmod 644 USMC.OORAH ls -lh USMC.OORAH -rw-r—r— 1 luke users 0 Jan 25 19:50 USMC.OORAH {{< /highlight >}}

My next post will take this one step further and add ACL's or access control lists to the permissions scheme which allows our permissions be become much more fine-grained and can include multiple users and groups.



authors: [“Luke Rawlins”] date: 2016-12-27 draft: false title: Search and replace with Vim and Sed description: “Using search and replace is a great way to save time when editing large files in Linux.” url: /search-and-replace-with-vim-and-sed/ tags: – search and replace – sed – VIM


Using search and replace is a great way to save time when editing large files in Linux.

Becoming proficient with this task will increase your efficiency and will reduce your time spent doing tedious and error-prone file edits by hand.

For the sake of this tutorial, I'm going to use a copy of the /etc/apt/sources.list file to illustrate some of the changes that we can make.

From your users home directory copy the sources.list file to your home directory. Or you can copy the contents of my list, shown a bit further down, into a new file.

This tutorial assumes that you are comfortable reading and writing files with vim. If not open a terminal and type: vimtutor

sudo cp /etc/apt/sources.list /home/luke/sources.txt
sudo chown $USER:$USER sources.txt


Here are the contents of my sources.list file.

cat sources.txt
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial main restricted
deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial-updates main restricted
deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial universe
deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial universe
deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial-updates universe
deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial multiverse
deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial multiverse
deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial-updates multiverse
deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial-updates multiverse

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial-backports main restricted universe multiverse
deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ xenial-backports main restricted universe multiverse

deb http://security.ubuntu.com/ubuntu xenial-security main restricted
deb-src http://security.ubuntu.com/ubuntu xenial-security main restricted
deb http://security.ubuntu.com/ubuntu xenial-security universe
deb-src http://security.ubuntu.com/ubuntu xenial-security universe
deb http://security.ubuntu.com/ubuntu xenial-security multiverse
deb-src http://security.ubuntu.com/ubuntu xenial-security multiverse


 Search and Replace with Vim

In vim, you can use the :substitute command, which is pretty much always abbreviated as just :s followed by a pattern that is separated by forward slashes like this. :s/find/replace/ . This command tells vim to search the current line for “find” and replace it with with “replace”.

To perform a search of all lines change :s to :%s . The % in front of “s” tells Vim that we want to search every line and will replace the first instance of our search term with our replace term as this example will show.

vim sources.txt
:%s/xenial/yakkety/


Pressing enter will show give you a summary of how many changes are being made. You can review the file for accuracy and then save the file or quit without saving. Notice that every line is searched.

Vim global search and replace

To change every instance of a word and not just the first instance on a line we need to add the global option to our command.

vim sources.txt
:%s/ubuntu/OLD YELLER/g


Here you can see that every instance of “ubuntu” has been changed to “OLD YELLER”.

If you wanted to find and delete a word you can use the format %s/search term// leaving the replace field empty. For example to remove comment's from the file.

:$s/#//


Next we'll see how all this is done without opening an interactive file editor.

Sed – Stream Editor

The command syntax for sed actually isn't much different from vim. By default sed will output to standard output and will not make changes to a file unless you specify options to allow it to save changes.

Similar to our vim example to change every occurrence of xenial to yakkety:

sed 's/xenial/yakkety/' sources.txt


sed global edit

As with vim, you can use the “g” option to make global changes. By default sed will only change the first instance of a search term unless you append g to the end of the search line.

sed 's/ubuntu/OLD YELLER/g' sources.txt


Deleting words from a file works the same way as vim as well.

sed 's/ubuntu//g' sources.txt


sed with in-place editing

You can make changes to files with sed-i. Using the -i option will create a backup copy of the original file before making edits, which will save you from inadvertently ruining a working configuration.

sed -i.bak 's/xenial/yakkety/' sources.txt


Learning to use search and replace in sed and vim will make your life as a Linux Administrator far easier and is well worth the time you will spend becoming comfortable with using these tools.


authors: [“Luke Rawlins”] date: 2016-12-17 draft: false title: Command not found! description: “Here is how to locate the packages you need to install in order to use commands that are not available on your system.” url: /command-not-found/ tags: – apt – cnf – package management – yum – zypper – CentOS – openSUSE – Ubuntu


So you're running through some instructions to configure software on your system, or troubleshoot some problem with a service and you see an error at the command line that says “command not found”. Here is how to locate the packages you need to install in order to use commands that are not available on your system.

CentOS/Red Hat – yum provides

Yum is an excellent package manager with lots of great built in functions. Using yum provides some_command_name will output a list of packages that provide the command you are trying to run.

Here is an example for what you would see if you were searching for the vgscan command.

sudo yum provides vgscan
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: repo1.sea.innoscale.net
* epel: mirror.cogentco.com
* extras: mirror.cisp.com
* nux-dextop: mirror.li.nux.ro
* updates: ftp.linux.ncsu.edu
7:lvm2-2.02.166-1.el7.x86_64 : Userland logical volume management tools
Repo        : base
Matched from:
Filename    : /usr/sbin/vgscan



7:lvm2-2.02.166-1.el7_3.1.x86_64 : Userland logical volume management tools
Repo        : updates
Matched from:
Filename    : /usr/sbin/vgscan


Something else to keep in mind is that yum provides can also search for files.

For example if you have a file on your system that you would like to match to a specific package or service yum can get that information for you. For example you might not be sure which package installed the file /etc/sysconfig/authconfig yum provides can get that information for you.

{{< highlight “hllines=9 17 25” >}} sudo yum provides /etc/sysconfig/authconfig Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: repo1.sea.innoscale.net * epel: mirror.cogentco.com * extras: mirror.cisp.com * nux-dextop: mirror.li.nux.ro * updates: ftp.linux.ncsu.edu authconfig-6.2.8-14.el7.x8664 : Command line tool for setting up authentication from network : services Repo : base Matched from: Filename : /etc/sysconfig/authconfig

authconfig-6.2.8-14.el7.x86_64 : Command line tool for setting up authentication from network : services Repo : installed Matched from: Filename : /etc/sysconfig/authconfig

authconfig-6.2.8-10.el7.x86_64 : Command line tool for setting up authentication from network : services Repo : @base Matched from: Filename : /etc/sysconfig/authconfig

{{< /highlight >}}

Ubuntu

With Ubuntu 14.04 and up you don't need to run a special command to find a program. For instance if you try to run the command sar without having first installed sysstat you will see the following message:

{{< highlight “hl_lines=5” >}} luke@test-srv01:~$ sar The program 'sar' can be found in the following packages: * sysstat * atsar Try: sudo apt-get install {{< /highlight >}}

It even tells you how to install the packages you need at the end of the message. Assuming you read the error messages you get when something doesn't work.... Some of us may or may not be guilty of neglecting to pay attention to error messages.

OpenSUSE/Suse Enterprise Linux – cnf

Similar to Ubuntu running a command that doesn't exist on your system will provide a suggestion to find the command you need.

luke@test-srv02:~> sar
If 'sar' is not a typo you can use command-not-found to lookup the package that contains it, like this:
    cnf sar


OpenSUSE suggests that we run another command (cnf) to find our package.

{{< highlight “hl_lines=6 7” >}} luke@test-srv02:~> cnf sar

The program 'sar' can be found in the following package: * sysstat [ path: /usr/bin/sar, repository: zypp (SMT-httpsmt-ec2susecloud_net:SLES12-SP2-Pool) ]

Try installing with: sudo zypper install sysstat {{< /highlight >}}

Suse like Ubuntu gives us a suggestion to install sysstat and even provides the full command to get it. A simple copy and paste should be enough to get the package you want and get back to work.



authors: [“Luke Rawlins”] date: 2016-12-15 00:44:25+00:00 draft: false title: What to do when df and du report different usage. description: “You may occasionally come across an issue where running df will produce output that disagree's with the output of the du command.” url: /df-and-du-report-different-usage/ tags: – filesystem – storage – troubleshooting


You may occasionally come across an issue where running df will produce output that disagree's with the output of the du command.

If you aren't familiar with these two commands do see my post about filesystem and directory size.

The reason for the difference in reported size is that df does not differentiate between files that are open in memory but have been deleted, or altered on the disk, whereas du will only see the files that are on the disk. You should recognize that these tools serve different functions and that you will need to rely on both of them to get a truly accurate portrayal of disk usage on your system.

Lets say you run df -h to get an idea of how much space you have on each of the filesystems on your server or PC only to see that /var is 98% full, 9.8G out of 10G just to keep it simple. Like a good admin you run du -h --max-depth=1 /var to find out which directories are the largest and may have files that need to be zipped up, moved, or deleted. The problem becomes apparent when du returns that just 3G are in use on that filesystem. What do you do now?

Check for deleted files in memory.

Have you heard the old saying around the Unix world, “Everything is a file”?

Well it's true, everything in Unix, and by association Linux, is a file. This includes deleted files that now live as chunks of memory that are in use by a process.

You can view all open files on a system with the lsof command, including deleted files that live in memory and are in use by a process (possibly an old configuration file). sudo lsof | grep root will show you a full output of all the files currently in use by the root user. (Probably a lot of files).

Running sudo lsof | less will show you all of the open files on your system. It will look something like this. (I'm only grabbing the first 3 lines for brevity).

COMMAND     PID   TID             USER   FD      TYPE             DEVICE SIZE/OFF       NODE NAME
systemd       1                   root  cwd       DIR              202,1     4096          2 /
systemd       1                   root  rtd       DIR              202,1     4096          2 /
systemd       1                   root  txt       REG              202,1  1577232     396000 /lib/systemd/systemd


Here you can see the command, the process id (PID), which user has the file open, the file descriptor (FD), the size in bytes, and the location. In our scenario we want to find out if there are any large files open that may have been deleted. We can find those files like this:

sudo lsof | grep -i deleted


Keep an eye on the 8th column which if you recall is the SIZE column. Once you identify your large files check which user has the file open (4th column), usually this will be a service account like www-data, apache, mysql. Or pay attention to the command column to identify the process or service that is using the old file.

After you identify the offending process all you need to do is restart the service using systemctl, service, or kill -HUP

In conclusion

Don't panic, take a breath, and assess what you are seeing, think about how your tools work and what they are showing you. Above all don't just start deleting things to free up space!

The reason df and du are having a disagreement here is that df see's these deleted files along with their replacements and calculates the total disk usage, du on the other hand only see's the new file.

Now that you know how to find the zombie files you shouldn't have too much trouble bringing these two system tools back into agreement.



authors: [“Luke Rawlins”] date: 2016-12-14 draft: false title: Managing packages with zypper description: “Zypper is a fast easy to use package management tool. In my opinion, zypper is hands down the best package manager out there. It provides meaningful, easy to read output, it resolves package dependencies quickly, and it has a cool name.” url: /managing-packages-with-zypper/ tags: – OpenSUSE – package management – Suse – zypper


Geeko

Suse Enterprise Linux and it's upstream community distro openSUSE use a package manager called zypper.

zypper is a fast easy to use package management tool. In my opinion, {{}}zypper{{}} is hands down the best package manager out there. It provides meaningful, easy to read output, it resolves package dependencies quickly, and it has a cool name.

zypper has many options that can be abbreviated; {{}}install (in){{}}, {{}}search (se){{}}, {{}}info (if){{}}, {{}}update (up){{}} and lots of others.

Basic package operations

Search for a package

Search with zypper search or {{}}zypper se{{}}.

luke@OpenSUSE-tst:~> sudo zypper search apache2
Loading repository data...
Reading installed packages...
S | Name                       | Summary                                                              | Type   
--+----------------------------+----------------------------------------------------------------------+--------
| apache2                    | The Apache Web Server Version 2.4                                    | package
| apache2-devel              | Apache 2 Header and Include Files                                    | package
| apache2-doc                | Additional Package Documentation                                     | package
| apache2-example-pages      | Example Pages for the Apache 2 Web Server                            | package
| apache2-icons-oxygen       | Oxygen icons for Apache 2                                            | package
| apache2-mod_apparmor       | AppArmor module for apache2                                          | package
| apache2-mod_auth_kerb      | Kerberos Module for Apache                                           | package
| apache2-mod_authn_otp      | Apache module for one-time password authentication                   | package
| apache2-mod_dnssd          | Apache2 module for Zeroconf support via DNS-SD                       | package
| apache2-mod_evasive        | Denial of Service evasion module for Apache                          | package
| apache2-mod_fastcgi        | A FastCGI Module for Apache 2                                        | package
| apache2-mod_fcgid          | Alternative FastCGI module for Apache2                               | package
| apache2-mod_jk             | Connectors between Apache and Tomcat Servlet Container               | package
| apache2-mod_mono           | Run ASP.NET Pages on Unix with Apache and Mono                       | package
| apache2-mod_nss            | SSL/TLS module for the Apache HTTP server                            | package
| apache2-mod_perl           | Embedded Perl for Apache                                             | package
| apache2-mod_perl-devel     | Embedded Perl for Apache - Development package                       | package
| apache2-mod_php5           | PHP5 Module for Apache 2.0   
| apache2-mod_php7           | PHP7 Module for Apache 2.0                                           | package
| apache2-mod_proxy_uwsgi    | uWSGI Proxy Module for Apache 2.0                                    | package
| apache2-mod_python         | A Python Module for the Apache 2 Web Server                          | package
| apache2-mod_scgi           | Apache module named mod_scgi that implements the client side of th-> | package
| apache2-mod_security2      | ModSecurity Open Source Web Application Firewall                     | package
| apache2-mod_tidy           | Apache Module using the Tidy Library to Clean Up HTML, XHTML, and -> | package
| apache2-mod_uwsgi          | uWSGI Module for Apache 2.0                                          | package
| apache2-mod_wsgi           | Python WSGI adapter module for Apache                                | package
| apache2-mod_wsgi-python3   | A WSGI interface for Python3 web applications in Apache              | package
| apache2-prefork            | Apache 2 "prefork" MPM (Multi-Processing Module)                     | package
| apache2-utils              | Apache 2 utilities                                                   | package
| apache2-worker             | Apache 2 worker MPM (Multi-Processing Module)                        | package
| perl-Apache2-AuthCookieDBI | An AuthCookie module backed by a DBI database.                       | package


As you can see the output is clean and includes a short summary of each matching item in the search.

Get information about a package

Use zypper info or {{}}zypper if{{}}.

luke@OpenSUSE-tst:~> zypper info apache2
Loading repository data...
Reading installed packages...
Information for package apache2:
--------------------------------
Repository     : Main Repository (OSS)  
Name           : apache2    
Version        : 2.4.23-4.3           
Arch           : x86_64
Vendor         : openSUSE
Installed Size : 4.0 MiB  
Installed      : No
Status         : not installed
Summary        : The Apache Web Server Version 2.4
Description    :
This version of httpd is a major release of the 2.4 stable branch, and represents the best available version of Apache HTTP Server. New features include Loadable MPMs, major improvements to OCSP support, mod_lua, Dynamic Reverse Proxy configuration, Improved Authentication/Authorization, FastCGI Proxy, New Expression Parser, and a Small Object Caching API. See /usr/share/doc/packages/apache2/, http://httpd.apache.org/, and http://httpd.apache.org/docs-2.4/upgrading.html.


The info option will show you which repository the package is in, its name, version, whether or not it is installed, and a brief description.

Install a package

To install a package type zypper install package_name or {{}}zypper in package_name{{}}

luke@OpenSUSE-tst:~> sudo zypper in apache2
Loading repository data...
Reading installed packages...
Resolving package dependencies...
The following 6 NEW packages are going to be installed:  apache2 apache2-prefork apache2-utils libapr1 libapr-util1 libnghttp2-14

6 new packages to install.
Overall download size: 1.8 MiB. Already cached: 0 B. After the operation, additional 5.4 MiB will be used.

Continue? [y/n/? shows all options] (y):


Running the install option solves for all dependencies quickly, and display's them in a way that allows an administrator to easily see everything that will be installed and how much disk space will be consumed. In this case we see 6 new packages.

Check for updates

Check for updates with zypper list-updates or {{}}zypper lu{{}}.

luke@OpenSUSE-tst:~> zypper lu
Loading repository data...
Reading installed packages...
S | Repository             | Name                        | Current Version | Available Version       | Arch  
--+------------------------+-----------------------------+-----------------+-------------------------+-------
v | Main Update Repository | ImageMagick                 | 6.8.8.1-20.1    | 6.8.8.1-22.2            | x86_64
v | Main Update Repository | alsa                        | 1.1.2-1.2       | 1.1.2-3.1               | x86_64
v | Main Update Repository | autoyast2-installation      | 3.1.151-1.2     | 3.1.154-3.1             | noarch
v | Main Update Repository | dolphin                     | 16.08.2-1.1     | 16.08.2-3.1             | x86_64
v | Main Update Repository | dolphin-part                | 16.08.2-1.1     | 16.08.2-3.1             | x86_64
v | Main Update Repository | ispell                      | 3.3.02-111.4    | 3.3.02-113.1            | x86_64
v | Main Update Repository | ispell-american             | 3.3.02-111.4    | 3.3.02-113.1            | x86_64
v | Main Update Repository | java-1_7_0-openjdk          | 1.7.0.111-35.1  | 1.7.0.121-37.2          | x86_64
v | Main Update Repository | java-1_7_0-openjdk-headless | 1.7.0.111-35.1  | 1.7.0.121-37.2          | x86_64
v | Main Update Repository | k3b                         | 2.0.3-6.21      | 2.0.3-9.1               | x86_64
v | Main Update Repository | k3b-lang                    | 2.0.3-6.21      | 2.0.3-9.1               | noarch
v | Main Update Repository | kernel-default              | 4.4.36-5.1      | 4.4.36-8.1              | x86_64
v | Main Update Repository | libMagickCore-6_Q16-1       | 6.8.8.1-20.1    | 6.8.8.1-22.2            | x86_64
v | Main Update Repository | libMagickWand-6_Q16-1       | 6.8.8.1-20.1    | 6.8.8.1-22.2            | x86_64
v | Main Update Repository | libasound2                  | 1.1.2-1.2       | 1.1.2-3.1               | x86_64
v | Main Update Repository | libblkid1                   | 2.28-5.10       | 2.28-7.1                | x86_64
v | Main Update Repository | libdolphinvcs5              | 16.08.2-1.1     | 16.08.2-3.1             | x86_64
v | Main Update Repository | libfdisk1                   | 2.28-5.10       | 2.28-7.1                | x86_64
v | Main Update Repository | libgc1                      | 7.2d-6.4        | 7.2d-8.1                | x86_64
v | Main Update Repository | libgit2-24                  | 0.24.1-1.16     | 0.24.1-3.1              | x86_64
v | Main Update Repository | libmount1                   | 2.28-5.10       | 2.28-7.1                | x86_64
v | Main Update Repository | libpcre1                    | 8.33-4.51       | 8.39-6.1                | x86_64
v | Main Update Repository | libpcre1-32bit              | 8.33-4.51       | 8.39-6.1                | x86_64
v | Main Update Repository | libpcre16-0                 | 8.33-4.51       | 8.39-6.1                | x86_64
v | Main Update Repository | libsensors4                 | 3.4.0-2.4       | 3.4.0-4.1               | x86_64
v | Main Update Repository | libsmartcols1               | 2.28-5.10       | 2.28-7.1                | x86_64
v | Main Update Repository | libsystemd0                 | 228-13.1        | 228-15.1                | x86_64
v | Main Update Repository | libsystemd0-32bit           | 228-13.1        | 228-15.1                | x86_64
v | Main Update Repository | libudev1                    | 228-13.1        | 228-15.1                | x86_64
v | Main Update Repository | libuuid1                    | 2.28-5.10       | 2.28-7.1                | x86_64
v | Main Update Repository | libwicked-0-6               | 0.6.38-2.1      | 0.6.39-4.1              | x86_64
v | Main Update Repository | systemd                     | 228-13.1        | 228-15.1                | x86_64
v | Main Update Repository | systemd-32bit               | 228-13.1        | 228-15.1                | x86_64
v | Main Update Repository | systemd-bash-completion     | 228-13.1        | 228-15.1                | noarch
v | Main Update Repository | systemd-logger              | 228-13.1        | 228-15.1                | x86_64
v | Main Update Repository | systemd-sysvinit            | 228-13.1        | 228-15.1                | x86_64
v | Main Update Repository | udev                        | 228-13.1        | 228-15.1                | x86_64
v | Main Update Repository | util-linux                  | 2.28-5.10       | 2.28-7.1                | x86_64
v | Main Update Repository | util-linux-lang             | 2.28-5.10       | 2.28-7.1                | noarch
v | Main Update Repository | util-linux-systemd          | 2.28-5.2        | 2.28-7.1                | x86_64
v | Main Update Repository | w3m                         | 0.5.3-158.2     | 0.5.3.git20161120-160.1 | x86_64
v | Main Update Repository | wicked                      | 0.6.38-2.1      | 0.6.39-4.1              | x86_64
v | Main Update Repository | wicked-service              | 0.6.38-2.1      | 0.6.39-4.1              | x86_64
v | Main Update Repository | xdm                         | 1.1.11-11.1     | 1.1.11-13.1             | x86_64
v | Main Update Repository | xdm-xsession                | 1.1.11-11.1     | 1.1.11-13.1             | x86_64
v | Main Update Repository | yast2-services-manager      | 3.1.42-16.1     | 3.1.43-18.1             | noarch


List updates presents a table that shows which repository an update will come from, which packages have an update, the current version, and the latest available version.

Update with zypper

Update the system with zypper update or {{}}zypper up{{}}.

luke@OpenSUSE-tst:~> sudo zypper up
Loading repository data...
Reading installed packages...
The following NEW package is going to be installed:
kernel-default-4.4.36-8.1
The following application is going to be REMOVED:
Dolphin
The following 45 packages are going to be upgraded:
alsa autoyast2-installation dolphin dolphin-part ImageMagick ispell ispell-american java-1_7_0-openjdk
java-1_7_0-openjdk-headless k3b k3b-lang libasound2 libblkid1 libdolphinvcs5 libfdisk1 libgc1 libgit2-24
libMagickCore-6_Q16-1 libMagickWand-6_Q16-1 libmount1 libpcre1 libpcre1-32bit libpcre16-0 libsensors4 libsmartcols1 libsystemd0 libsystemd0-32bit libudev1 libuuid1 libwicked-0-6 systemd systemd-32bit systemd-bash-completion systemd-logger systemd-sysvinit udev util-linux util-linux-lang util-linux-systemd w3m wicked wicked-service xdm xdm-xsession yast2-services-manager

45 packages to upgrade, 1 new.
Overall download size: 121.1 MiB. Already cached: 0 B. After the operation, additional 237.9 MiB will be used.

Continue? [y/n/? shows all options] (y):


Once again you see each package that is to be updated along with disk usage.

After the update has completed use zypper ps to show which processes, if any, need to be restarted. In this case, we updated the kernel so it's a good idea to do a full reboot.

List installed packages

To get a list of all installed packages try running

sudo zypper se -i


Find patterns

Suse uses the term patterns to describe groups of packages that are commonly installed together. For instance a mail server will have an email service, maybe an antivirus scanner, a spam blocker...

Try sudo zypper patterns to get a list of installable patterns.

More info

For further reading, you can always check the man pages man zypper or check out Suse documentation here: https://documentation.suse.com/sled/15-SP1/html/SLED-all/cha-sw-cl.html#sec-zypper