Important command :: Rsync

Posted on December 26th, 2006 by Alex P..
Categories: Cpanel Hosting, Server Security, Linux.

What is Rsync?

Rsync is a very useful alternative to rcp written by Andrew Tridgell and Paul Mackerras. Rsync is a program, which is used for taking the backups on regular basis. However it can also be configured to upload large portals and any other data on the remote servers. . This is a great method for backups as it is low on bandwidth and it is also commonly used to keep clustered servers working together. It is very similar to rcp but with many more features. It is based on remote-update protocols, which means that it only sees the differences between the source and destination files and if the files has been changed, takes the backup otherwise leaves it without taking any backup. This difference may be the time stamp difference i.e. the date when it was modified or created is changed. By virtue of which it speeds up the file transfers even if it is of several gigabytes.

Rsync configuration files

There are three configuration files to run rsync server viz.
1. /etc/rsyncd.conf,
2. /etc/rsyncd.pid
3. /etc/rsyncd.secrets
1) /etc/rsyncd.conf

When you run Rsync with –daemon mode /etc/rsyncd.conf is the runtime configuration file through inetd daemon. This way rsync becomes an rsync server listening on TCP port 873. Connections from Rsync clients are accepted for either anonymous or authenticated Rsync sessions. The rsyncd.conf file controls authentication, access, logging and available modules.

2. The file /etc/rsyncd.secrets can be like this containing the username and password with a colon in between on the client side i.e.

username : yourpassword

3. To create an empty file /etc/rsyncd.pid to place the process related arguments.

The general form is:

rsync source destination

A few real life examples are:

rsync -vrplogDtH /old/var/named/ /var/named/

That will sync the /old/var/named/ directorty to the /var/named/ directory on the same server.

If you want to use rsync to go between servers then use the following:

rsync -ave ssh root@192.168.0.1:/backup/ /backup/

That will take the backup directory on 192.168.0.1 and copy it to the server the command is run from. The command will also accept a remote destination if you adjust the command line accordingly. Below is the readout of rsync –help which also shows more of the command options available:

Usage: rsync [OPTION]… SRC [SRC]… [USER@]HOST:DEST

or rsync [OPTION]… [USER@]HOST:SRC DEST

or rsync [OPTION]… SRC [SRC]… DEST

or rsync [OPTION]… [USER@]HOST::SRC [DEST]

or rsync [OPTION]… SRC [SRC]… [USER@]HOST::DEST

or rsync [OPTION]… rsync://[USER@]HOST[:PORT]/SRC [DEST]

or rsync [OPTION]… SRC [SRC]… rsync://[USER@]HOST[:PORT]/DEST

SRC on single-colon remote HOST will be expanded by remote shell

SRC on server remote HOST may contain shell wildcards or multiple

sources separated by space as long as they have same top-level

Useful command-line switches

v, –verbose increase verbosity
-q, –quiet decrease verbosity
-c, –checksum always checksum
-a, –archive archive mode, equivalent to -rlptgoD
-r, –recursive recurse into directories
-R, –relative use relative path names
-b, –backup make backups (default ~ suffix)
–backup-dir make backups into this directory
–suffix=SUFFIX override backup suffix
-u, –update update only (don’t overwrite newer files)
-l, –links copy symlinks as symlinks
-L, –copy-links copy the referent of symlinks
–copy-unsafe-links copy links outside the source tree
–safe-links ignore links outside the destination tree
-H, –hard-links preserve hard links
-p, –perms preserve permissions
-o, –owner preserve owner (root only)
-g, –group preserve group
-D, –devices preserve devices (root only)
-t, –times preserve times
-S, –sparse handle sparse files efficiently
-n, –dry-run show what would have been transferred
-W, –whole-file copy whole files, no incremental checks
–no-whole-file turn off –whole-file
-x, –one-file-system don’t cross filesystem boundaries
-B, –block-size=SIZE checksum blocking size (default 700)
-e, –rsh=COMMAND specify the remote shell

0 comments.

Learn about SOA records

Posted on December 26th, 2006 by Alex P..
Categories: Cpanel Hosting, Linux, web Hosting, VPS / OpenVZ.

To an Administrator, there is nothing more peaceful than a stable and optimized DNS server. The moment there is a wrong configuration, the server wakes up and starts crying, sites and email goes down. An important part of keeping DNS that way is properly setting up the SOA records.

What are DNS Records. DNS records or Zone files are used for mapping URLs to an IPs. Located on servers called the DNS servers, these records are typically the connection of your website with the outside world. Requests for your website are forwarded to your DNS servers and then get pointed to the WebServers that serve the website or to Email servers that handle the incoming email.

This is how a typical Zone file (containing many common DNS records) looks like.

; Zone file for sgugal.com.

@       86400 IN SOA ns1.sgugal.com.   root.manou.sgugal.com. (

2006061904

86000

7200

3600000

86400 )

sgugal.com. NS IN 86400 ns1.sgugal.com.

sgugal.com. NS IN 86400 ns2.sgugal.com.

sgugal.com. 14400 IN A 69.20.54.201

localhost. sgugal.com. 14400 IN A 127.0.0.1

sgugal.com. 14400 IN MX 0 sgugal.com.

mail 14400 IN CNAME sgugal.com.

www 14400 IN CNAME sgugal.com.

ftp 14400 IN CNAME sgugal.com.

SOA Records

An SOA(State of Authority) Record is the most essential part of a Zone file. The SOA record is a way for the Domain Administrator to give out simple information about the domain like, how often it is updated, when it was last updated, when to check back for more info, what is the admins email address and so on. A Zone file can contain only one SOA Record.

A properly optimized and updated SOA record can reduce bandwidth between nameservers, increase the speed of website access and ensure the site is alive even when the primary DNS server is down.

Here is the SOA record. Notice the starting bracket “(“. This has to be on the same line, otherwise the record gets broken.

; name TTL class rr Nameserver email-address

@ 86400 IN SOA ns1.sgugal.com. root.linux5.nettoolz.net. (

2006061904 ; Serial number

86000 ; Refresh rate in seconds

7200 ; Update Retry in seconds

3600000 ; Expiry in seconds

86400 ; minimum in seconds )

• name - mydomain.com is the main name in this zone.

• TTL - 86400 - TTL defines the duration in seconds that the record may be cached by client side programs. If it is set as 0, it indicates that the record should not be cached. The range is defined to be between 0 to 2147483647 (close to 68 years !)

• Class - IN - The class shows the type of record. IN equates to Internet. Other options are all historic. So as long as your DNS is on the Internet or Intranet, you must use IN.

• Nameserver - ns.nameserver.com. -The nameserver is the server which holds the zone files. It can be either an external server in which case, the entire domain name must be specified followed by a dot. In case it is defined in this zone file, then it can be written as “ns’’ .

• Email address – root.linux5.nettoolz.net. -This is the email of the domain name administrator. Now, this is really confusing, because people expect an @ to be in an email address. However in this case, email is sent to [EMAIL=”root@ns.nameserver.com”] root@ns.nameserver.com[/EMAIL], but written as root.ns.nameserver.com . And yes, remember to put the dot behind the domain name.

• Serial number - 2006061904 - This is a sort of a revision numbering system to show the changes made to the DNS Zone. This number has to increment, whenever any change is made to the Zone file. The standard convention is to use the date of update YYYYMMDDnn, where nn is a revision number in case more than one updates are done in a day. So if the first update done today would be 2006061904 and second update would be 2006061905.

• Refresh - 86000 - This is time(in seconds) when the slave DNS server will refresh from the master. This value represents how often a secondary will poll the primary server to see if the serial number for the zone has increased (so it knows to request a new copy of the data for the zone). It can be written as “23h88M’’ indicating 23 hours and 88 minutes. If you have a regular Internet server, you can keep it between 6 to 24 hours.

• Retry - 7200 - Now assume that a slave tried to contact the master server and failed to contact it because it was down. The Retry value (time in seconds) will tell it when to get back. This value is not very important and can be a fraction of the refresh value.

• Expiry - 3600000 - This is the time (in seconds) that a slave server will keep a cached zone file as valid, if it can’t contact the primary server. If this value were set to say 2 weeks ( in seconds), what it means is that a slave would still be able to give out domain information from its cached zone file for 2 weeks, without anyone knowing the difference. The recommended value is between 2 to 4 weeks.

• Minimum - 86400 - This is the default time(in seconds) that the slave servers should cache the Zone file. This is the most important time field in the SOA Record. If your DNS information keeps changing, keep it down to a day or less. Otherwise if your DNS record doesn’t change regularly, step it up between 1 to 5 days. The benefit of keeping this value high, is that your website speeds increase drastically as a result of reduced lookups. Caching servers around the globe would cache your records and this improves site performance.

Increasing site speed

The time it takes to access a website on a browser includes the time it takes to look it up on the domain name server. By increasing the “Minimum’’ value, we’re telling the contacting clients to keep their copies of the zone file for a longer time. In effect, reducing the lookups to the nameserver. By reducing the number of times a client has to lookup, we’re increasing the site speed.

However, this also means that if you make changes to the DNS record, it will take longer to propagate. If you require to make frequent updates to your DNS records, make sure your Minimum value is lesser than 1 day. That means longer lookup times, but accurate information for the clients

If you are planning a major update on the DNS zone file(say moving to another server or hosting service), reduce the Minimum value a couple of days prior to the change. Then make the change and then jack up the minimum value again. This way the caching clients all over the world will pick up the changes quicker and yet you do not need to sacrifice on site speed thereafter.

How to improve backup

Always keep a secondary DNS server and keep a higher Expiry value. This will mean that even if the Primary server goes down, the secondary will have the cached copy(for as long as the Expiry value stands) and it will keep serving lookups. Keeping a secondary server but a low expiry value defeats the purpose of a Backup.

How to test SOA records

You have set the new SOA values, and you want to know whether the update has taken place. “Dig’’ is a good tool to troubleshoot and check for DNS information.

For example to check out the SOA records of yahoo.com from all the nameservers, primary and secondary, all you need to do is

# dig google.com +nssearch

SOA ns1.sgugal.com. root.manou.sgugal.com. 2006072101 28800 7200 3600 86400 from server ns2.sgugal.com in 1 ms.

SOA ns1.sgugal.com. root.manou.sgugal.com. 2006072101 28800 7200 3600 86400 from server ns1.sgugal.com in 28 ms.

0 comments.

Exim Command Basics

Posted on December 25th, 2006 by Alex P..
Categories: Cpanel Hosting, Linux, web Hosting, EXIM, VPS / OpenVZ.

Message-IDs and spool files

The format of the message ID is three groups of base 62 digits, separated by hyphens. and take the form of: XXXXXX-YYYYYY-ZZ. The first group, of six digits, gives the integral number of seconds since the epoch. The second group, also of six digits, gives the process ID. The third group, of two digits, gives the fractional part of the number of seconds since the epoch, in units of 1/2000 of a second (500 us). The function does not return until the clock has advanced far enough that another call would generate a different ID.

Logs Files

Files in /var/spool/exim/msglog contain logging information for each message and are named the same as the message-id.

Files in /var/spool/exim/input are named after the message-id, plus a suffix denoting whether it is the envelope header (-H) or message data (-D).

Basic information

Print a count of the messages in the queue:

root@ashok# exim -bpc

Print a listing of the messages in the queue (time queued, size, message-id, sender, recipient):

root@ashok# exim -bp

Print a summary of messages in the queue (count, volume, oldest, newest, domain, and totals):

root@ashok# exim -bp | exiqsumm

Print what Exim is doing right now:

root@ashok# exiwhat

Run a pretend SMTP transaction from the command line, as if it were coming from the given IP address. This will display Exim’s checks, ACLs, and filters as they are applied. The message will NOT actually be delivered.

root@ashok# exim -bh 192.168.11.22

Display all of Exim’s configuration settings:

root@ashok# exim -bP

Searching the queue with exiqgrep

Exim includes a utility that is quite nice for grepping through the queue, called exiqgrep.

First, various flags that control what messages are matched. These can be combined to come up with a very particular search.

Use -f to search the queue for messages from a specific sender:

root@ashok# exiqgrep -f [luser]@domain

Use -r to search the queue for messages for a specific recipient/domain:

root@ashok# exiqgrep -r [luser]@domain

Use -o to print messages older than the specified number of seconds. For example, messages older than 1 day:

root@ashok# exiqgrep -o 86400 […]

Use -y to print messages that are younger than the specified number of seconds. For example, messages less than an hour old:

root@ashok# exiqgrep -y 3600 […]

Use -s to match the size of a message with a regex. For example, 700-799 bytes:

root@ashok# exiqgrep -s ‘^7..$’ […]

Use -z to match only frozen messages, or -x to match only unfrozen messages.

There are also a few flags that control the display of the output.

Use -i to print just the message-id as a result of one of the above two searches:

root@ashok# exiqgrep -i [ -r | -f ] …

Use -c to print a count of messages matching one of the above searches:

root@ashok# exiqgrep -c …

Print just the message-id of the entire queue:

root@ashok# exiqgrep -i

Managing the queue

The main exim binary (/usr/sbin/exim) is used with various flags to make things happen to messages in the queue. Most of these require one or more message-IDs to be specified in the command line, which is where `exiqgrep -i` as described above really comes in handy.

Start a queue run:

root@ashok# exim -q -v

Start a queue run for just local deliveries:

root@ashok# exim -ql -v

Remove a message from the queue:

root@ashok# exim -Mrm message-id [ message-id … ]

Freeze a message:

root@ashok# exim -Mf message-id [ message-id … ]

Thaw a message:

root@ashok# exim -Mt message-id [ message-id … ]

Deliver a message:

root@ashok# exim -M message-id [ message-id … ]

Force a message to fail and bounce as “cancelled by administrator”:

root@ashok# exim -Mg message-id [ message-id … ]

Remove all frozen messages:

root@ashok# exiqgrep -z -i | xargs exim -Mrm

Remove all messages older than five days (86400 * 5 = 432000 seconds):

root@ashok# exiqgrep -o 432000 -i | xargs exim -Mrm

Freeze all queued mail from a given sender:

root@ashok# exiqgrep -i -f luser@example.tld | xargs exim -Mf

View a message’s headers:

root@ashok# exim -Mvh message-id

View a message’s body:

root@ashok# exim -Mvb message-id

View a message’s logs:

root@ashok# exim -Mvl message-id

Add a recipient to a message:

root@localhost# exim -Mar message-id

 [ address … ]

Edit the sender of a message:

root@localhost# exim -Mes message-id  address

Searching the logs with exigrep

The exigrep utility (not to be confused with exiqgrep) is used to search an exim log for a string or pattern. It will print all log entries with the same internal message-id as those that matched the pattern, which is very handy since any message will take up at least three lines in the log. exigrep will search the entire content of a log entry, not just particular fields.

One can search for messages sent from a particular IP address:

root@ashok# exigrep ‘<= .* [12.34.56.78] ' /path/to/exim_log

Search for messages sent to a particular IP address:

root@ashok# exigrep ‘=> .* [12.34.56.78]’ /path/to/exim_log

This example searches for outgoing messages, which have the “=>” symbol, sent to “user@domain.tld”. The pipe to grep for the “<=" symbol will match only the lines with information on the sender - the From address, the sender's IP address, the message size, the message ID, and the subject line if you have enabled logging the subject. The purpose of doing such a search is that the desired information is not on the same log line as the string being searched for.

root@ashok# exigrep ‘=> .*user@domain.tld’ /path/to/exim_log | fgrep ‘<='

Generate and display Exim stats from a logfile:

root@ashok# eximstats /path/to/exim_mainlog

Same as above, with less verbose output:

root@ashok# eximstats -ne -nr -nt /path/to/exim_mainlog

Same as above, for one particular day:

root@ashok# fgrep YYYY-MM-DD /path/to/exim_mainlog | eximstats




0 comments.

Understanding and Setting CRON

Posted on November 26th, 2006 by Admin.
Categories: Cpanel Hosting, Linux.

Cron allows users to automate repetitive system administration tasks such as Periodic execution of a script, database reorganization, and general file cleanups (such as emptying log files and queues).

While Setting up Crons , have to use below parameters .
* * * * * represents < minute hour date month day>

Minute:- At what minute the commands will run between 0 and 59.
Hour:- At what hour the commands will run between 0 and 23.
Date:- At what date of month the commands will run between 1 and 31.
Month:- At what month the command will run between 1 and 12.
Day:- At which day of week commands will run between 0 and 7 where 0,7 = Sunday.

If you don’t want to specify a value for a field then just place * in that field.

0 comments.

How to Block IPs with APF Firewall

Posted on November 26th, 2006 by Admin.
Categories: Server Security, Linux.

Blocking IPs with APF Firewall1) /etc/apf/apf -d IP Reason
–> The -d flag means DENY the IP address
–> IP is the IP address you wish to block
–> Reason is comments to why the IP is being blocked

Example:

./apf -d 267.14.188.1 TESTING

vi /etc/apf/deny_hosts.rules

Shows the following:

# added 267.14.188.1 on 08/23/05 01:25:55
# TESTING
267.14.188.1

2) pico /etc/apf/deny_hosts.rules

You can then just add a new line and enter the IP you wish to block. Before this becomes active though you’ll need to reload the APF ruleset.

/etc/apf/apf -r

0 comments.

How to Unblock IPs from APF Firewall

Posted on November 26th, 2006 by Admin.
Categories: Server Security, Linux.

Unblocking IPs with APF Firewall through below steps :1) pico /etc/apf/deny_hosts.rules

Find where the IP is listed and remove the line that has the IP.
After this is done save the file and reload apf to make the new changes active.

/etc/apf/apf -r

2) If the IP isn’t already listed in deny_hosts.rules and you wish to allow it, then,

/etc/apf/apf -a IP Reason
–> The -a flag means ALLOW the IP address
–> IP is the IP address you wish to allow
–> Reason is comments to why the IP is being unblocked
Example:

./apf -a 267.14.188.1 Reason

pico /etc/apf/allow_hosts.rules

# added 267.14.188.1 on 09/15/06 03:12:50
# UNBLOCKING
267.14.188.1

0 comments.

Installing and Configuring APF Firewall on Linux Server

Posted on November 26th, 2006 by Admin.
Categories: Server Security, Linux.

cd /usr/local/src
wget http://rfxnetworks.com/downloads/apf-current.tar.gz
tar -zxf apf-current.tar.gz
cd apf-0.*
./install.sh

Now edit config file
vi /etc/apf/conf.apf

Scroll down to the “Common ingress (inbound) TCP ports section. At this point you need to find the correct configuration for your control panel.

IG_TCP_CPORTS=”20,21,22,25,26,53,80,110,143,443,465,993,995,2082,2083,2086,2087,2095,2096″
IG_UDP_CPORTS=”21,53,873″

EGF=”1″
EG_TCP_CPORTS=”21,22,25,26,27,37,43,53,80,110,113,443,465,873,2089″
EG_UDP_CPORTS=”20,21,37,53,873″

save the file
start apf
apf -s

If everything still works then edit the config file and turn dev mode off. Make sure you can start a new ssh session before changing dev mode off. If you are kicked out you need to go back and look at what caused the problem!
DEVEL_MODE=”0″

restart APF
apf -r

0 comments.

Cpanel License issue

Posted on November 25th, 2006 by Admin.
Categories: Cpanel Hosting.

If you are getting Cpanel license expired page then check out below 2 things ..
1. Run /usr/local/cpanel/cpkeyclt on server
2. Verify at http://layer1.cpanel.net/verify

0 comments.

How to Compile Linux Kernel

Posted on November 24th, 2006 by Admin.
Categories: Server Security, Linux.

Below are steps to compile Linux Kernel …

cd /usr/local/src/
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.11.12.tar.bz2
wget http://grsecurity.net/grsecurity-2.1.6-2.6.11.12-200506141713.patch.gz
tar xvfj linux-2.6.11.12.tar.bz2
gunzip grsecurity-2.1.6-2.6.11.12-200506141713.patch.gz
patch -p0 < grsecurity-2.1.6-2.6.11.12-200506141713.patch
cd linux-2.6.11.12
make clean
make mrproper
cp /boot/config-’uname -r’ .config
make oldconfig
make menuconfig
make dep
make modules_install
make install
grubby –bootloader-probe
pico /etc/grub.conf
grub-install /dev/hda
cat /boot/grub/device.map
shutdown -r now

0 comments.

Major 352 Linux Distributions

Posted on November 23rd, 2006 by Alex P..
Categories: Linux.

There are more than 352 Linux Distributions till now which are all Open Source and Free .It also include some BSD and UNIX systems but they are also all open source and free. More information can be found at http://www.gridter.com/linx/linux.html .

0 comments.

Older » « Newer