Author: Michael Blood
Ubuntu Server Time Resetting
Ubuntu Server Time Resetting
We have a server that was having trouble resetting the date on the server to todays date and time, in the year 2020. It appeared that the problem happened randomly and in some cases it would happen and then go away. Here are some of the steps I went through to debug this.
My server has a daily 1:01 AM cronjob to the the date from another local server (to keep all of our servers in sync)
This command syncs the date with that server.
/usr/sbin/ntpdate -v my.localsever.com
Anytime I noticed the date off at 2020, when i would run this command and it would properly reset to the correct time, so it seems it has to be coming from somewhere other than the my.localserver.com
So I decide to try to pinpoint when this happened. Do to this I started a cron log, which dumps the date, every 30 seconds into a file, so I could look at that file to find out when the dates change
/bin/date >> /tmp/bin.date.log
Now, next time it happens I will have a history of the minute during which the issue happens and perhaps I can tie it to some process I have running.
Test for and run a dynamically named javascript function
Test for and run a dynamically named javascript function
Sometime in the world of dynamic coding, you need the ability to dynamically create and run javascript functions depending on whether the javascript function is defined on the page. (so i can run different code depending on the context)
function email_action(action,obj)
{
var functionname='email_action_'+action
var t = setTimeout('alert("invalid function name: "'+functionname+'" n Cannot run email action: "'+action+')',100)
if (typeof(eval(functionname))!= 'function')
return;
clearTimeout(t)
var func = eval(functionname)
func(obj)
}
Check SPF Records when receiving mail in postfix
Check SPF Records when receiving mail in postfix
This simple install assumes you already have policyd installed
apt-get install postfix-policyd-spf-perl
Another tutorial, said that an executable file might be installed at /usr/sbin/policyd-spf, however it was installed elsewhere on my ubuntu 14.04 system, here is how I found it
#updatedb
#locate policyd-spf|bin
/usr/sbin/postfix-policyd-spf-perl
User the path to add this entry to your /etc/postfix/master.cfm
policy-spf unix - n n - - spawn
user=nobody argv=/usr/sbin/postfix-policyd-spf-perl
So, now we need to update /etc/postfix/main.cf by adding the following line
spf-policyd_time_limit = 3600s
and updateing the ‘smtpd_receiptient_restrictions’ to have the following in the list of services, I added mind after ‘permit_mynetworks’ and another ‘check_policy_service 127.0.0.1:10011’ entry I have.
smtpd_recipient_restrictions = permit_mynetworks,
check_policy_service inet:127.0.0.1:10011,
check_policy_service unix:private/policy-spf,
.....
Reload and watch your logs
/etc/init.d/postfix restart grep spf /var/log/mail.log
Code Source Text and White Space Quick Count Evaluation Tool
Code Source Text and White Space Quick Count Evaluation Tool
I created a simple website utility which can be used to evaluate the white space in some code or returned from a website.
Paste in the code from your editor, or you can type in a URL and it will download the html and evaluate that
The results counts up all of your characters, shows you how much white space there is.
It also shows you how many blank lines you have, whether you are using tabs or spaces at the beginning of your lines (this is useful to me to help coordinate similar code indenting formats between developers)
This tool can be pretty useful when wanting to figure out things related to white space usage.
https://www.matraex.com/character-coding-evaluator.php
At the bottom of the reusults, every line is displayed with details of each line and how many spaces are at the beginning of the line
if you have suggestions, email me with ‘White Space Evaluation’ in the subject michael @ matraex.com.
Company Potluck – 4/27/2015
Postgrey does not start – postfix rejecting mail – cannot connect to check_policy
Postgrey does not start – postfix rejecting mail – cannot connect to check_policy
On one of our mail servers as we were transitioning to a new server had an identical configuration of postfix, policyd and postgrey put on it, however it was rejecting mail with the following messages.
451 4.3.5 Server configuration problem;
A look at the messages just proceeding this in the mail.log, showed the real reason.
warning: connect to 127.0.0.1:60000: Connection refused warning: problem talking to server 127.0.0.1:60000: Connection refused
A quick look into our postfix configuration shows that we had been running our greylist policy service there.
#grep 60000 /etc/postfix/ -r
/etc/postfix/main.cf:greylist = check_policy_service inet:127.0.0.1:60000
I check that the greylist service was installed correctly, checked the open ports, which ports the service ws supposed to run on and updated the postfix configuration file to use the correct port
# dpkg -l|grep postgrey ii postgrey 1.34-1.2 all greylisting implementation for Postfix #lsof -i:60000 #no results confirms that nothing is listening on port 60000 # grep OPTS /etc/default/postgrey #a lookup of what options would run when postgrey is started shows what I should run to debug the daemon POSTGREY_OPTS="--inet=10023" #postgrey --inet=10023 2015/04/25-09:29:44 postgrey (type Net::Server::Multiplex) starting! pid(1569) Resolved [localhost]:10023 to [127.0.0.1]:10023, IPv4 Resolved [localhost]:10023 to [::1]:10023, IPv6 Binding to TCP port 10023 on host 127.0.0.1 with IPv4 Binding to TCP port 10023 on host ::1 with IPv6 ERROR: Can't connect to TCP port 10023 on ::1 [Cannot assign requested address] at /usr/sbin/postgrey line 776. #lsof -i:10023 #no results confirms that nothing is already listening on port 10023
So I have two changes I had to make,
- update the postfix and postgrey to be operating on the same port
- Find out what is stopping postgrey from listening on port 10023
To change postfix to operate on the same channel as postgrey was a simple choice, either tell postfix to check the policy service on 10023, or change postgrey to be on channel 60000. Since the postgrey was installed with apt-get and automatically choose 10023, I figured that is more standard and since my postfix configuration is older, i would move to 10023.
#sed -i -e's/60000/10023/' /etc/postfix/main.cf #/etc/init.d/postfix reload
Now I have to figure out what postgrey could not start on 10023, the first thing I will do is run it on a couple of different ports to find out if I get the same results
lsof -i:10022 #no results confirms that nothing is already listening on port 10023 #postgrey --inet=10022
A closer look at the error, describes the problem, postgrey is trying to start using IPv6.
Can't connect to TCP port 10023 on ::1 [Cannot assign requested address]
A quick check on the system shows that the system has IPv6 disabled.
#sysctl -a|grep 'ipv6.*disable' net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 net.ipv6.conf.eth0.disable_ipv6 = 1 net.ipv6.conf.lo.disable_ipv6 = 1
so I tested starting postgrey explicity using the IPv4 loopback local address
#postgrey --inet=127.0.0.1:10023 2015/04/25-09:49:09 postgrey (type Net::Server::Multiplex) starting! pid(6085) Binding to TCP port 10023 on host 127.0.0.1 with IPv4 Setting gid to "115 115" Setting uid to "106"
Success! I just had to update the postgrey OPTS and restart postgrey and my postfix problems are solved
# sed -i -e 's/10023/127.0.0.1:10023/' /etc/default/postgrey #/etc/init.d/postgrey start * Starting postfix greylisting daemon postgrey #lsof -i:10023 #results confirm that postgrey is listening COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME /usr/sbin 6557 postgrey 5u IPv4 8665045 0t0 TCP localhost:10023 (LISTEN)
And there we have it, it is now working, I confirmed postgrey was working by greppign my mail.log for postgrey and greylist and found results, I also confirmed that I no longer had any server rejections or errors connecting to post 60000 or 100023.
The Linux find command is awesomely powerful!
The Linux find command is awesomely powerful!
At least I think it is awesome. Here are a couple of useful commands which highlight some of it more powerful features. (these are just ones I used recently, as soon as you start chaining sed, awk, sort and uniq, the commands get even more powerful)
Changing the ownership of all files which do not have the correct ownership (useful to me when doing a server migration where the postfix user was uid 102 and changed to uid 105)
This command also lists the details of the file before it runs the chown command on it.
find . -not -uid 105 -exec chown postfix {} ;
Get a list of all of the files that have been modified in the last 20 minutes
find . -mmin -20
find all log files and their sizes older than 60 days, I use awk to sum the size of these up.
find /data/webs/ -path '*/logs/*' -name '*log' -mtime +60 -exec du {} ; |awk '{t+=$1; print t" "$0}'
Often times I just turn around and delete these files if I do not need them , the command above helps me know what kind of space I would be recovering and if there are any HUGE file size offenders.
find /data/webs/ -path '*/logs/*' -name '*log' -mtime +60 -delete
How to skip certain directories when running updatedb
How to skip certain directories when running updatedb
To skip certain directories when running updatedb edit
/etc/updatedb.conf
and add the directories you want to skip to the PRUNEFS configuration variable
PRUNEFS="/tmp /my/giant/directory"
That is it, then run updatedb again, it will skip the listed directory, in my case updatedb ran much faster.
AIDE huge daily reports
AIDE huge daily reports
AIDE is giving me HUGE daily reports on moderately used sites.
It was originally installed simply using ‘apt-get isntall aide’, and then I ran /usr/sbin/aideinit.
This generated 144 files in the /etc/aide/aide.conf.d folder and has been sending me an email every morning since with such a huge volume of changes, that it is useless.
The first item to cover is determining exactly what is in the reports. the way that AIDE works, it is not incremental by day, so if any file has changed since the day that you originally ran the aideinit command, it will be reported daily. So if you have not run the aideinit report since you made configuration changes or installed something, you need to do that, otherwise those configuration changes or installs will show up.
Use the -y switch to avoid having to wait and type ‘y’ in order to overwrite the old database
Then run the daily cron which should show you that no changes have occured.
/usr/sbin/aideinit -y /etc/cron.daily/aide
If your report is huge with log files or other standard, acceptable and planned for changes, you may want to exclude those from being monitored.
One school of thought might have been to remove EVERYTHING and then only put in place the items you want specifically to track, however I thought it would be better to explicitly remove the items which do not need to be monitored as they are sent to me. It seems more prudent, as we could potentially miss items we are not thinking about.
(Be sure to update your policy documents when making this change)
To address this on the command line, I went into the /etc/aide/aide.conf.d folder and started to remove the items I do not need.
Browser, Tablet and Mobile Device Detection in PHP
Browser, Tablet and Mobile Device Detection in PHP
Many websites present different webpages, versions, navigation and styles depending the type of device.
We often run into this issue and have found a couple of tools to address it.
- Browscap –http://browscap.org/ – A feature built into PHP which compares the USER AGENT string against a HUGE (50MB+) ini file with regular expression matching. The matching is comprehensive and very specific, however the library of User Agents requires frequent updates in this world of every changing devices. Each lookup can take several seconds, and a high volume PHP site will likely need to come up with some sort of caching to handle the load caused by this tool.
- Mobile Dectect – http://mobiledetect.net/ – A lightweight PHP Class which returns quickly. While it has worked in many reguards, the smaller size of the listing of User Agents leave me lacking in confidence that it will match all of the devices out there.
This sounds like an opportunity for me to build a comparison tool. I will call it: browser-detect-compare and it will have the following features
- When a user visits the page, their user agent is logged
- Each of the tools in the comparison is queried to return the results
- The site displays whether the results match between the tools for:
- Is Mobile
- Is Tablet
- Browser Name
- OS Name
- OS Version
- Browser Version
- If the results do not match, the user will be allowed to select which results are correct.
- The site will then display statistics about how correct each of the tools are for each of the tracked metrics.
Please leave comments of other features that will be use ful here.