Home › Monthly Archives › September 2011
23Sep

Setup a Local Test Mail Server with Ubuntu Linux

Yusuf World 0 comment

Well, I do a lot of testing and developing of/with web application. Mostly testing of Web Softwares and plugins requires mail functionality, for example self-service password forgot function and others. Using ISPs smtp server for these is rather difficult at times and some requires hacking the code. Setting up a local mail transfer agent (mta) has become a top most requirement. I use Ubuntu Linux and the default seems to be sendmail but I have been struggling to setup sendmail as a smarthost (to use gmail to relay emails). The configuration was harder than I thought. So I switched to postfix and was able to setup a local mail server within minutes. Here follows the setups for my future references and for anyone who need this.

1. Install postfix and certificates

1
sudo apt-get install postfix libsasl2-2 libsasl2-modules ca-certificates

In the install wizard that follows select Internet with smarthost and add [smtp.gmail.com]:587 as the relayhost

2. Additional postfix configurations
As sudo edit and add the following to the /etc/postfix/main.cf
Just below the line that says mydestination =
add the following (I prefer Maildir over Mbox)

1
2
3
inet_interfaces = all
mydestination = mail.yusufge.local, yusuf-ET1665, localhost.localdomain, localhost
home_mailbox = Maildir/

Just below the line that says relayhost = [smtp.gmail.com]:587
add the following

1
2
3
4
5
6
relayhost = [smtp.gmail.com]:587
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/gmail_passwd
smtp_sasl_security_options = noanonymous
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_use_tls = yes

3. Add SMTP authenticating user/password and adjust the file
As sudo create and add the following to /etc/postfix/gmail_passwd

1
[smtp.gmail.com]:587    youremailadd@gmail.com:password

Next adjust the permission of this newly created file

1
sudo chmod 700 /etc/postfix/gmail_passwd

Also translate this file to a db file that postfix can understand

1
sudo postmap /etc/postfix/gmail_passwd

4. Restart the postfix

1
sudo /etc/init.d/postfix restart

With the above 4 step your local mail server will be up and running so let us test it. In the terminal:

1
telnet localhost smtp

Trying 127.0.0.1…
Connected to localhost.
Escape character is ‘^]’.
220 yusuf-ET1665 ESMTP Postfix (Ubuntu)

1
ehlo localhost

250-yusuf-ET1665
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

1
mail from:<yusuf>

250 2.1.0 Ok

1
rcpt to:<yusuf>

250 2.1.0 Ok

1
data

354 End data with .

1
2
Hello there
.

250 2.0.0 Ok: queued as DCB0717E0141

1
quit

221 2.0.0 Bye
Connection closed by foreign host.

Instead of yusuf, use your box username or a valid email address above in the mail from: and rcpt to:, but if you use any other name than the gmail account that you authenticated with, it will end up as spam or rejected by some mail severs.

Also note, do not use this for high traffic servers, only recommended for home based setup for testing. In addition, the Google account that you use will store all outgoing messages (via Internet) in the Google Send Mail folder!

As for in my testing, the mail will end up in the following folder at: /home/yusuf/Maildir/new

1
2
cd /home/yusuf/Maildir/new/
ls

1316721453.V801I1301bc6M935924.yusuf-ET1665
To check the content of the file type cat the file (you can use tab key auto complete)

1
cat 1316721453.V801I1301bc6M935924.yusuf-ET1665

Return-Path:
X-Original-To: yusuf
Delivered-To: yusuf@mail.yusufge.local
Received: from localhost (localhost [127.0.0.1])
by yusuf-ET1665 (Postfix) with ESMTP id DCB0717E0141
for ; Fri, 23 Sep 2011 00:56:51 +0500 (MVT)
Message-Id: <20110922195701.DCB0717E0141@yusuf-ET1665>
Date: Fri, 23 Sep 2011 00:56:51 +0500 (MVT)
From: yusuf@mail.yusufge.local

Hello there

09Sep

Youtube Video Download Ruby Script

Yusuf Coding 4 comments

The nation is mourning on the loss of 4 students and the principal of Hiriyaa School in a drowning accident. After some time out with a friend I decided to spend the day indoor. When at home and with my computer can’t help, but get productive. So I decided to do a little experiment for a download manager that we are working on. Here is a ruby experimental script that can download YouTube Videos. Working as of September 9, 2011 but YouTube changes a lot so let me know when this breaks.

Note this a rough script, so feel free to improve.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
require 'open-uri'
youtubepath = 'http://www.youtube.com/watch?v=6tWLqFmaNdQ'
uri = URI.parse(youtubepath)
open(uri) do |file|
  openedsource = file.read
#  search for the title
  rgtitlesearch = Regexp.new(/\<meta name="title" content=.*/)
  vidtitle = rgtitlesearch.match(openedsource)
 vidtitle = vidtitle[0].gsub("<meta name=\"title\" content=\"","").gsub("\">","").gsub(/ /,'')+".flv"
# search for the download link
  rglinksearch = Regexp.new(/,url=.*\\u0026quality=/)
  vidlink = rglinksearch.match(openedsource)
 vidlink[0].split(",url=").each do |foundlinks|
   vidlink = foundlinks.gsub(",url=","").gsub("\\u0026quality=","").gsub("%3A",":").gsub("%2F","/").gsub("%3F","?").gsub("%3D","=").gsub("%252C",",").gsub("%253A",":").gsub("%26","&")
  end
def download vidlink, vidfile
 writeOut = open(vidfile, "wb")
 writeOut.write(open(vidlink).read)
 writeOut.close
end
 download(vidlink,vidtitle)

 puts "Download Done!"
end

Things missing and you might want to improve…

- Easier YouTube link inputs
- Assign Video Type (mp4, flv, etc) and Quality
- Download Progress Bar
- Clean up the code, including remove dups

08Sep

Wikileak Cables.csv to Mysql

Yusuf World 0 comment

Here is a ruby script for importing the wikileak cables.csv (1.7GB) file to a MySQL database. You can grab the file from http://cryptome.org/ (while in the site search for “z.7z”). As usual I did it in a hurry so feel free to modify and amend! A database with the name wikileaks is required, than again feel free to change. Enjoy!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
require 'rubygems'
require 'mysql'
database_name = "wikileak"
table_name = "cable"
csvfile = '/pathtocablefile/cables.csv'
m = Mysql.new("localhost", "username", "password")
m.select_db(database_name)
m.query("DROP TABLE IF EXISTS "+table_name)
ptable_create = "CREATE TABLE "+table_name+"(
id INT auto_increment primary key,
date CHAR(20),
code CHAR(50),
embassy CHAR(100),
class CHAR(50),
state CHAR(50),
sendto CHAR(200),
cabletext TEXT);"
m.query(ptable_create)
def save_to_db(cable_entry,table_name,m)
entry_field = 0
db_entry  = []
db_entry[7] = ''
#Remove unnecessary Commas
cable_entry = cable_entry.gsub('","',"$$")
cable_entry = cable_entry.gsub(",","")
cable_entry = cable_entry.gsub("$$",'","')
save_entry = cable_entry.split(",")
save_entry.each do |da_entry|
if entry_field == 7
db_entry[7] = db_entry[7]+da_entry
else
db_entry[entry_field] = da_entry
entry_field = entry_field + 1
end
valo = 0
end
pquery = "INSERT INTO `"+table_name+"` VALUES("+db_entry[0]+","+db_entry[1]+","+db_entry[2]+","+db_entry[3]+","+db_entry[4]+","+db_entry[5]+","+db_entry[6]+","+db_entry[7]+");"
m.query(pquery)
end
cable_entry = ''
seeknum = 1
seeker = '"'+seeknum.to_s()+'"'
File.open(csvfile).each do |record|
if record.match(seeker)
if seeknum &gt; 1
save_to_db(cable_entry,table_name,m)
puts "Saves: "+seeknum.to_s()
end
cable_entry = record
seeknum += 1
c_record_num = 1
seeker = '"'+seeknum.to_s()+'"'
else
cable_entry = cable_entry+record
end
end
m.close