Jonathan Levin's Axioms

Ideas, Rants, Theories vs Getting Things Done

12:24 PM

Email Validators - My Contribution

Posted by Jonathan |

So after my recent battle with "dirty" email addresses, I decided to find some validating functions to help people and myself in the future. Most of them use regular expressions.
So here they are:

MySQL stored function
Taken from - http://forge.mysql.com/snippets/view.php?id=62

DELIMITER $$
DROP FUNCTION IF EXISTS `test`.`is_valid_email` $$
CREATE DEFINER=`root`@`localhost` FUNCTION `is_valid_email`(p_email varchar(64)) RETURNS tinyint(1)
BEGIN
CASE
WHEN NOT (SELECT p_email REGEXP '^[A-Z0-9._%-]+@[A-Z0-9.-]+\.[A-Z]{2,4}$')
THEN
-- bad data
RETURN FALSE;
ELSE
-- good email
RETURN TRUE;
END CASE;
END $$
DELIMITER ;

In VBA (for excel, access and SQL server) – you need to enable “Microsoft VBScript Regular Expressions 5.5” in the preference

Dim myRegExp As VBScript_RegExp_55.RegExp

Function ValidEmail(ByVal email As String) As Boolean
Set myRegExp = New RegExp
myRegExp.IgnoreCase = True
myRegExp.Global = True
myRegExp.Pattern = "^[A-Z0-9._%-]+@[A-Z0-9.-]+\.[A-Z]{2,4}$"
ValidEmail = myRegExp.Test(email)
End Function

In JavaScript
Taken from - http://neural.cs.nthu.edu.tw/jang/sandbox/javascript/examples/chkemail.asp

function CheckEmail(str) {

var testresults=true
var filter=/^\w+([\.-]?\w+)*@\w+([\.-]?\w+)*\.(\w{2}|(com|net|org|edu|int|mil|gov|arpa|biz|aero|name|coop|info|pro|museum))$/
if (filter.test(str)){
var tempstring = str.split("@")
tempstring = tempstring[1].split(".")
} else {
testresults=false
}
return (testresults)
}

In Flash ActionScript

Taken from someone at my work. No regular expression.

private function checkEmail(value:String):Boolean{
var mail1= value.split("@")[0]
var mail2= value.split("@")[1]
var mail2_end = mail2.split(".")[mail2.split(".").length-1]
var lastCharASCII = value.charCodeAt(value.length-1)
if(value.split("@").length!=2) return false
if(mail1=="" || mail2== "" || mail2.split(".").length<2)>3) return false;
if((lastCharASCII<65>122)||(lastCharASCII>90 && lastCharASCII<97))>


Hope it helps.

Data Warehousing Continued

Part of the reason I was hired by my company was to help the company’s largest customer connect their data warehouse to the company’s website database. I have data basing experience but I have never been involved in such a large project (don’t tell my bosses). Luckily for me, I wont be doing everything, but for starters, I had to get some customer data, mainly email addresses and build a newsletter mailing list. Sounds simple enough..
Unfortunately for me, I had no idea how difficult it is to take even simple data fields from 2-3 sources and merge it into one. Some of the systems were so old, that back when they recorded the email address, there were no validation rules to enforce the input from the users. I saw emails like this http://www.163.com@yingying837.cn (163.com is a Chinese email server). It seemed to me that the people who inputted this email address only recently purchased a computer.

Anyway, after looking at the long list, running some filters and changing by hand some things, I figured I have 95% cleaned and the rest will be done with the email program to check if the email address still exists.

I was amazed how complicated it is to data cleanse old systems and afterwards make it comply with a data warehouse schema. Especially in the case of data that’s in more the one languages and even more so when it’s a language that uses characters like Chinese.


Conclusion

From now on I plan to put a strong emphasis on data quality from any data base I start working on.
I think it would be good to use data quality methods in my simplified business rules engine idea and this whole thing made me realize about possible directions my “new pet project” might go. I have already written a few things down about it on paper.

Thank you for reading my blog.

Yet Another Sun Bought MySQL Post

So Sun bought MySQL…. dunno about you, but I was surprised. My first impression was “nooooooooooooooo, I liked it when it was small and my contributions made a difference”, but I realized that it has obvious big advantages.

The main one, which has been mentioned over and over again, is the support of MySQL will grow to the level which I might see it in companies that opted not to get it and use commercial databases.

When I was interviewed for my current job, I learnt that the company made a website for a large customer and that they were using MySQL along side open source frameworks for it. The customer would want in the future to integrate the data, the website gathers, to their CRM system. Because of that, they were thinking about moving to using Oracle. They were also thinking of moving to Oracle anyway, because and I quote “we have more then 500 views a day on the website and we should upgrade our database”. Now, I didn’t say anything, but I was thinking “if Google and Facebook use MySQL, why would he think of using Oracle for just 500 views”.

When I joined the company, I asked someone about that and he answered “have you ever tried to setup a MySQL cluster??”
This guy was a developer, so I asked why is he doing it when he needs a DBA to do that.
He mentioned that it was very difficult to find MySQL DBA in our area (although I don’t know how hard they tried) and that with Oracle you can just phone in and get support. I answered that MySQL also has a support line and that they would be more then happy to offer their service. He mumbled a bit and was “yeah.. you know.. this is like that and its not so easy…”


Conclusion

In the end, it was clear to me that there is something in the heads of people about using important things like databases with open source projects. I am hoping that this will change in the case with MySQL now that Sun will help build its reputation for offering support.

5:52 PM

The Tow-Truck Approach

Posted by Jonathan |

I want to coin a new term - The Tow-Truck Approach (for project managers mainly).

What is it?
Think up of a practical, but different from the norm - solution and carry everyone kicking and screaming until you get there

  • Come up with or bring to your project innovative ideas to turn the way things have been done till now, on their heads.
  • Write achievable short term goals that prove that continuing this idea is a real option.
  • Emphasize on execution of the milestones and tasks.
  • Keep a close eye on time and money spent. Over spending and the project taking too long will kill the project and the idea.
  • Keep a close eye on skill and people – if they are able to get the job done or not
  • Spend less time on designing and more on developing – just start with something (prototype) instead of analysis paralysis
  • Communicate very clearly what you want. Preferably with meetings, short documents and diagrams. Clarity is crucial for everything in this process.
  • If people don’t agree or fight with you about the changes you are implementing, explain to them the end goals and their benefits. Listen to their feedback and react accordingly. However, if this still doesn’t work, replace them.
  • Constantly evaluate if the idea can be carried out or is it just a day dream. Sometimes the circumstances are not right for your idea or changes need to be made.
  • Constantly reinforce and praise people while you push forward to get things finished.

The emphasis is execution and communication. Get things done and move on to something else.


Think its good? Think its bad? then please leave your feedback in the comment box

4:14 PM

New Job, New Learning Opportunities

Posted by Jonathan |

It’s been a while since my last post. To be honest I have been extremely busy with so many personal things that I’m surprised I made it out alive.
During this time, I managed to find a new job in an international advertising company that has an office, here in Beijing. I still haven’t completely managed to get my visa situation completed here in China (a word of caution, with the Beijing Olympics coming up soon, there has been a serious crack down on foreigners living in Beijing and worsening of conditions) and I need to leave the country and come back just to renew my visa (almost all foreigners need to do that).

In my new job, they use Oracle databases, so I’m worried about how much time I can spend on MySQL related issues. I am, however, committed to do my “pet projects” on MySQL – which is why I have this blog.

I am eager to learn what Oracle has to offer. I haven’t worked for such a large company before and I would like to see what real solutions they have to offer (and maybe steal some ideas). I have always worked for small companies which forced me to learn about (and be responsible for) everything, which is what lead me to MySQL in the first place.


Also, I was asked to write an article for the MySQL developer zone. I was a bit surprised since I don’t have as much experience as the previous authors there, but I decided that I would try to write it anyway. Even though I am not (yet) as technical as the people who wrote there, I tried to instead, simplify and make clear some things that I already wrote about in my blog. I hope that the article gets accepted there.

I am still very interested in decision trees, business rules and decision theory. Basically, how us humans make decisions and then how do we explain it to computers. However, I am still sticking to the business side of making decisions and not the AI side which is to make computers make decisions based on data.

I would like to come up with something that just makes things simpler. I strongly believe that when you make thing simpler, you can challenge more complicated things. I am still planning on how to best use my simplified business rules engine idea and I hope that experience from my new job will help me solidify my idea.


Thank you for reading my blog.

Subscribe