I moved to a new position within IBM at the beginning of the year, joining the Enterprise Initiatives organization responsible for delivering IBM Smart Business Development and Test on the IBM Cloud.
Before I joined this team, Development and Test launched as a free public beta in 2009. Throughout 2010 we?ve incrementally improved [...]

Recently, I began working alongside Wez Furlong here at Message Systems. One of the many tools we use is Mtrack. This tool is a port of Trac into PHP, along with the addition of some great new features. Spearheaded by Wez, it?s a great tool that we use internally for our projects, and since it?s [...]

Artikel sebelumnya :

Buka YAST | Network Services | Samba Server
Pada wizard pertama, masukkan nama workgroup/domain. Untuk contoh saya, saya menggunakan nama [...]

Artikel sebelumnya :

Pilih menu YAST | Network Services | LDAP Server
Klik pada bagian Schema Files pada panel bagian kiri
Klik Add dan tambahkan Samba3.Schema sehingga kini LDAP Schema terdiri dari schema, core, cosine, inetorgperson, rfc2307bis,yast [...]

Artikel sebelumnya : Samba PDC+OpenLDAP+File Server Pengganti Windows AD+File Server Bagian 1-Pendahuluan
INSTALASI OPENSUSE 11.2 VERSI SERVER
Sebelum memulai instalasi LDAP server dan melakukan konfigurasi LDAP client, kita harus menyiapkan sistemnya terlebih dahulu, jadi langkah pertama adalah instalasi openSUSE 11.2 versi Server.
Panduan instalasi openSUSE 11.2 versi server bisa dibaca disini : Tutorial Instalasi openSUSE 11.2 Versi Server [...]

Kalau ada polling, layanan service apa dari Windows server yang paling banyak digunakan di Indonesia dan kerap membuat para Admin Windows ragu untuk migrasi ke Linux, Active Directory mungkin bisa menjadi salah satu jawaban terbanyak selain aplikasi database server.
Windows active directory adalah service server yang memberikan layanan domain logon, domain authentication, share authentication, permission, hak [...]

I was reading a post by Dathan Vance Pattishall titled “Cassandra is my NoSQL solution but..“. In the post, Dathan explains that he uses Cassandra to store clicks because it can write a lot faster than MySQL. However, he runs into problems with the read speed when he needs to get a range of data back from Cassandra. This is the number one problem I have with NoSQL solutions.

SQL is really good at retrieving a set of data based on a key or range of keys. Whereas NoSQL products are really good at writing things and retrieving one item from storage. When looking at redoing our architecture a few years ago to be more scalable, I had to consider these two issues. For what it is worth, the NoSQL market was not nearly as mature as it is now. So, my choices were much more limited. In the end, we decided to stick with MySQL. It turns out that a primary or unique key lookup on a MySQL/InnoDB table is really fast. It is sort of like having a key/value storage system. And, I can still do range based queries against it.

But, back to Dathan’s problem: clicks. We store clicks at dealnews. Lots of clicks. We also store views. We store more views than we do clicks. So, lots of views and lots of clicks. (Sorry for the vague numbers, company secrets and all. We are a top 1,000 Compete.com site during peak shopping season.) And we do it all in MySQL. And we do it all with one server. I should disclose we are deploying a
second server, but it is more for high availability than processing
power. Like Dathan, we only use about the last 24 hours of data at any given time. There are three keys for us doing logging like this in MySQL.

Use MyISAM

MyISAM supports concurrent inserts. Concurrent inserts means that inserts can add rows to the end of a table while selects are being performed on other parts of the data set. This is exactly the use case for our logging. There are caveats with range queries as pointed out by the MySQL Performance Blog.

MySQL (and InnoDB in particular) really sucks at deleting rows. Like, really sucks. Deleting causes locks. Bleh. So, we never delete rows from our logging tables. Instead, nightly we rotate the tables. RENAME TABLE is an (near) atomic process in MySQL. So, we just create a new table.

create table clicks_new like clicks;rename table clicks to clicks_2010032500001, clicks_new to clicks;

Tada! We now have an empty table for today’s clicks. We now drop any table with a date stamp that is longer than x days old. Drops are fast, we like drops.

For querying these tables, we use UNION. It works really well. We just issue a SHOW TABLES LIKE ‘clicks%’ and union the query across all the tables. Works like a charm.

Gearman

So, I get a lot of flack at work for my outright lust for Gearman. It is my new duct tape. When you have a scalability problem, there is a good chance you can solve it with Gearman. So, how does this help with logging to MySQL? Well, sometimes, MySQL can become backed up with inserts. It happens to the best of us. So, instead of letting that pile up in our web requests, we let it pile up in Gearman. Instead of having our web scripts write to MySQL directly, we have them fire Gearman background jobs with the logging data in them. The Gearman workers can then write to the MySQL server when it is available. Under normal operating procedure, that is in near real time. But, if the MySQL server does get backed up, the jobs just queue up in Gearman and are processed when the MySQL server is available.

BONUS! Insert Delayed

This is our old trick before we used Gearman. MySQL (MyISAM) has a neat feature where you can have inserts delayed until the table is available. The query is sent to the MySQL server and it answers with success immediately to the client. This means your web script can continue on and not get blocked waiting for the insert. But, MySQL will only queue up so many before it starts erroring out. So, it is not as fool proof as a job processing system like Gearman.

Summary

To log with MySQL:

• Use MyISAM with concurrent inserts
• Rotate tables daily and use UNION to query
• Use delayed inserts with MySQL or a job processing agent like Gearman

Happy logging!

PS: You may be asking, “Brian, what about Partitioned Tables?” I asked myself that before deploying this solution. More importantly, in IRC I asked Brian Aker about MySQL partitioned tables. I am paraphrasing, but he said that if I ever think I might alter that table, I would not trust it with the partitions in MySQL. So, that kind of turned me off of them.

2 hari kemarin saya melakukan simulasi integrasi antara Zimbra Mail Server 6.0.5 dengan Samba PDC menggunakan database user berbasis LDAP. Agar sysadmin dapat mengelola account-account single sign on secara mudah, Samba disetting agara menggunakan LDAP milik Zimbra. Berikut adalah beberapa catatan penting terkait integrasi tersebut, catatan selengkapnya akan saya publikasikan dalam bentuk tutorial lengkap :

Penulisan [...]

Recently, I finished my first production-quality Zend Framework application. It was a new website that made use of an old backend that powers another application I wrote using a custom framework. I wanted to use Zend Framework to practice on it, and to learn things I hadn?t yet learned since I had yet to put [...]

pbr/ I am still a Windows XP user thinking about trying out a Linux distribution./pbr/pbr/ I installed VLC Media Player recently on XP, and then applied my ultimate test for any media player: I popped in a DVD and checked if VLC could play it. Even on a copy of XP, it could not as the required codecs were missing./pbr/pbr/ If I switch to Linux I would like to be able to play DVDs legally, even if I have to pay to use the codecs./pbr/br/pbr/ I am still a Windows XP user thinking about tryinRead More…