Pencarian

Rss Posts

 

 

 

Berita pada kategori ‘Sindikasi’

The Mysterious Phantom Reference

Sep 29, 2011

I talk about java.lang.ref.* in my performance tuning course because these things (along with anything that implements finalize) are more expensive to create than normal objects and require at least two rounds of GC before you’re completely rid of them. Of the bunch, that includes Reference, WeakReference, SoftReference and two other private reference classes that get mixed up with finalization, PhantomReference has to be the strangest.

 

Though I talk about PhantomReference, I’ve never used to nor could I even think of a situation where I’d even think to use it! Which is, I guess, why I’ve never used it. In fact, I can’t remember ever seeing them used any where in the wild. But, as is the case with just about everything,  if you talk to enough people you will eventually run into someone that has tried to used even the most arcane features in Java. True to this point, this week, I finally ran into someone that had a convincing use case for PhantomReference.

 

This person was trying to track down who was leaking JDBC connections. His idea was to wrap the connection in a PhantomReference and when the connection was discarded (and not closed), the garbage collector would put the object into a supplied ReferenceQueue, you’d grab the object from the ReferenceQueue and not only close it, you’d try to sort out who should have closed it. As with just about everything, the devil is in the details and in this case the details are; reference queue doesn’t return the object wrapped in the PhantomReference, it returns the PhantomReference and PhantomReference.get() always returns null. The reason for this is that the wrapped object is half collected and should not be reconnected to anything. I get that but what I don’t get it given this condition, why is PhantomReference marketed as alternative mechanisum to finalization. WIth no means to access the wrapped object, (reflection aside) there isn’t much one can do to clean things up. Consider the following code.

 

public class Foo {

 

    private String bar;

 

    public Foo(String bar) {

        this.bar = bar;

    }

 

    public String foo() {

        return bar;

    }

}

 

So lets say after the object has been completely dereferenced by the application I want to some how call foo(). Here is some code that I expected to work that would do this with one niggle.

 

        

 

// initialize

ReferenceQueue<Foo> queue = new ReferenceQueue<Foo>();

ArrayList< PhantomReference<Foo>> list=new ArrayList<PhantomReference<Foo>>();

 

for ( int i = 0; i < 10; i++) {

    Foo o = new Foo( Integer.toOctalString( i));

    list.add(new PhantomReference<Foo>(o, queue));

}

 

// make sure the garbage collector does it’s magic

System.gc();

 

// lets see what we’ve got

Reference<? extends Foo> referenceFromQueue;

for ( PhantomReference<Foo> reference : list)

    System.out.println(reference.isEnqueued());

 

while ( (referenceFromQueue = queue.poll()) != null) {

    System.out.println(referenceFromQueue.get());

    referenceFromQueue.clear();

}

 

PhantomReference takes an instance of Foo and a ReferenceQueue. Since no handles are kept to Foo, it should immediately be dead. Next, tell the VM to collect as there isn’t enough in heap for it to trigger a collection naturally. The first thing I’m going to ask the PhantomReference is; have you been enqueued. In this case the answer will be true. Next I ask the queue for the reference but as you can see, calling get() always returns null.

 

About the only solution that made sense is to wrap the resources or objects you wanted to interact with in a subclass of PhantomReference.

 

public class FinalizeStuff<Foo> extends PhantomReference<Foo> {

 

    public FinalizeStuff(Foo foo, ReferenceQueue<? super Foo> queue) {

        super(foo, queue);

    }

 

    public void bar() {

        System.out.println("foobar is finalizing resources");

    }

}

 

In this case I’m not going to wrap Foo in the subclass as that would seem to violate the spirit of PhantomReference. Instead I’m going to wrap resources associated with Foo and interact with them. Now I can do this.

 

// initialize

ReferenceQueue<Foo> queue = new ReferenceQueue<Foo>();

ArrayList< FinalizeStuff<Foo>> list = new ArrayList<FinalizeStuff<Foo>>();

ArrayList<Foo> foobar = new ArrayList<Foo>();

 

for ( int i = 0; i < 10; i++) {

    Foo o = new Foo( Integer.toOctalString( i));

    foobar.add(o);

    list.add(new FinalizeStuff<Foo>(o, queue));

}

 

// release all references to Foo and make sure the garbage collector does it’s magic

foobar = null;

System.gc();

 

// should be enqueued

Reference<? extends Foo> referenceFromQueue;

for ( PhantomReference<Foo> reference : list) {

    System.out.println(reference.isEnqueued());

}

 

// now we can call bar to do what ever it is we need done

while ( (referenceFromQueue = queue.poll()) != null) {

    ((FinalizeStuff)referenceFromQueue).bar();

    referenceFromQueue.clear();

}

 

This works though in some variations of this implementation main thread was racing against another thread (GC is my best guess). Note the strange need to cast. I could not sort out how to avoid it so if someone wants to comment….. Returning to the use case, the subclass that was created for the JDBC leak captured a stacktrace which was logged when the PhantomReference was pulled from the reference queue.

 

I’m happy for comments from anyone that has actually found a good use for PhantomReference as quite frankly, I still don’t understand why anyone would use it in leu of finalization. While finalization isn’t perfect, it’s not the dog that everyone makes it out to be and it’s far safer to use than PhantomReference is. For example, if anything bad happens during finalization, you’ll only shoot down a helper thread. Furthermore, the only way finalize will not be called is if the VM fails catastrophically or you’ve requested a shutdown but have failed to specify that finalizers (RunTime.runFinalizersOnExit()) should run before doing so. No such guarantees exist for PhantomReference.

 

PS, One point in favor of PhantomReference over finalize is that you could configure your system to optionally wrap objects. But then, you could use an interface with two implementations, one that implemented finalize() and one that doesn’t to achieve the same effect. So, I’m still scratching my head over this one.

MariaDB: the new MySQL? Interview with Michael Monty Widenius.

Sep 29, 2011

?I want to ensure that the MySQL code base (under the name of MariaDB) will survive as open source, in spite of what Oracle may do.? — Michael ?Monty? Widenius. Michael ?Monty? Widenius is the main author of the original version of the open-source MySQL database and a founding member of the MySQL AB company. [...]

InnoDB at Oracle OpenWorld

Sep 28, 2011

Sunny and I will be presenting at the Oracle OpenWorld next week:

Introduction to InnoDB, MySQL’s Default Storage Engine,? 10/04/11 Tuesday 01:15 PM, ? Marriott Marquis – Golden Gate C3, ? ? Calvin Sun
InnoDB Performance Tuning,? 10/04/11 Tuesday? 03:30 PM, ? Marriott Marquis – Golden Gate C2, ? Sunny Bains

The first session is for beginners, who are new to InnoDB and MySQL. The second session will cover many new performance features in MySQL 5.5 and 5.6, and share some tuning tips to maximize MySQL performance.
What to learn more about MySQL? There will be something for everyone. Come to join us!

 

JavaScript: Asynchronous Script Loading and Lazy Loading – Federico Cargnelutti

Jul 12, 2011

Most of the time remote scripts are included at the end of an html document, right before the closing body tag. This is because browsers are single threaded and when they encounter a script tag, they halt any other processes until they download and parse the script. By including scripts at the end, you allow the browser to download and render all page elements, style sheets and images without any unnecessary delay. Also, if the browser renders the page before executing any script, you know that all page elements are already available to retrieve.

However, websites like Facebook for example, use a more advanced technique. They include scripts dynamically via DOM methods. This technique, which I?ll briefly explain here, is known as ?Asynchronous Script Loading?.

Lets take a look at the script that Facebook uses to download its JS library:

(function () {
    var e = document.createElement('script');
    e.src = 'http://connect.facebook.net/en_US/all.js';
    e.async = true;
    document.getElementById('fb-root').appendChild(e);
}());

When you dynamically append a script to a page, the browser does not halt other processes, so it continues rendering page elements and downloading resources. The best place to put this code is right after the opening body tag. This allows Facebook initialization to happen in parallel with the initialization on the rest of the page.

Facebook also makes non-blocking loading of the script easy to use by providing the fbAsyncInit hook. If this global function is defined, it will be executed when the library is loaded.

window.fbAsyncInit = function () {
    FB.init({
        appId: 'YOUR APP ID',
        status: true,
        cookie: true,
        xfbml: true
    });
};

Once the library has loaded, Facebook checks the value of window.fbAsyncInit.hasRun and if it?s false it makes a call to the fbAsyncInit function:

if (window.fbAsyncInit && !window.fbAsyncInit.hasRun) {
    window.fbAsyncInit.hasRun = true;
    fbAsyncInit();
}

Now, what if you want to load multiple files asynchronously, or you need to include a small amount of code at page load and then download other scripts only when needed? Loading scripts on demand is called ?Lazy Loading?. There are many libraries that exist specifically for this purpose, however, you only need a few lines of JavaScript to do this.

Here is an example:

$L = function (c, d) {
    for (var b = c.length, e = b, f = function () {
            if (!(this.readyState
            		&& this.readyState !== "complete"
            		&& this.readyState !== "loaded")) {
                this.onload = this.onreadystatechange = null;
                --e || d()
            }
        }, g = document.getElementsByTagName("head")[0], i = function (h) {
            var a = document.createElement("script");
            a.async = true;
            a.src = h;
            a.onload = a.onreadystatechange = f;
            g.appendChild(a)
        }; b;) i(c[--b])
};

The best place to put this code is inside the head tag. You can then use the $L function to asynchronously load your scripts on demand. $L takes two arguments: an array (c) and a callback function (d).

var scripts = [];
scripts[0] = 'http://www.google-analytics.com/ga.js';
scripts[1] = 'http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.js';

$L(scripts, function () {
    console.log("ga and jquery scripts loaded");
});

$L(['http://connect.facebook.net/en_US/all.js'], function () {
    console.log("facebook script loaded");
    window.fbAsyncInit.hasRun = true;
    FB.init({
        appId: 'YOUR APP ID',
        status: true,
        cookie: true,
        xfbml: true
    });
});

You can see this script in action here (right click -> view page source).


Filed under: Design Patterns, Programming, Software Architecture the original (another 826 bytes)

PHP Returning Numeric Values in JSON – Lorna Mitchell

Jul 12, 2011


When I wrote about launching a prototype of a new
joind.in API, quite a few people started to try it out. My friend David Soria Parra emailed me to point out that many of the numbers in the API were being returned as strings. He said:

It’s just a standard problem of PHP REST services. When I try to access it with java I have to convert it over and over again to ints.

I did have a quick look at the PHP manual page for json_encode but I didn’t see anything mentioning this. A few weeks later (my inbox is a black hole and it takes a while to process these things) I fell over a throwaway comment to an undocumented constant JSON_NUMERIC_CHECK, and I added the constant name to my todo list. In the time it took for me to actually get around to googling for this, some wonderful person updated the PHP manual page (this is why I love PHP) to include it as a documented option, and someone else had added a user contributed note about using it.

It turns out, this constant does exactly what I need. Here’s a simple use case:

echo json_encode(array(‘event_id’ => ‘603′));
echo json_encode(array(‘event_id’ => ‘603′), JSON_NUMERIC_CHECK);
?

and the output:

{"event_id":"603"}
{"event_id":603}

There are probably some situations in which you don’t want all your looks-like-a-number data to be returned as a number, but for now it seems to be a good fit for api.joind.in.

PHP color scheme – Henrik Sarvell

Jul 09, 2011

Copyright ? 2011 http://www.prodevtips.com. Visit the original article at http://www.prodevtips.com/2011/07/09/zenburn-for-geany-php-color-scheme/.

Color theming in Geany is quite a no-brainer if you know how to go about it…. Read More

MySQL Connection Timeouts

Apr 20, 2011

Sometimes on very busy MySQL server you will see sporadic connection timeouts, such as Can’t connect to MySQL server on ‘mydb’ (110). If you have connects timed in your
application you will see some successful connections taking well over the second. The problem may start very slow and be almost invisible for long time, for example having one out of million
of connection attempts to time out… when as the load growths it may become a lot more frequent.

If you time the connect you will often see connection times are being close to 3 and 9 seconds. These are “magic” numbers which I remember from years ago, which correspond to SYN packet being dropped during connection attempt and being resent. 3 seconds corresponds to 1 packet being dropped and 9 seconds correspond to two. If this is happening it is possible you have network issues or more likely you have listen queue overflow. You can check if it is the case by running netstat -s and finding something like:

38409 times the listen queue of a socket overflowed
38409 SYNs to LISTEN sockets dropped

This means some SYN packets have to be dropped because kernel buffer of connection requests on LISTEN socket is overflow – MySQL is not accepting connections as quickly as it needs.
There are 2 tuning places you need to consider if this is what is happening.
First – Linux kernel net.ipv4.tcp_max_syn_backlog This is size of kernel buffer for all sockets.
Default I have on my kernel is 2048 though it might vary for different versions, you might need to increase it to 8192 or so if you have intense connection. I’ll explain the math below.
Second – is MySQL parameter back_log which has default value of just 50. You may want to set it to 1000 or even higher. You may also need to increase
net.core.somaxconn kernel setting which contains the maximum depth of listen queue allowed. The kernel I’m running has it set to just 128 which would be too low for many
conditions.
Now lets look more into the problem and do some Math. First lets look into how MySQL accepts connection. There is single main thread which is accepting connections coming to LISTEN
sockets. Once there is connection coming it it needs to create a new socket for incoming connection and create a new thread or take one out of the thread cache. From this point on MySQL processes network communication in multiple threads and can benefit from multiple cores but this work done by main thread does not.
Usually main thread is able to accept connections pretty quickly, however if it stalls waiting on mutex or doing any other work such as launching new thread takes a lot of time you can have the listen queue to overflow. Lets look at the database which accepts 1000 of connects/sec in average. This is a high number but you can see ones even higher. In most cases because of “random arrivals” nature of traffic you will see some seconds where as much as 3000 connections come in. Under such conditions the default back_log of 50 is enough just for 17 milliseconds, and if main thread stalls somewhere longer than, some SYN packets may be lost.
I would suggest sizing your tcp_max_syn_backlog and back_log value to be enough for at least 2 seconds worth of connection attempts. For example If I have 100 connects/sec which means I should plan for 300 connections using 3x for “peak multiplier”. This means they should be set to at least 600.
Setting it to cover much more than 2 seconds does not make much sense because if client does not get a response within 3 seconds it will consider SYN packet is lost and will send the new one anyway.
There is something else. If you’re creating 1000 of connections a second to MySQL Server you might be pushing your luck and at very least you’re using a lot of resources setting up and tearing down connections. Consider using persistent connections or connection pool at least for applications which are responsible for most of connections being created.

Setting up replication with XtraBackup

Apr 19, 2011

I attended Vadim Tkachenko’s talk on XtraBackup during MySQL conference in Santa Clara last week. Backups are obviously very important, but the use case I had in mind is this:
Replicating a database that has Innodb tables in it, while keeping both master and slave on line if possible.
Tangent: by the way, I love the native backup utility that was once promised in MySQL 6.0, similar to SQL Server’s way of backup. It was like running “BACKUP myDb to DISK = ‘/backupDirectory/myDb.bak’” under mysql client, but I digress…
I have used mysqldump to accomplish this in the past, but I wondered how XtraBackup would fare in this task, especially after hearing Vadim’s talk and reading news on Percona’s development effort. To cut to the chase, this is my conclusion. Reproducing steps are listed immediately afterwards.
1. innobackupex provides a consisten database backup, spitting out log file and log positions in stdout, which is nice and useful for slave initiation;
2. It works with both MyISAM and innodb tables;
3. If MyISAM tables are all you have, just run innobackupex –prepare /directoryWhereBackupIs, and then move the database directory from under /directoryWhereBackupIs to under your slave’s datadir, then make the necessary group and owner change to said directory and its content files, and you are ready to run the “change master” command and start slave;
4. If the database has innodb tables, then in addition to step 3, you will also need to stop mysql on slave, move the ibdata1 file to datadir, then restart mysql, and run “change master…” and “start slave” commands. It does not matter if you are using innodb_file_per_table or not.
It will be nice if I can keep the slave up and running during this step when the database has innodb tables in it. Did I do anything wrong? Is there a better way? What if the slave has a database that has innodb tables and thus uses ibdata1 to begin with? What do you do then? Should I play with Tungsten’s replication? What are the compelling reasons to use Tungsten’s replication?
In any case, from my limited testing, I think I will use innobackupex for future replication creation tasks, if I can afford a mysqld restart. Overall, it feels a bit easier than mysqldump approach that I’ve been using in the past.
Here are the steps needed to reproduce:
1. Fire up 2 Rackspace CentOS 5.5 servers. Rackspace cloud servers beat Amazon EC2 servers hands down, in my view, for developing/sandboxing purposes;
2. Install the required mysql client, server, and XtraBackup on both servers;
3. Make /etc/my.cnf by cloning the sample cnf files under /usr/share/my-small.cnf. 3 minimum changes were necessary: log-bin=mysql-bin, server-id=a unique number, datadir=/var/lib/mysql. The first 2 are necessary for replication, the last is needed for innobackupex
Well, while you are at it, on slave, add in read-only and skip-slave-start if appropriate. That’s best practice for read only slave.
4. Add master server’s public key to authorized_keys on slave, to facilitate easy ssh connection.
5. On master, run this command:

innobackupex –databases=test –stream=tar /tmp/ –slave-info | ssh root@slave "tar xfi – -C /root"
When it finishes, you should see something like this:
110419 18:54:21 innobackupex: completed OK!
tar: Read 6656 bytes from -

Take note of 3 lines immediately above it, where it states the binlog file and log position, like this:

innobackupex: MySQL binlog position: filename 'mysql-bin.000002', position 2515

6. On slave, run this command:

innobackupex –apply-log /locationWhereBackupIs

then, assuming the database name is test, run the 2 commands below to change the group and owner to mysql:

chgrp -R mysql test
chown -R mysql test

move the directory under mysqld’s datadir:

mv test/ /mysql/datadir

If test database has innodb tables in it, stop mysql on slave, then copy ibdata1 to datadir, restart mysql.
7. On master, open up port 3306 if it is not already open, then create the replication account:

grant replication slave, replication client on *.* to repl@'50.56.121.%' identified by 'p@ssw0rd';

8. On slave, run:

change master to master_host='50.56.121.96', master_user='repl', master_password='p@ssw0rd', master_log_file='see output from innobackupex backup command on master', master_log_pos=numFrominnobackupexOutputOnMaster;

start slave;

show slave status\G

EclipseLive: Upcoming Event: Reminder – C/C++ Development Tooling for Eclipse Indigo

Apr 19, 2011

Event Date: April 21, 2011 4:00 pm GMT-8

Register Now

Doug Schaefer (Wind River)
?
Abstract:

The CDT Project provides professional strength tools for C/C++ developers. Doug Schaefer, lead of the Eclipse CDT project, will present an overview of the CDT and demo the latest work for the Indigo release, coming at the end of June. Topics will include support for Android native development, refactoring, CODAN static analsis, Visual C++ integration, support for desktop and embedded development and what’s next for the project.

Total running time will be approximately 1 hour

9:00 am PST / 12:00 pm EST / 4:00 pm UTC / 6:00 pm CET – Convert to other time zones

Thanks to Adobe for contributing their Adobe Acrobat Connect product to host this webinar.


delicious delicious | digg digg | dzone dzone

MySQL & NoSQL Survey

Mar 13, 2011

Hello,

Could you please take the time and fill in this short survey about using MySQL and NoSQL in companies.
I will publish the results in a week.

Thank you for your time.

&lt;p&gt;Loading…&lt;/p&gt;