Guyub – Konsultan F/OSS

I was getting inconsistent performance results while running sysbench to generate a workload of point-updates and point-lookups. The rate of rows updated per second would vary between 200 and 600 and the variance appeared to be random. From PMP there was a lot of contention on the transaction log system mutex. It took me about one day to guess at the root cause — the InnoDB transaction log was not in the OS buffer cache and 512-byte aligned log writes frequently required disk reads to get a 4kb aligned page into the cache to apply the write. The problem is avoided when either the transaction log remains in the OS buffer cache or you use the Percona patch that adds all_o_direct as an option for innodb_flush_method.?The problem was harder to debug than it should be because InnoDB didn’t report log write latency via a separate metric. All synchronous writes, log and doublewrite buffer, were reported via the “Sync writes” line and that combines large/slow writes to the doublewrite buffer with small/fast writes to the transaction log:Sync writes: 4836209719 requests, 0 old, 5009.82 bytes/r, svc: 498087.30 secs, 0.10 msecs/r?I have a diff out to fix that:Log writes: 2328355 requests, 0 old, 955.39 bytes/r, svc: 14.44 secs, 0.01 msecs/rDoublewrite buffer writes: 19718 requests, 12 old, 946736.45 bytes/r, svc: 41.29 secs, 2.09 msecs/r?I suspect this is even harder to debug in official MySQL which doesn’t have any of the metrics above in SHOW INNODB STATUS output or SHOW STATUS counters. Perhaps the performance schema makes this easier to debug but I don’t know much about that feature.?I then reproduced the problem by starting the benchmark with the transaction log in the OS buffer cache and then running?echo 1 > /proc/sys/vm/drop_caches to remove it from cache. The results below show the impact on both the rate of rows read and updated per second. The test used 128 client threads doing point-lookups and 32 client threads doing point-updates. The database was 240G on disk and the InnoDB buffer cache was 30G. The rates drop significantly when the cache is dropped.??The results are great prior to removing the transaction log from cache. On a server with 8 10k RPM SAS disks I was able to get 2800 point-lookups and 500 point-updates per second. Using 16kb InnoDB pages the server sustained 2500 page reads/second from disk and 500 page writes/second to disk.??The final graph uses logscale for the y-axis to plot the rate of rows updated/second and the average latency for a log write in microseconds. The update rate drops when the log write latency spikes to more than 10ms per write. It was less than 10us prior to that.?

Sunny and I will be presenting at the Oracle OpenWorld next week:

I’ve already dealt with this argument so far… but it’s really so crazy that I can’t prevent myself from blogging again on it, also taking advantage of this article by ACM titled “Should code be Released”. The subcaption says it all “Software code can provide important insights into the results of research, but it’s up to individual scientists whether their code is released – any many opt not to.“.

So, some scientists still refuse to publish the code that helped them in achieving a certain theory. While I’m certainly not so naive to assert that they should publish to SourceForge since their first commit, once one has published his research to a couple of relevant places, and has bound his name to that research, arguing that releasing the code could help others to “steal” is really hilarious. On the contrary, we all know how big a difference in quality the open source approach can deliver. Science is based on peer review: how the hell can be that a theory is peer reviewed if you can’t reproduce the steps to get to the underlying model? While in our community we are only poor technologists and not scientists, everybody would scream in disgust if I only dared to assert “I have demonstrated that Java is 5x faster than C”, but I don’t release the benchmark code so everybody can try it.

 I can only conclude that many scientists are not confident at all with their theories, or they are purportedly cheating.

According to the latest java.net poll, the Java community is reacting positively to the news that IBM will be collaborating with Oracle on the OpenJDK project. A total of 267 votes were cast in the poll, which ran for the past week. Here is the actual poll question and the results:

What’s your view of the news that Oracle and IBM will collaborate in the OpenJDK project?

• 18% (48 votes) – It’s the best Java news I’ve heard in a long time
• 30% (81 votes) – It’s definitely positive
• 34% (90 votes) – We’ll see how it works out
• 2% (6 votes) – Makes no difference
• 10% (27 votes) – I consider it a negative development
• 4% (10 votes) – I don’t know
• 2% (5 votes) – Other

Summing the first three options tells us that 82% of the voters think or hope that IBM collaboration in OpenJDK will produce positive results. Only 10% of the voters consider the collaboration to have a downside (unfortunately, no one took the time to post a comment describing the downsides they see).

New poll: Apple’s Java announcement and the future of Java on Mac

The new java.net poll focuses on the Apple announcement that its Java port is being deprecated. This new poll perhaps has a relationship with the previous poll, if one considers the OpenJDK as potentially filling the Java gap that will be left on Mac platforms due to Apple’s decision.

The new poll asks What does the announced deprecation of Java from Mac OS X mean for the future of Java on Mac platforms? Voting will be open for the next week.

Java Today

Dustin Marx provides Ten Tips for Using Java Stack Traces:

Most Java developers are somewhat familiar with Java stack traces and how to read and analyze stack traces. However, for beginning Java developers, there are some areas of stack traces that might be a little confusing. In this post, I look at some tips for reading Java stack traces and responding appropriately based on what the stack trace reports…

Alexis Moussine-Pouchkine introduces A practical guide to configuring and testing GlassFish 3.1 Clustering:

The main theme for GlassFish 3.1 is clustering which really encompasses centralized admin, load-balancing and in-memory state replication (HA). These features are all available in the 2.x family and are now being introduced in the OSGi-based and JavaEE6-compatible GlassFish product. While a lot of engineering time has been spent on making the clustering configuration as easy as possible…

Adam Bien recalls Steve Jobs at JavaOne, Mac OS X and Java, then fast-forwards to today:

Scott McNealy and Steve Jobs at JavaOne. “…One of the big surprises was the presence of the venerable Steve Jobs. Jobs underscored the commitment by Apple Computer to ship the Java 2 Standard Edition (J2SE) with their upcoming MacOS X release later this year. This is great news for Mac users and Java developers alike…”

The JCP Program Office announces the availability of the JCP EC Elections Discussion Board:

The final phase of the 2010 JCP program EC Elections is going on now and ends November 1 at midnight PST (November 2 at 8:00 AM UTC). This is an opportunity to have your thoughts, views, and opinions heard-all while helping shape the future of Java technology. There is a discussion board on jcp.org for community members to post questions to the candidates for the 2010 JCP EC Elections…

Spotlights

Our latest java.net Spotlight is the DEVOXX Supporting JUGs page:

67 Java User Groups have registered as supporting DEVOXX this year. The java.net Java User Groups Community notes that the “official” annual Java User Group leaders networking BOF at DEVOXX is scheduled for November 18 at 20:00.

We’re also highlighting Jim Weaver’s “Eye on Visage: Compiler Preview #1 Available”:

The Visage Programming Language is moving forward, with Compiler Preview #1 available now.  This preview features Default Properties, which create a simplified syntax that makes it easier to read nested data structures.  A logo has also been chosen for the project as well…

Poll

Our current java.net poll asks What does the announced deprecation of Java from Mac OS X mean for the future of Java on Mac platforms? Voting will be open through next Monday.

Subscriptions and Archives: You can subscribe to this blog using the java.net Editor’s Blog Feed. You can also subscribe to the Java Today RSS feed and the java.net blogs feed. You can find historical archives of what has appeared the front page of java.net in the java.net home page archive.

– Kevin Farnham

Twitter: @kevin_farnham

I get asked this question often. It was mentioned again recently in a NYTECH executive breakfast with RedHat CIO Lee Congdon.
The short answer is No.
There is clear evidence that in the short to medium term Oracle will continue to promote and enhance MySQL. Some of these indicators include:

It is clear from these sources that Oracle intends to incorporate MySQL into Oracle Backup and Security Vault products. Both a practical and necessary step. There is also a clear mention of focusing on the Microsoft platform, a clear indicator that SQL Server is in their sights without actually saying it.
What is unknown is exact how and when features will be implemented. Also important is how much these may cost the end user. Oracle is in the business of selling, now an entire H/W and S/W stack. They also have a complicated pricing model of different components with product offerings. I assume this will continue. There are already two indications, InnoDBbackup included for Enterprise Backup (from April Keynote) and 5.1 enterprise split. (Note: while this split may have existed prior to Oracle, it is now more clearly obvious).
MySQL can never be seen as drawing away from any Oracle sales of the core entry level database product. It is likely Oracle will provide a SQL Syntax compatibility layer for MySQL within 2 years, however it will I’m sure be a commercial add-on. Likewise, I would suspect a PL/SQL lite layer within 5 years, but again at a significant cost to offset the potential loss of sales in the low end of the server market. There continues to be active development in the MySQL Enterprise Monitor, MySQL Workbench and MySQL Connectors which is all excellent news for users.
Moving forward, how long will this ancillary development of free tools continue? What will happen to the commercial storage engine, OEM and licensing model after the 5 year commitment? How will the MySQL ecosystem survive.? There is active development in Percona, MariaDB and Drizzle forks, however unless all players that want to provide a close MySQL compatible solution work together, progress will continue to be a disappointing disjointed approach. The 2011 conference season will also see a clear line with competing MySQL conferences in April scheduled at the same time, the O’Reilly MySQL conference in Santa Clara California and the Oracle supported(*) Collaborate 2011 in Orlando, Florida.
I have a number of predictions on what Oracle ME MySQL may look like in 5 years however this is a topic for a personal discussion.