Thrift is a wonderful protocol, it’s all sum up in a single paragraph:

Thrift is a software framework for scalable cross-language services development. It combines a powerful software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, and Ruby. Thrift was developed at Facebook and released as open source.

And I really like it especially now since the system I’ve build and supporting now are written in python / c++ / ruby, these systems need to talk to each other, thrift, it’s like the answers to the prayers! : )

I have back-end systems that runs in Linux, and has client app that runs in Windows. Compiling and using thrift in Linux based system are straight forward, but unfortunately its not as easy in MS Windows platform.

At this time of writing, the code checkout from Thrift Apache Incubator still need to be further modify so that it can be compile with MSVC. These are the steps I’ve went through to get the thrift for python to build:

Ingredient:
1. Python 2.6.4
2. Visual C++ 2008 Express Edition
3. thrift-incubating-0.2.0.tar.gz
4. Windows Xp (obviously)

Read this post at http://issues.apache.org/jira/browse/THRIFT-252 to understands what needs to be edit.

Build thrift for python:
1. On Linux machine, apply the patch to the thrift source. Yes, on Linux machine, or you’ll have line ending problem.
2. Install python 2.6.4 and Visual C++ 2008 Express Edition on your Windows XP machine.
3. Copy the patched thrift source to your Windows machine, e.g: C:\tmp\thrift-0.2.0\
4. Start a command prompt and follow the following steps:

C:\> cd c:\tmp\thrift-0.2.0\lib\py\
C:\tmp\thrift-0.2.0\lib\py> python setup.py install

That’s it! Everything should compile and install to your python site-package directory. To test just do the following:

C:\> python
Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import thrift
>>>

No error is good news : )

I had to modify the patch by Michael Greene to fix a problem where the LITTLE_ENDIAN and BIG_ENDIAN was not define in my attempt of getting it build with MSVC9, other than that, everything else he did work marvelously, thanks! : )

Here’s the diff file that work for me, and the binary build that should work for python 2.6.4 on Windows.

just the diff file to modify fastbinary.c and setup.py
thrift-0.2.0-msvc-patch.diff

the patched source package:
thrift-0.2.0-msvc-patched.tar.gz

if you want to skip the compilation and just want to thrift right away with python 2.6.4, here’s the binary, extract and copy to C:\Python26\Lib\site-packages\
thrift-python-2.6.4-win32.zip

NOTE: Due to the source of thrift are still actively being improve, it’s best not to apply the diff to the source you’ve just check out, as it MAY have already been fix, its best to open up the related files, and read through, to make sure the patch is still needed.

I’m working on a new project, with new VS2008 on my new desk and new chair, what better time to update the log4cxx library that I’ve been using since then : )

I’ve quickly downloaded the source package from the official log4cxx site and I’m really happy that it comes with MSVS project files now!

But soon I realize that it only build DLL versions of the log4cxx, and since I need the static version of log4cxx I’ll have to go back to the ANT tool.

As usual you’ll still need to get the apr, apr-util and other contrib package and setup ANT for the task, and since I don’t have that environment anymore I just had to get all these package allover again.

Everything went a lot smoother then the last time I did this about a year ago, with the right package and build environment everything went just smooth (hooraay~)

The ANT tool still generates the static/shared/release/debug projects files separately. So just like what I did before, I generate the VC6 projects for each build type, and merge it into a single solutions that can compile and build all build type. Plus I also included the required external dependencies (apr / apr-util / etc) so you will just need to download single package and starts get log4cxx to compile on your machine.

I’ve made two package available, one for VC6, another for VC2008 (VC9) from my previous blog entry here.

p/s: did I told you that I’ve got my new desktop machine and new laptop machine as well? ^_^

With our current Sprint’s Backlog item completed, me and my colleague decided to trying out Pair Programming just for the fun of it. I volunteer to be the driver while he be the navigator (I actually think “back-seat driver” would be a more appropriate term )

We dig-in on my station, and working on a backlog item (for the next sprint, which yet to start yet). I must admit that the progress of completing the task is pretty fast, we exchange idea on the solution before code it in, and on the first run the code is working And not to mention that the whole activity it keeps me awake! which I normally doze-off in the afternoon while working alone~

I have a feeling that we’re going to do this more often in the future

I’ve got quite a number of request in email requesting me for a simple log4cxx sample program that illustrate the usage of log4cxx ever since I’ve wrote this article. I’ve sent my sample program in emails to those request, but I guess it will be more productive if I would just put up a sample usage here.

This will be a to-the-point log4cxx tutorial, you will probably need the binary here before you proceed:

The log4cxx XML Configuration file ( MyLogConfig.xml ) will shows you the usage of following appender:

1. org.apache.log4j.ConsoleAppender
2. org.apache.log4j.FileAppender
3. org.apache.log4j.rolling.RollingFileAppender
4. org.apache.log4j.RollingFileAppender
5. org.apache.log4j.net.SMTPAppender
6. org.apache.log4j.jdbc.JDBCAppender
7. org.apache.log4j.net.XMLSocketAppender

The MyLogConfig.xml is properly commented, so it should be self explanatory. ( and no, there is no typo, the configuration file actually refer it as log4j )

Writing The Code

Create a C++ Console project in VC++, and put in the following code:

#include <log4cxx\logger.h>
#include <log4cxx\xml\domconfigurator.h>
#include <windows.h>
 
using namespace log4cxx;
using namespace log4cxx::xml;
using namespace log4cxx::helpers;
 
// Define static logger variable
LoggerPtr loggerToFile(Logger::getLogger( _T("MyLogger") ) );
LoggerPtr loggerMyFunctionA(Logger::getLogger( _T("MyFunctionA") ));
 
void MyFunctionA()
{
    LOG4CXX_INFO(loggerMyFunctionA, _T("Executing MyFunctionA."));
}
 
int _tmain(int argc, _TCHAR* argv[])
{
    // Load configuration file
    DOMConfigurator::configure("MyLogConfig.xml");
 
    // Loop something
    for(int i=0; i&lt;10; ++i)
    {
        LOG4CXX_DEBUG(loggerToFile, _T("this is a debug message."));
        LOG4CXX_INFO (loggerToFile, _T("this is a info message, just ignore."));
        LOG4CXX_WARN (loggerToFile, _T("this is a warn message, dont worry too much."));
        LOG4CXX_ERROR(loggerToFile, _T("this is a error message, something serious is happening."));
        LOG4CXX_FATAL(loggerToFile, _T("this is a fatal message, crash and burn!!!"));
        MyFunctionA();
 
        Sleep(1000);
        printf("i = %d\n", i);
    }
 
    getchar();
    return 0;
}

Writing the log4cxx Configuration File

The log4cxx configuration file as follows ( MyLogConfig.xml ):

<?xml version="1.0" encoding="UTF-8" ?>
 <log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/"> 
 	<!-- Output the log message to system console.
 	-->
 	<appender name="MyConsoleAppender" class="org.apache.log4j.ConsoleAppender"> 
 		<param name="Target" value="System.out"/> 
 		<layout class="org.apache.log4j.PatternLayout"> 
 			<param name="ConversionPattern" value="%-5p %c{1} - %m%n"/> 
 		</layout> 
 	</appender> 
 
 
 	<!-- Output the log message to a log file named "NormalLogFile.log"
 	-->
 	<appender name="MyNormalAppender" class="org.apache.log4j.FileAppender">
 		<param name="file" value="NormalLogFile.log" />
 		<param name="append" value="true" />
 		<layout class="org.apache.log4j.PatternLayout"> 
 			<param name="ConversionPattern" value="%d %5p %c{1} - %m%n" /> 
 		</layout>
 	</appender>
 
 
 	<!-- the following appender with the name "TimeBasedLog.log", every night a few seconds after
 	     12::00PM the old log will be renamed with append the date in filename, and a new log file
 	     with the name "TimeBasedLog.log" will be create. 
 	     notice the RollingFileAppender is under "org.apache.log4j.rolling" namespace
 	-->
 	<appender name="MyRollingAppenderDaily" class="org.apache.log4j.rolling.RollingFileAppender">
 		<rollingPolicy class="org.apache.log4j.rolling.TimeBasedRollingPolicy">
 			<param name="FileNamePattern" value="TimeBasedLog.%d{yyyy-MM-dd}.log"/>
 			<param name="activeFileName" value="TimeBasedLog.log"/>
 		</rollingPolicy>
 
 		<layout class="org.apache.log4j.PatternLayout">
 			<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss,SSS} %x [%p] (%F:%L) %m%n"/>
 		</layout> 
 		<param name="file" value="TimeBasedLog.log"/>
 		<param name="append" value="true"/>
 	</appender>
 
 
 	<!-- On application startup, a log file named "SizeBasedLog.log" will be create if not exist.
 	     When the log file reach beyond 5KB, it will be renamed "SizeBasedLog.log.1", when the log
 	     index reach "SizeBasedLog.log.5", the next rename will be back to "SizeBasedLog.log.1" and
 	     overite the old log. 
 	-->
 	<appender name="MyRollingAppenderSize" class="org.apache.log4j.RollingFileAppender">
 		<param name="file" value="SizeBasedLog.log"/>
 		<param name="append" value="true"/>
 		<param name="MaxFileSize" value="5KB"/>
 		<param name="MaxBackupIndex" value="5"/>
 		<layout class="org.apache.log4j.PatternLayout">
 			<param name="ConversionPattern" value="%d %-5p [%c] %m%n"/>
 		</layout>
 	</appender> 
 
 
 	<!-- the following appender creates a logfile in log4j XML format, suitable to viewing with
 	     XML log viewer such as Chainsaw (http://logging.apache.org/log4j/docs/chainsaw.html)
 	-->
 	<appender name="MyLogFileAppenderXml" class="org.apache.log4j.RollingFileAppender">
 		<param name="file" value="XmlLog.txt" />
 		<param name="append" value="true" />
 		<param name="ImmediateFlush" value="true" />
 		<layout class="org.apache.log4j.xml.XMLLayout" />
 	</appender> 
 
 
 	<!-- This appender will send email through SMTP server.
 	-->
 	<appender name="MySMTPAppenderEmail" class="org.apache.log4j.net.SMTPAppender">
 		<param name="BufferSize" value="512" />
 		<param name="SMTPHost" value="smtp.youremailserver.com" />
 		<param name="SMTPPort" value="25" />
 		<param name="From" value="yourname@youremailserver.com (mailto:yourname@youremailserver.com)" />
 		<param name="To" value="yourname@youremailserver.com (mailto:yourname@youremailserver.com)" />
 		<param name="CC" value="someoneelse@youremailserver.com (mailto:someoneelse@youremailserver.com)" />
 		<param name="SMTPUsername" value="yourusername" />
 		<param name="SMTPPassword" value="yourpassword" />
 		<layout class="org.apache.log4j.PatternLayout">
 			<param name="ConversionPattern" value="[%d{ABSOLUTE},%c{1}] %m%n"/>
 		</layout>
 	</appender> 
 
 
 	<!-- This appender will write the log message to a database through ODBC connection
 	     to Database.
 	-->
 	<appender name="MyOdbcMysqlAppender" class="org.apache.log4j.odbc.ODBCAppender">
 		<param name="URL" value="Driver={MySQL ODBC 5.1 Driver};Server=localhost;Database=errorlog;User=logger;Password=abc123;Option=3;"/>
 		<layout class="org.apache.log4j.PatternLayout">
 			<param name="ConversionPattern" value="INSERT INTO errorlog (errormessage) VALUES ('%d - %c - %p - %m')"/>
 		</layout>
 	</appender> 
 
 	<appender name="MyOdbcPostgresAppender" class="org.apache.log4j.odbc.ODBCAppender">
 		<param name="URL" value="Driver={PostgreSQL UNICODE};Server=localhost;Port=5432;Database=errorlog;Uid=postgres;Pwd=abc123;"/>
 		<layout class="org.apache.log4j.PatternLayout">
 			<param name="ConversionPattern" value="INSERT INTO errorlog (errormessage) VALUES ('%d - %c - %p - %m')"/>
 		</layout>
 	</appender> 
 
 
 	<!-- This appender will write log message and send it through XMLSocketAppender, a receiver
 	     with XMLSocketReceiver such as Chainsaw (http://logging.apache.org/log4j/docs/chainsaw.html)
 	     will be able to receive and interpret it.
 	-->
 	<appender name="MyXmlAppender" class="org.apache.log4j.net.XMLSocketAppender">
 		<param name="Port" value="1234"/>
 		<param name="RemoteHost" value="10.7.5.15"/>
 		<param name="ReconnectionDelay" value="60000"/>
 		<param name="LocationInfo" value="true" />
 	</appender> 
 
 
 	<!-- Using XMLSocketAppender couple with pattern layout to send text lines to 
 	     a ordinary TCP listener, any application that implements TCP socuket listening
 	     will be able to receive the log message.
 	-->
 	<appender name="MyXmlAppenderFormated" class="org.apache.log4j.net.XMLSocketAppender">
 		<param name="Port" value="1235"/>
 		<param name="RemoteHost" value="localhost"/>
 		<param name="ReconnectionDelay" value="60000"/>
 		<param name="LocationInfo" value="true" />
 		<layout class="org.apache.log4j.PatternLayout">
 			<param name="ConversionPattern" value="%d %-5p [%c] %m%n"/>
 		</layout>
 	</appender>
 
 
 	<!-- Setup the root category, add the appenders and set the default level 
 	     5 level of logging,  ALL < DEBUG < INFO < WARN < ERROR < FATAL 
 	     The root level is set with INFO, which mean any message greater or same
 	     as INFO will be log down, in this case, DEBUG is not logged. 
 	     To log all regardless of logging level, set <priority value="ALL">
 	-->
 	<root>
 		<priority value="all" />
 		<appender-ref ref="MyRollingAppenderDaily"/>
 		<appender-ref ref="MyRollingAppenderSize"/>
 		<appender-ref ref="MyConsoleAppender"/>
 		<appender-ref ref="MyLogFileAppenderXml"/>
 		<appender-ref ref="MyXmlAppenderFormated"/>
 		<appender-ref ref="MyXmlAppender"/>
 		<appender-ref ref="MyOdbcMysqlAppender"/>
 		<appender-ref ref="MyOdbcPostgresAppender"/>
 	</root> 
 
 	<!-- Specify the level for some specific categories -->
 	<category name="MyFunctionA" >
 		<priority value ="info" />
 		<appender-ref ref="MyNormalAppender" />
 	</category> 
 
 </log4j:configuration>

You may notice that the tag has log4j in it, that’s fine, it’ll work

I can’t remember where did I gather all the pieces up there, but I kind of remember that it was pieces from different sources before I have put in together. Let me know if you need additional instructions. Is this helpful? Happy Logging ; )

Updated 27 Nov 2008:

As Greg pointed out, the JdbcAppender is not working with log4cxx. I did some digging in the source and realize that the real reason is that JdbcAppender is not there in the source at all, or it has not been ported yet. However, I’ve updated the XML config file to use OdbcAppender to write to PostgreSQL and MySql database. Sorry for the misleading sample of XML configuration

Note: When testing with Mysql5 with log4cxx through odbc, my test app would crash on exit, the call stack shows that myodbc5.dll crashed. Everything works fine with PostgreSQL, so I’m not sure if this is MySql specific problem or not… hm…

I was dealing with a cryptic message of the followin:

[color=red]ERROR 2006 (HY000) at line 88: MySQL server has gone away[/color]

Apparently a SQL query *can* be too big and MySQL would just *gone away* …
[url=http://dev.mysql.com/doc/refman/5.0/en/packet-too-large.html]http://dev.mysql.com/doc/refman/5.0/en/packet-too-large.html[/url]

In case you have huge query like I do, change the my.ini file to allow bigger query is a very good idea.

[code]my.ini
[mysqld]
max_allowed_packet=32M
[/code]

I bump into this nifty tool [url=http://www.rauchy.net/regionerate]Regionerate[/url] made by Omer Rauchwerger that apply style setting (xml configured) on your C# source and group them under #region sections. Of course saying anything is pretty pointless here since you should watch the [url=http://www.rauchy.net/regionerate/2007/06/regionerate-101.html]screencast here[/url]

I quickly downloaded version 0.6.5.0 and give it a try on my project (after making sure I committed the source into my repository first ; )

The default codelayout dint suite me well, so I hack up code layout “Mass.Separation” code layout by Derik Whittaker and produce the following layout:

[code][rgn] Delegates and Events
[rgn] Event Handlers
[rgn] Enums

[rgn] Constant Variables
[rgn] Static Variables
[rgn] Instance Variables

[rgn] Public Properties
[rgn] Private Properties
[rgn] Protected Properties

[rgn] Constructors
[rgn] Public Methods
[rgn] Private Methods
[rgn] Protected Methods[/code]
Me being un-imaginative, named it Mass.SeparationEx ; ) Credit should go to Omer Rauchwerger and Derik Whittaker. The CodeLayout file for Regionerate is as follows:
Read more »

Sometimes classes become far too intimate and spend too much time delving in each other’s private parts.

I was laughing my heart out when I read that line in Refactoring: Improving The Design Of Existing Code by Martin Fowler.

Apart from making a point of one should avoid “over intimacy” between class, that paragraph trigger a memory of a colleague trying to explain “friend” class in C++ to me a little over seven years ago, he says:

Imagine you have Boy class A and Girl class B, obviously they can only touch on each other’s public methods, however, if you grant “friend” Boy class A and Girl class B to become Boy friend class A and Girl friend class B, then this newly relationed class will be able to touch each other’s private parts.

You get the idea?

Its been a while I got this pain that my blog does not have the paging link ( e.g: ) at the bottom of my page to facilitate browsing old pages.

Sepaking of that, I found a nice little plugin, [color=blue]NP_ShowBlogs[/color], to add that pagination to my blog. To my supprise, it append the page link at the [i]TOP[/i] and the [i]BOTTOM[/i]!

A closer look at the documentation stated that “[color=green]# (*1) pageswitch will be displayed top and bottom, twice[/color]“.

An even closer look at the NP_ShowBlogs.php reveal that the page link is indeed printed twice, so, to remove the page link at the top of the page, follow the following instructions:

[code]1. Open NP_ShowBlogs.php with your favorite text editor

2. Find the following line (at line 363) and comment it out.
echo $page_switch['buf'];

3. Save the file and re-upload it.[/code]
Note: I’m using NP_ShowBlogs.php 2.66.3 scat

Yes, I wanted to have a uniform logging library for all my dev projects, after looking at log4cpp / log4cplus / log4cxx, I end up choosing log4cxx simply its obvious ability to use a external config file (just like log4j), which log4cpp and log4cplus is missing from.

Unfortunately using the library in Win32 is less then obvious.

The official site at http://logging.apache.org/log4cxx/ , provides “log4cxx-0.9.7″ but it comes with a warning: “At this point, log4cxx-0.9.7 is substantially out of date, has known serious deficiencies that have been resolved in the CVS, and should be avoided for new code.” *_*

After digging around, I downloaded the HEAD version, which suppose to be version 0.10.0, the source does not have a Visual Studio Project files, reading the installation instructions it says that I need ANT and bunch of other stuff to get it to build… fine… I can tolerate that…

After 3 hours of downloading and install the required package with exactly the same version of files required as stated in the installation guide and I figure, finally the moment of truth! (actually, later I found out its actually a beginning of another few hours of frustrations!)

The ant build went on, and its throwing all kind of error, and finally display a fail build status, I don’t understands, all the util are exactly the same version as request in the installation guide. After a few hours of search and debug, reading through the build script, nothing seems to help. Then a wild idea come up, “was it because there is a space in the folder name?“, I was building in my desktop, which gives a directory “C:\Document and Setting…” (you get the idea), I move the source folder to C:\tmp and start ant build there, and it runs!

Rebuilding with ant build, and it complain “Assertion failed, hunk …“, ok… I recognize that, it because of the sed util can’t handle CRLF in win32, so I went on writing a script, and change all CRLF in *.patch file into CR, that got rid of the error.

Rebuilding with ant build, building going on… splash BUILD FAILED! Apparently the unit test failed, did some search in the mailing list, nothing comes up…

Ok… I figure since ant build supports an option to build a VS2005 project file, which can be open with VS2005, it might be a better idea then relying on the Ant tool for the build.

Building the VS2005 project file gives a BUILD SUCCESS!

I open up the generated project file with VS2005, and start the compilation process, everything seems fine, until at the last bit of the compilation it splat me “fatal error LNK1104: cannot open file ‘.obj’ “. WTF is “.obj” ? I look into the vs project properties, no such thing in linker, and the path for link library is properly encase with double quote as well, every setting seems fine, but the darn library just wont compile… *_*

Ok… fine… ant tool still have another options to build a VC6 project file, oh hell why not? I’ve come this far… VC6 project file generation with ant tool yield BUILD SUCCESS (yes.. again). Load it up with VC6 and compile… Pronto! Success! I’ve finally got log4cxx binary!

I open up the VC6 project file with VS2005, and upgrade it to VS2005 compatible, and with my finger cross I hit compile… and it WORKS too!

Only problem now is that the ant build tool only generate a vs project file once per configuration, that mean if I want Debug Static library I have to build one file, Release Static library I have to build another, Debug Dynamic Link Library another one… so on and so forth…

I went on and build all the VC6 project file, rename it each time, since the build tool will overwrite its only use the same file name.

I upgrade all the VS6 project files to VS2005 compatible, and later hand edit them, so that it all fit nicely in a single solution, that will build both the Static and Dynamic version of log4cxx, Debug / Release build.

By the time I did that, its already the 4th day since I started of this log4cxx journey… pain pain pain… but it all came through…

To make your life less miserable, here’s the downloads:
VS2005 Binary Build of log4cxx v0.10.0 (out dated May 2007)
VS2005 Project + Source for log4cxx v0.10.0 (out dated May 2007)

VC6 Project + Source for log4cxx v0.10.0 (out dated May 2007)
(I’ve not try to compile this on a VC6 before, the purpose of me making this available is to allow you to convert the project file to other version of VS if you’re not using VS2005)

VC6 Binary Build + Source of log4cxx v0.10.0 (updated June 2008)
VC2008 Binary Build + Source of log4cxx v0.10.0 (updated June 2008)
(if you use other version of MSVC, then just get the VC6 package and upgrade the project files to your MSVC version)

Now please say thank you : )

Note:
To use the static link log4cxx library, the macro LOG4CXX_STATIC MUST be defined in your projects preprocessor, otherwise your compilation will suffer unresolved linking error.

I’m getting tired that to make sure everyone put in the ignore pattern in their copy of TortoiseSVN so that the un-necessary garbage file from being committed into the repository, so I decided to put the mechanism on the server side to enforce this instead.

After some search I’m really surprise that there’s no one at all that has publish their solution, well except for Simon.K. He was nice enough to publish his script, and I made a little changes to suite my need, and viola! Here’s the script, I hope it helps you too:
[code]

import os, sys, tempfile, re, fnmatch

try :
# NOTE(sk): the 'script' name is the first argument
if not len(sys.argv) == 3 :
sys.exit("Invalid arguments")

# NOTE(sk): Args popped off in reverse order
# so the repospath is expected before the transaction id
transaction = sys.argv.pop()
repospath = sys.argv.pop()

# NOTE(sk): Match 4 characters and the remainder is the filename
status_re = re.compile("(.{4})(.+)")
# NOTE(sk): Match a file with a specific file extention or a directory (ending with '/')
ext_re = re.compile(".+?(\.(ilk|bsc|sbr|projdata|pdb|obj|tmp|temp|csproj.user|vbproj.user|o|obj|ncb|suo|pyc|pch)$|/$)")

# NOTE(sk): 'svnlook' command expected in global environment variables'
svnpipe = os.popen("svnlook changed -t %s %s" % (transaction, repospath))
contents = svnpipe.read()
svnpipe.close()

# Process the output
for line in contents.rstrip().split('\n') :
m = status_re.match(line)
if not m :
sys.exit("Incorrect format: " + line)
attributes = m.group(1)
filename = m.group(2)
if attributes[0] == 'A' :
if not filename[len(filename) - 1] == '/' :
if ext_re.match(filename) :
sys.exit("Invalid extension: " + filename)

except SystemExit:
raise
except Exception, e:
# NOTE(sk): Only the 'SystemExit' exception seems to set the return
# result correctly
sys.exit(str(e))
[/code]