Website traffic analysis

Since the middle of october I run awstats, a CGI program for reports about website visits and traffic. It reveals interresting results for both the websites running on my virtual server – e-lehmann.de and tloona.tk.

The summaries for october:

Summary for e-lehmann.de
Reported period Month Oct 2006 Unique visitors Number of visits Pages Hits Bandwidth Viewed traffic * 309 711 (2.3 visits/visitor) 2102 (2.95 pages/visit) 5654 (7.95 hits/visit) 205.29 MB (295.66 KB/visit) Not viewed traffic * 14573 14731 213.85 MB

* Not viewed traffic includes traffic generated by robots, worms, or replies with special HTTP status codes.

Summary for tloona.tk
Reported period Month Oct 2006 Unique visitors Number of visits Pages Hits Bandwidth Viewed traffic * 288 397 (1.37 visits/visitor) 3317 (8.35 pages/visit) 8116 (20.44 hits/visit) 264.52 MB (682.27 KB/visit) Not viewed traffic * 1268 1725 56.35 MB

* Not viewed traffic includes traffic generated by robots, worms, or replies with special HTTP status codes.

As can be seen, there were more visits on my homepage than on tloona so far. Ok, e-lehmann.de is around since some time now and also listed on google and other search engines. The number of different visitors (probably determined by IP address in the logs) is almost equal, whereas the number of visits per unique visitor is around double as high for e-lehmann.de than for tloona.tk. This is also not very surprising… tloona.tk appears at google since one week or so and most visitors come to tloona.tk probably through e-lehmann.de. Also, Tloona is not as popular at the moment as I’d like to see ;-). Sadly, it is not ranked well on Google when you search for Tcl/Tk IDE or similar. This should change – question is only how?

A more interresting aspect is the viewed bandwidth traffic. It is much higher for tloona.tk than for e-lehmann.de although the number of visits is by far smaller than for e-lehmann.de. So, what are the possible reasons for that? One reason I can imagine is, that the site is new and therefore crawled intensively by google & co. But a view a little down the page shows, that the traffic generated by crawlers is much less for tloona.tk than it is for e-lehmann.de:

Robots/Spiders visitors (Top 10) – e-lehmann.de

12 different robots* Hits Bandwidth Last visit Googlebot 12247 158.79 MB 31 Oct 2006 – 07:00 Inktomi Slurp 269 4.42 MB 31 Oct 2006 – 05:10 MSNBot 184 2.54 MB 30 Oct 2006 – 15:29 Unknown robot (identified by ‘crawl’) 184 3.11 MB 31 Oct 2006 – 03:52 WISENutbot 46 586.86 KB 30 Oct 2006 – 09:15 GigaBot 9 56.00 KB 31 Oct 2006 – 04:20 AskJeeves 6 224.58 KB 29 Oct 2006 – 22:14 SurveyBot 4 114.29 KB 22 Oct 2006 – 23:16 Unknown robot (identified by ’spider’) 3 70.29 KB 26 Oct 2006 – 21:15 Voyager 1 9.63 KB 29 Oct 2006 – 22:44 Others 2 74.79 KB

* Robots shown here gave hits or traffic “not viewed” by visitors, so they are not included in other charts.

Robots/Spiders visitors (Top 10) – tloona.tk
6 different robots* Hits Bandwidth Last visit Googlebot 853 11.21 MB 31 Oct 2006 – 07:44 Inktomi Slurp 66 607.60 KB 30 Oct 2006 – 11:28 Unknown robot (identified by ‘crawl’) 14 122.92 KB 23 Oct 2006 – 00:58 MSNBot 13 112.28 KB 18 Oct 2006 – 03:57 Unknown robot (identified by ‘robot’) 2 11.77 KB 12 Oct 2006 – 14:55 WISENutbot 1 19.23 KB 29 Oct 2006 – 20:57

* Robots shown here gave hits or traffic “not viewed” by visitors, so they are not included in other charts.

But – as the star note indicates, this is displayed s “not viewed traffic” in the summary. What is definitely also displayed as “not viewed traffic”, is the svn traffic – if made by an svn client rather than websvn… So I think the traffic situation can be explained well enough:

• The viewed traffic for tloona.tk probably contains the nightly builds of Tloona – which is around 2.5MB each
• The not viewed traffic is 4x higher for e-lehmann.de than for tloona.tk. Although e-lehmann.de is crawled much more often by search engines than tloona.tk, this indicates the higher svn traffic for tclkit. The svn traffic for tloona is not so high by nature (Tloona is not nearly as large as Tclkit) and also, the nightly builds resemble the svn repository – so there is no real need to generate svn traffic for people who just want to use and test Tloona
• e-lehmann.de is more frequently visited, but the viewed web pages generate not much traffic
• tloona.tk is not as widely known and thus not so often visited – but there is something to download. So, the traffic is higher although the number of visited pages is less. Also, the screenshots generate more traffic than simple HTML pages. As indicated elsewhere on the report, most people who visit tloona.tk, take a look at the screenshots section.

I think that’s a good explanation for the situation. What I want to mention also is, that there are 22.94 average visits per day for e-lehmann.de at a total of 711 visits. For tloona.tk, there are 12.81 average visits per day with a total of 397 visits. Both statistics are measured from 12/10/06 until today.

Actually, that is quite good for a semi-private homepage with some structured random content that might be helpful to others ;-). But tloona.tk could do better…

Fresh Tclkit builts for 8.5a5

In order to match the new Release of Tcl8.5a5, I refreshed the Tclkit builts. The new binaries can be downloaded as usual from here.

Two things have changed in the new Tclkits. First, the Atkdebugger patch is not applied to the sources. I tried it, but is has no effect anymore – which means I can not stop the execution of a Tcl script on the as-is basis of the atkdebugger extension. This is not too bad, since (1) The debugger API did not work to 100% anyway – there were some flaws in retrieving the executing code fragment which Arthur Trzewik told me. (2) Arthur does not maintain the extension anymore. And, last but not least (3) There is a TIP about to go in Tcl8.5 that mensions extensive debugging support. It is TIP #280 (although I can not find a specification on how it will be possible to stop running code…). This seems to be more powerful and Tloona’s debugger should rely on this rather than on Atkdebugger.

The second thing that has changed is the extended [info level] procedure, that I wrote. This is not needed anymore when TIP #280 is included – and currently it is not needed because debugging support is not enabled anyway. So, [info level] works from now on as usual.

In other words, the new Tclkits are more standard conform than the older ones ;-). The old kits can still be downloaded, but maybe will be removed from here at some point in the future.

mod_tcl distribution + win32 dll

Given this to be a lazy sunday afternoon, I created a distribution of mod_tcl, including a windows dll.

The module can be compiled now against Apache 2.2.x as a shared library. It is not longer required to include it in the httpd source tree and compile it statically into apache. For windows, I created a VC Express 2005 project file and also a binary in the Release directory. If you want to compile it yourself on windows, you need VC 2005 (Express) installed, Tcl (e.g. ActiveTcl) and Apache. You don’t need the complete Apache sources, the installer from http://httpd.apache.org/ comes with headers and libraries included. You just need to adjust the paths to the Apache/Tcl include and lib directories from within the Project properties.

For Linux/Unix, the make install target utilizes apxs to install the module and generate a LoadModule line for it in httpd.conf. On windows, this must be done manually:

• install Tcl (ActiveTcl from http://www.activestate.com/ is fine)
• create the environment variable TCL_LIBRARY=%TCLHOME%\lib\tcl8.4 (%TCLHOME% is the directory where your Tcl installation lives, e.g. C:\Tcl). This can be done via the System->advanced tab in the control panel.
• Copy the file mod_tcl.dll from the Release directory into %APACHE_HOME%\modules, next to the other modules. Don’t worry that the others end with .so – it doesn’t matter. %APACHE_HOME% is where you installed Apache
• Add the line LoadModule tcl_module modules/mod_tcl.dll to the modules section in %APACHE_HOME%\conf\httpd.conf
• Restart your server. In the error_log (%APACHE_HOME%\logs\error.log) appears a line like [notice] Apache/2.2.3 (Win32) mod_tcl/1.0.1 configured — resuming normal operations.

Now you can start to write cool web development frameworks in Tcl ;-).

There is a TODO item for multithreaded servers: on newly created threads, there should also Tcl threads or new interpreters be created. When this is in place, Tcl is the first dynamic language that can cope with Apache2’s mpm-worker model! PHP has massive problems with that and the others… Who knows?
Now, for the download… It will soon be found on http://tcl.apache.org/mod_tcl/. Currently this site is down, so the distribution can be temporarily downloaded here.

New home for Tloona

Everything related to Tloona can now be found on the expressive url http://tloona.tk/*. I finally managed to enable subversion access via http (webdav), which includes readonly access for everyone. Also, a script generates nightly builds of Tloona from the repository status at 00:05 every day. The new homepage is meant to be a project forum and a publishing platform for Tloona.

Thanks in advance to everybody who contributes as user or developer.

* Which programming language can say that it has its own toplevel domain :-)?

The mythos on Tcl’s popularity

People keep complaining about the fact, that Tcl is not as popular as e.g. Python or Ruby. A discussion about this came up recently on comp.lang.tcl. Here is the Thread and a very delicious reply :-). Well, this discussion comes up every 3-6 months on this newsgroup…

I think I found the mythos about Tcl’s popularity problem. It is simple, really – according to this link, which describes the “Top ten geek business myths”. The answer can be extrapolated from myth No. #6: What you know matters more than who you know. Here is a cite of the last paragraph:

Ironically, C programmers understand this much better than Lisp programmers. One of the ironies of the programming world is that using Lisp is vastly more productive than using pretty much any other programming language, but successful businesses based on Lisp are quite rare. The reason for this, I think, is that Lisp allows you to be so productive that a single person can get things done without having to work together with anyone else, and so Lisp programmers never develop the social skills needed to work effectively as a member of a team. A C programmer, by contrast, can’t do anything useful except as a member of a team. So although programming in C hobbles you in some ways, it forces you to form groups whose net effectiveness is greater than the sum of their parts, and who collectively can stomp on all the individual Lisp programmers out there, even though one-on-one a Lisper can run rings around a C programmer.

It is funny and there is much truth inside. Tcl can be well compared to Lisp. It’s a prefix language as well, it’s simple and easy to learn and use. You get to speed with it very easy and fast. People who know and use Tcl are usualy very competent and get things done, fast and reliable. Same applies probably for Lisp programmers. This is contrahery to the crowd who claims “I know Java” and where you have to be very selective to find good professionals.
But there is maybe really a small lack of “communication skills” amongst Tcl’ers. I don’t want to say that Tcl developers are not able to work in a team – this is certainly not true. But for many tasks that can be done in Tcl, it is not necessary to build big teams. This is good if project pressure is high and resources are limited. But in other respects this is not so good, and this includes the development of popularity, selling and presentation of the language and tools or apps that are written in it.
Certain efforts are going on in this direction now, after many complainings and discussions on comp.lang.tcl. I will contribute as well (via this website, for instance) and hope that it goes on a fertile ground.