More WiX articles on Python problems and blogging location

First:

I felt it was time to clear up some more of the WiX-situation with regards to pip on windows.
As, for example, both pyodbc and lxml simply won't install without vcvarsall.bat, several packages aren't installable via pip on Windows.

The most elegant solution to this is not installing Visual Studio on all machines you want to run your application, but to build .msi-installers that are easily included in your installation.

So don't wait, but go and see the article on the Optimal BPM wiki: How to make .msi packages from Pythonlib .exe-files.

I can answer some questions in the comments here if needed.


Second:

This blog will move to optimalbpm.se soon.

Bye!

MongoDB,BPM, QAL 0.4.0, Optimal Sync 0.9.0 and improved WiX article

Releases

Recently, QAL and Optimal Sync(Optimal BPM Tools) was released in new versions(QAL, Optimal Sync). Many a new features are included and especially Optimal Sync is shaping up to become a pretty useful tool. Especially, the Windows installation of Optimal BPM Tools(Optimal Sync) is much improved.

MongoDB

I am currently investigating using MongoDB as a database backend for the system. There is currently a backend written that uses a EAV structure that works in all backends(DB2, Oracle, Postgres, MySQL, SQL server).

However, since Optimal BPM is geared towards simplicity, it is pointless to make the users learn installing and maintaining a complex relational database backend, when possibly, there are better options.

The document-oriented approach of MongoDB seems, at least so far, be a perfect fit for a BPM system. All possible possible issues, like data integrity, schema upgrade issues and similar concerns, are under investigation and hopefully resolved this week.

This means a couple of things for Optimal BPM:

1. It will be even easier to install.
2. It can have a far simpler data layer.
3. The system only needs to support one database backend for installations.
4. QAL doesn't need to support more DDL functionality, and can evolve further into the data mining and transformation direction in a more clean manner.
5. The UBE project, and its upgrader functionality, can be discontinued.

A surprising way of using Optimal Sync, if I choose to develop that functionality, is to make it able to transform from an old MongoDB logical schema to a new, using a old JSON-schema and a new JSON-schema and map the differences. This way one can handle database structure upgrades without having to write scary upgrade scripts.

WiX

The bane of many a developer, WiX, has now been honored by an improved write-up at the wiki. The script/custom action section has been moved to its own article:  http://www.optimalbpm.se/wiki/index.php/Wix_and_scripting_languages .
The old article is now more as it is supposed to be, a pre-tutorial introduction.

QAL API documentation

The QAL API documentation is now online for your browsing pleasure!

Python3-Sphinx was used, a small article on that subject may develop over time as well.


Cheers!

Windows installer done at last!

Phew. That was actually pretty hard. The WiX and MSI documentation is both lacking and confusing.

Anyway, the installer is now available for download at the files section at source forge.

It is burn-based, and downloads dependency packages and installs them.

As I mentioned before, I made a WiX primer, I have now updated that with some information on how to install pip packages:
http://www.optimalbpm.se/wiki/index.php/WiX.


Cheers!

WiX, .msi and a setuptools(easy install) and pip bundle

Hi,

I am forging ahead with the making of the .msi-files for the windows installation of Optimal BPM tools (or rather Optimal Sync, to be fair).

During the process of making the installation bundle(I want to include everything) i have been forced to create som  .msi:s that I would think could be pretty useful outside this project, namely a setuptools and pip windows installation bundle. Please download and if you encounter any problem, just create a ticket at the Sourceforge project page.

If course, for a Python application, these programs are very important. So now there is a way to easily include easy install and pip in an Python 3.3 windows deployment for those who want that. The installation checks for a Python 3.3 installation and installs the files in its /Scripts-folder.

By the way, WiX is one of the least well documented tools i have ever encountered. At the same time it is pretty complicated. And strange. So it hasn't been easy. Far from it.

However, in the end, I have managed to create a pretty flexible setup using WiX preprocessor variables. Which actually works in a predictable manner as opposed to WiX Properties and normal WiXVariable constructs which might or might not be available at "bind" time.

My current recommendation is to use preprocessor variables instead of the other stuff as they don't resolve in an equally predictable manner.
I will elaborate on that further on my writeup later on.



Until then, have a nice time.

Windows, MSI and fixes

Hi everybody,

The seriously bug fixed Optimal Sync(as of now at 0.8.3)  is about to be installable on Windows as well soon. I have started building an .msi-package, which turned out to be a bit more problematic than expected. However, I think I have overcome most of the problems.

So much, actually, that I have felt compelled to poste a guide on how WiX works and its caveats. The guide is thought of as a primer before you get into the official tutorial, which is a bit wordy and difficult to get.

I consider the problems I encountered quite typical of what most that develop an .msi-package encounters and that my example file is more useful. Especially, I use the paraffin.exe-utility instead of the quite insufficient heat.exe . I would imagine that most does, maybe the article can save them some time.

Anyway, the guide is at the wiki, here, and I hope it will help you understand WiX and its somewhat maddening quirks a bit better.

Cheers!

First release of the Optimal Sync application/script - 0.8!

 

Tagline(s):

What? Sales department saving their contacts in an excel sheet? You want their edits maintained in the CRM database? Do you regularly want to download a data set from a table in a web page? And transform the data a little before you insert it?

It's out!

Optimal Sync is a small and simple multi-platform(well soon) BPM utility that is able to move and affect data between different many sources and destinations.
As it is dependent on Python 3.3 and just recently released open source software packages, it requires at least Debian "Jessie" or Ubuntu 13.04. It will however stay with stable releases after its initial release. 0.9 will mean a tested Windows version and 1.0 OS X.
That needs to be restated, this is a first release, so there are some quirks. However the ambitions are high and there is an aggressive release schedule ahead.
Please try it out and don't hesitate to report any issues as tickets here at the project page and you'll get a response.
(The GUI is "ugly"? Well, please share how to make it prettier!)
Download packages at Sourceforge:
https://sourceforge.net/projects/optimalbpm/files/Optimal%20BPM%20tools/
Install QAL before Optimal Sync:
http://www.optimalbpm.se/wiki/index.php/Optimal_Sync#From%20packages
More documentation at:
http://www.optimalbpm.se/wiki/index.php/Optimal_Sync

And phew.

The above is obviously a quote from the Sourceforge blog post.
So finally something actually installable and runnable has emerged from all this experimenting and fiddling around. Saying it is a cool application is perhaps not the right word, the GUI is very much leaning towards "functional" and leaning away from "aesthetic".

BUT, it is all open source, and it uses the latest libraries available. In fact, so new are they, that this could not have been released a month ago. There will now be a consolidation phase where only the application and QAL will evolve as the underlying libraries goes into stabler repositories. Cool stuff will be added indeed, when all the technical hurdles has been cleared.

Anyway, happiness abound!

Happy Easter!

Simple replicator

The first thing coming out of the Optimal BPM project will be the 0.8 version of a tool called Simple Replicator. The Business Process Management angle is that transfer and transformation of data is a fundamental part of BPM. Also, the simple replicator is just that, simple. It just scratches the surface of what is possible to do with QAL.

I'll begin with Debian, then Windows and Redhat(v 0.9), Mac OS X (1.0) packages, But if you just can't wait for .deb packages, you can try out the source already, it will not change much until release:
git clone git://git.code.sf.net/p/optimalbpm/code optimalbpm-code



However, in most situations you don't have to join rows from a webpage HTML tag tree with rows from a RDBMS with those of a .csv file. So that is what "simple" means.
The full potential of QAL and Optimal BPM will instead be managed in a web-based interface. If you want to se examples of what QAL can do, look at the the unit tests. Information about  manual installation is on the wiki, see below.

I have already started using the ticket system at Sourceforge to track some of the development, it works ok for that purpose, however, and of course, it is not near what bugzilla and other bugtrackers can provide, but will suffice for now. You can view and add tickets here. All questions are welcome.

There is also the beginning of a wiki brewing. So far there is only an article about QAL's database abstraction layer and the Optimal BPM tool Simple Replicator

Cheers!