November 30th, 2009

NHibernate Lazy Loading BLOB column

For my last project we used Oracle 10g and NHibernate ORM framework (the project was started 4-5 yars ago, long before LINQ and ADO.NET Entity Framework). I had a database table which holds payment information along with scanned images of the payment documents. Usually such tables are mapped with a class fully corresponding to the table fields. The usual approach works well but has a performance problem: when the table needs to be displayed, the binary images holding the payment documents do not need to be loaded from the database because they consume much memory and take time to load. NHibernate supports lazy loading for associations only but does not support lazy column / property loading. To achieve a normal performance I needed to do a quick hack and I believe it could be useful for other projects so I want to shera it with the community.

The hack is to map all columns of the database table except the large binary column (BLOB type in Oracle). The large binary column is handle differently with additional select / update operations. See the payment entity definition below:

using System;
using System.Collections.Generic;
using System.Xml.Serialization;
using NHibernate;
using NHibernate.Criterion;
using NHibernate.Mapping.Attributes;

namespace Example
{
    [Class(0, Table = "PAYMENTS")]
    [XmlType("payment")]
    public class Payment
    {
        private int id;
        private DateTime paymentDate;
        private decimal paymentAmmount;
        private string paymentRefNumber;
        private string paymentDocumentFileName;

private static Dao<payment> dao = new Dao</payment><payment>();

[Id(1, Column = "ID", Type = "System.Int32"),
            Generator(2, Class = "native"), Param(3, Content = "SQ_PAYMENTS", Name = "sequence")]
        [XmlElement("id")]
        public virtual int ID
        {
            get { return id; }
            set { id = value; }
        }

[Property(Column = "PAYMENT_DATE", Type = "System.DateTime")]
        [XmlElement("paymentDate")]
        public virtual DateTime PaymentDate
        {
            get { return paymentDate; }
            set { paymentDate = value; }
        }

[Property(Column = "PAYMENT_AMMOUNT", Type = "System.Decimal")]
        [XmlElement("ammount")]
        public virtual decimal PaymentAmmount
        {
            get { return paymentAmmount; }
            set { paymentAmmount = value; }
        }

[Property(Column = "PAYMENT_REF_NUMBER", Type = "System.String")]
        [XmlElement("paymentRefNumber")]
        public virtual string PaymentRefNumber
        {
            get { return paymentRefNumber; }
            set { paymentRefNumber = value; }
        }

[Property(Column = "PAYMENT_DOC_FILE_NAME", Type = "System.String")]
        [XmlElement("paymentDocumentFileName")]
        public virtual string PaymentDocumentFileName
        {
            get { return paymentDocumentFileName; }
            set { paymentDocumentFileName = value; }
        }

public static IList</payment><payment> GetAll()
        {
            return dao.GetAll("paymentDate");
        }

public byte[] PaymentDocument
        {
            // Implement lazy loading for the PAYMENT_DOC blob (for better performance)
            get
            {
                ISession hbSession = NHibernateSessionManager.Instance.GetSession();
                ISQLQuery query = hbSession.CreateSQLQuery("SELECT PAYMENT_DOC FROM PAYMENTS WHERE ID=:payment_id");
                query.SetParameter("payment_id", this.ID);
                byte[] paymentDoc = (byte[])query.UniqueResult();
                return paymentDoc;
            }
            set
            {
                if (this.ID == 0)
                {
                    throw new Exception("Can not assing PaymentDocument for non-persistent payments!");
                }
                ISession hbSession = NHibernateSessionManager.Instance.GetSession();
                ISQLQuery query = hbSession.CreateSQLQuery(
                    "UPDATE PAYMENTS SET PAYMENT_DOC=:payment_doc WHERE ID=:payment_id");
                query.SetParameter("payment_id", this.ID);
                query.SetParameter("payment_doc", value);
                query.ExecuteUpdate();
            }
        }

public static Payment Save(Payment payment)
        {
            return dao.Save(payment);
        }
    }
}

The database schema of the PAYMENTS table is as follows:

I use NHibernate annotations because I want to have the entity class and the .hbm mapping in the same file.

The idea behind the “column lazy loading” hack is to skip mapping the large BLOB column and to red/write it on demand with additional SELECT / UPDATE command.
This approach is applicable to other ORM frameworks that do not support lazy column loading.

Enjoy.

Posted by nakov as .net, blog at 11:24 PM EET

Comments Off

November 21st, 2009

New Book: Introduction To Programming With C#

Today I with Vesko Kolev started a new project: writing a C# book in Bulgarian for beginners that covers the fundamental concepts of computer programming, data structures and algorithms as well as object-oriented concepts, high-quality programming code and problem solving. The book is titled “Introduction to Programming with C#” and will be based on its well-known sister, the book “Introduction to Programming with Java”. Most of the book content will be adaptation of its Java variant but some new concepts will also be introduced. The book focuses on the fundamental concepts of computer programming, not just the C# language. It covers topics like recursion, data structures (linear, trees, graphs, hash-tables and sets), combinatorial algorithms and problem solving which can not be found in every C# book. The book does not cover software engineering, Web programming, databases, desktop applications, mobile applications and other software technologies, just the bases of computer programming, logical thinking and problem solving. The book is free and open-source and is driven by volunteers who like to share their knowledge and skills with the young developers who try to make their first steps in computer programming and software engineering.

The book’s official Web site will be located at: http://www.introprogramming.info.

The developer’s Web site is located in Google Code: http://code.google.com/p/introcsharpbook/.

The SVN repository containing all project assets is located at: http://introcsharpbook.googlecode.com/svn/trunk/.

The developer’s mailing list (discussion group) is located at: http://groups.google.com/group/introcsharpbook.

Authors and editors are welcome to help the project. To join to this open-source voluntary project please contact Svetlin Nakov or Vesko Kolev. The working language is Bulgarian (some day we plan to publish an English version but let’s do the things step-by-step).

Posted by nakov as .net, news, blog at 2:46 AM EET

Comments Off

November 17th, 2009

Playing with the Scanner with WIA and C#

Last week I needed to implement a piece of software for scanning small documents (bank payment orders). The problem was to scan documents with specific size (less than A4) in 300 DPI and with normal brightness and contrast. I found that in C# there are two general approaches to this problem:

Using the TWAIN interface
Using Windows Image Acquisition (WIA) API

Both interfaces are not natively supported in .NET Framework so some kind of wrappers was necessary to be developed. I choose WIA because it was simpler and higher level API with sufficient enough power.

To use WIA you need to download it from Microsoft (it is a COM server, distributed freely): http://www.microsoft.com/downloads/details.aspx?familyid=a332a77a-01b8-4de6-91c2-b7ea32537e29&displaylang=en.

The next step is to add it as COM object reference in Visual Studio (you need a single file called wiaaut.dll that you need to register with regsvr32). Once you add the reference to WIA COM object in Visual Studio, you get the namespace WIA. All you need to scan image is in this namespace. The most important classes are: Device, Item and CommonDialogClass.

I will not get into more details because developers need code and usually read the text only if the code is not working. Am I right? Did you tread this text?

using System;
using System.IO;
using System.Windows.Forms;
using WIA;

namespace PlayingWithTheScanner
{
    public partial class MainForm : Form
    {
        public MainForm()
        {
            InitializeComponent();
        }

private void buttonScan_Click(object sender, EventArgs e)
        {
            CommonDialogClass commonDialogClass = new CommonDialogClass();
            Device scannerDevice = commonDialogClass.ShowSelectDevice(WiaDeviceType.ScannerDeviceType, false, false);
            if (scannerDevice != null)
            {
                Item scannnerItem = scannerDevice.Items[1];
                AdjustScannerSettings(scannnerItem, 300, 0, 0, 1010, 620, 0, 0);
                object scanResult = commonDialogClass.ShowTransfer(scannnerItem, WIA.FormatID.wiaFormatPNG, false);
                if (scanResult != null)
                {
                    ImageFile image = (ImageFile) scanResult;
                    string fileName = Path.GetTempPath() + DateTime.Now.ToString("dd-MM-yyyy-hh-mm-ss-fffffff") + ".png";
                    SaveImageToPNGFile(image, fileName);
                    pictureBoxScannedImage.ImageLocation = fileName;
                }
            }
        }

private static void AdjustScannerSettings(IItem scannnerItem, int scanResolutionDPI, int scanStartLeftPixel, int scanStartTopPixel,
            int scanWidthPixels, int scanHeightPixels, int brightnessPercents, int contrastPercents)
        {
            const string WIA_HORIZONTAL_SCAN_RESOLUTION_DPI = "6147";
            const string WIA_VERTICAL_SCAN_RESOLUTION_DPI = "6148";
            const string WIA_HORIZONTAL_SCAN_START_PIXEL = "6149";
            const string WIA_VERTICAL_SCAN_START_PIXEL = "6150";
            const string WIA_HORIZONTAL_SCAN_SIZE_PIXELS = "6151";
            const string WIA_VERTICAL_SCAN_SIZE_PIXELS = "6152";
            const string WIA_SCAN_BRIGHTNESS_PERCENTS = "6154";
            const string WIA_SCAN_CONTRAST_PERCENTS = "6155";
            SetWIAProperty(scannnerItem.Properties, WIA_HORIZONTAL_SCAN_RESOLUTION_DPI, scanResolutionDPI);
            SetWIAProperty(scannnerItem.Properties, WIA_VERTICAL_SCAN_RESOLUTION_DPI, scanResolutionDPI);
            SetWIAProperty(scannnerItem.Properties, WIA_HORIZONTAL_SCAN_START_PIXEL, scanStartLeftPixel);
            SetWIAProperty(scannnerItem.Properties, WIA_VERTICAL_SCAN_START_PIXEL, scanStartTopPixel);
            SetWIAProperty(scannnerItem.Properties, WIA_HORIZONTAL_SCAN_SIZE_PIXELS, scanWidthPixels);
            SetWIAProperty(scannnerItem.Properties, WIA_VERTICAL_SCAN_SIZE_PIXELS, scanHeightPixels);
            SetWIAProperty(scannnerItem.Properties, WIA_SCAN_BRIGHTNESS_PERCENTS, brightnessPercents);
            SetWIAProperty(scannnerItem.Properties, WIA_SCAN_CONTRAST_PERCENTS, contrastPercents);
        }

private static void SetWIAProperty(IProperties properties, object propName, object propValue)
        {
            Property prop = properties.get_Item(ref propName);
            prop.set_Value(ref propValue);
        }

private static void SaveImageToPNGFile(ImageFile image, string fileName)
        {
            ImageProcess imgProcess = new ImageProcess();
            object convertFilter = "Convert";
            string convertFilterID = imgProcess.FilterInfos.get_Item(ref convertFilter).FilterID;
            imgProcess.Filters.Add(convertFilterID, 0);
            SetWIAProperty(imgProcess.Filters[imgProcess.Filters.Count].Properties, "FormatID", WIA.FormatID.wiaFormatPNG);
            image = imgProcess.Apply(image);
            image.SaveFile(fileName);
        }
    }
}

The above example scans a small area of the page in 300 DPI (color mode), saves the result in PNG format and displays it in a picture box. Download the entire example’s source code: playingwiththescanner.zip.

The interface of WIA is really ugly and is not C# friendly, but that’s the life. Enjoy….

Posted by nakov as .net, news, blog at 1:36 AM EET

Comments Off

November 10th, 2009

Security and Administration of Oracle Database - Course @ FMI, Sofia University

I am involved in the teaching and organization of the course “Security and Administration of Oracle Database” which is held for the MSc programs in the Faculty of Mathematics and Informatics (FMI) of Sofia University. The course is part of the MSc program “Information Protection in the Computer Systems and Networks”. The main lecturer in the course Nikolay Manchev is the former Oracle employee with very rich experience with Oracle Database administration and author of a comprehensive book about Oracle Security in 10g and 11g.

Today was our first lecture of the course. The nice thing is that the group of students is small (less than 30 people). The small group of trainees (as it was reported at OpenFest’2009 - see Vasil Kolev’s talk on education at OpenFest) is very important for the effective training. When all people are not much and all of them sit in a relatively small lab, this allows interesting discussions and direct interraction with the lecturers. Most of the students have some experience with databases and SQL so their questions and opinions during the lecture were interesting. I expect very interesting and useful course with exiting experience for both speakers and students.

Posted by nakov as news, blog at 1:01 AM EET

Comments Off

November 9th, 2009

My Talk at OpenFest2009 - Writing Open Source Books with Large Team of Authors

Yesterday I delivered a talk at OpenFest’2009 to share my rich experience in writing free open-source books with a large team authors and editors. Particuarly I presented the methodology, the organizational and management practices I applied during the work on my last successfull project of this kind: the open source book “Introduction to Programming with Java” written in less than 6 months by a team of 30 authors, consisting of 25 chapters and 962 pages. In the presentation and demonstrations I focused on the most important activities and practices that make such projects successfull:

Defining the scope of the book (list of chapters)
Creating a detailed guidelines for the authors specifying the writing style and practices
Defining the templates for each chapter (topic) containing its structure (headings, subheadings and TODOs)
Conducting the team meeting to assigning a chapter to each of the authors
Creating a project plan, assigning tasks and deadlines for each author and tracking the work progress
Managing the authors and working with authors missing their deadlines
Performing a review of the text

My presentation (in Bulgarian) and all demonstration real world examples are available for download:

Visit the Web site of the book Introduction to Programming with Java to see the result of described book writing methodology: www.introprogramming.info.

Posted by nakov as news, blog at 9:33 PM EET

Comments Off

November 8th, 2009

Tesseract OCR Engine - Technical Talk at OpenFest’2009

Today I had a technical talk at OpenFest 2009 along with my colleague Vesko Kolev. We presented the OCR technologies and the open source OCR engine Tesseract.

I demonstrated hot to download and compile tesseract, how it works, when it recognizes text correctly and when it fails and how to train it in a new language. I trained Tesseract for Bulgarian and English. The full presentation is available for download along with the demonstration scripts:

Tesseract-OCR-Engine-v1.01.ppt - the presentation
Tesseract-demonstrations-8-Nov-2009.zip - the training materials (scripts and executables are Windows based but can be adopted for other platforms)

Enjoy.

Posted by nakov as news, blog at 2:33 PM EET

Comments Off

Svetlin Nakov - Blog

Thoughts on Software Engineering

My Projects

Useful Links

NHibernate Lazy Loading BLOB column

New Book: Introduction To Programming With C#

Playing with the Scanner with WIA and C#

Security and Administration of Oracle Database - Course @ FMI, Sofia University

My Talk at OpenFest2009 - Writing Open Source Books with Large Team of Authors

Tesseract OCR Engine - Technical Talk at OpenFest’2009