Connecting ERP and E-Commerce Systems

MASARYKOVA UNIVERZITA FAKULTA}w¡¢£¤¥¦§¨ INFORMATIKY !"#$%&'()+,-./012345

Connecting ERP and e-commerce systems

DIPLOMOVÁ PRÁCA

Ján Segé ˇn

Brno, 2014 Declaration

Hereby I declare, that this paper is my original authorial work, which I have worked out by my own. All sources, references and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source.

Ján Segéˇn

Advisor: Ing. Leonard Walletzký, Ph.D.

ii Acknowledgement

I would like to express my gratitude to Ing. Leonard Walletzký, Ph.D. for his guidance and assistance during the writing of this thesis. Furt- hermore I would like to thank my family, friends, ﬂat mates and my girlfriend for the continuous support and faith they have given me. The ﬁnal thanks goes to my newly acquired angry birds mascot who has supplied me with luck for the duration of writing this thesis and hopefully will continue to do so.

iii Abstract

The goal of this thesis is to analyze and compare the way different online shops store and process information. Find useful similarities and utilize them to implement a tool that enables the open-source ERP system iDempiere to establish a communication link to the electronic stores categorized as compatible, effectively giving iDempiere e-commerce capabilities.

iv Keywords

ERP,iDempiere, e-commerce, e-shop, electronic store, eConnect, plug- in, data connector, import, export, data synchronization

v Obsah

1 Introduction ...... 1 2 ERP Systems ...... 3 2.1 History of ERP ...... 4 2.2 ERP classiﬁcation ...... 6 2.3 Current trends in ERP ...... 7 2.4 Adapting ERP ...... 8 2.5 Open-source ERP ...... 10 3 iDempiere ...... 12 3.1 About iDempiere ...... 12 3.2 History ...... 12 3.3 Working with iDempiere ...... 13 3.4 Development of new features ...... 14 3.5 Community ...... 15 4 E-Commerce ...... 16 4.1 Introduction ...... 16 4.2 History ...... 17 4.3 Online shopping ...... 18 5 Connecting ERP and e-commerce systems ...... 19 5.1 Introduction ...... 19 5.2 Current situation ...... 20 5.2.1 iDempiere export and import functionality . . . 20 5.2.2 E-shop side data connectors ...... 21 5.2.3 Middleware ...... 22 5.3 Summary ...... 24 6 eConnect ...... 25 6.1 Analysis of demands ...... 25 6.2 E-Commerce systems analysis ...... 26 6.2.1 Data architecture similarities ...... 27 6.2.2 Speciﬁc example ...... 27 7 Implementation ...... 33 7.1 Custom tables creation ...... 33 7.2 Custom tables registration ...... 36 7.3 iDempiere plug-in ...... 37 7.3.1 Export processes package ...... 38 7.3.2 Import processes package ...... 40

vi 7.3.3 Utility classes package ...... 42 8 Testing and Deployment ...... 43 8.1 Problems encountered during implementation and testing ...... 43 8.2 Universality of eConnect ...... 44 8.3 Setting up eConnect ...... 47 8.4 Functionality scenarios ...... 47 9 Conclusion ...... 49 10 Electronic attachments ...... 50 11 Literature ...... 51

vii 1 Introduction

Information technology has changed the way we live, how we work and how we conduct our daily affairs. E-mail, instant messaging and voice transfer software enables us to communicate more efficiently and comfortably than ever before. Text and table processors allow us to work with higher amounts of information with the same ease. The internet has provided us with a connection to the whole world. It is also the largest information repository known today. Businesses were also influenced in many ways by the advance- ments in information technology. One of the many impacts it had is the introduction of automated company and business process management, which is nowdays present in the form of an Enterprise resource planning (ERP) software. This system integrates key business components, such as manufacturing, sales, business partner management, human resources, and many more. By correctly implementing and utilizing the ERP software, it can bring many advantages to the company in form of process optimalization, increased data accessa- bility and transparency and ultimately in internal savings. A very important factor that impacts the success of a company is a correctly chosen ERP that optimally fits the company’s business processes. A large scale of different ERP systems is currently available on the market. Companies should devote a reasonable amount of time and resources to analyze what ERP system is the most suitable depending on many implicite factors. One of the factors is the ERP system’s capability to incorporate e- commerce solutions. The modern business slowly moves its sphere of influence to the internet, which is a fresh and evloving market. The popularity of e-commerce applications is rising rapidly, and are becoming a standard part of the modern business. The goal of this diploma thesis is to provide e-commerce connection capabilities to the open-souce ERP system iDempiere. This is achieved by developing a tool called eConnect, which is capable of accepting specific settings that allow it to establish a connection, migrate and synchronize data with variety of today’s electronic stores, thus present a certain level of universality which allows companies to implement e-commerce tools that currently, due to various

1 1. INTRODUCTION reasons, don’t have a connection method available. This document is divided into nine chapters. The first four are devoted to more closely introduce the role of ERP and e-commerce systems, and also acquaint the reader with the history and role of iDempiere in today’s ERP environment. The fifth chapter explains why and how the connection between ERP and e-commerce is established. The last four chapters are devoted to the eConnect tool, its implementation, testing and deployment. Also present is a functional analysis and a final conclusion summarizing the benefits of eConnect.

2 2 ERP Systems

An Enterprise Resource Planning (ERP) system is the name for a type of business process management software that is dedicated to integrate all departments and functions across a company into a single computer system. ERP systems incorporate most or all of the company’s core stages of business [1], which are:

• Product planning, cost and development • Manufacturing • Marketing and sales • Inventory management • Shipping and payment

Each department typically has its own customized information system optimized for the particular way that department works. The ERP system provides the same functionality and apart from that, also incorporates each department subsystem into a central database which allows for improved communication and information sharing between individual departments. This integrated approach can dramatically increase the company’s internal savings if used correctly. It is helpful not only from a financial perspective. It also has high time-saving capabilities which come from increased data visibility and workflow automation between departments. This integrated approach also brings some negatives consisting of increased initial overhead. Hidden costs for training, integration and testing, customization and data conversion are followed by symp- toms like continuous implementation or so called “Post-ERP dep- ression”. New risks to the company are also introduced in form of greatly increased responsibility requirements cast on its employees. A minor fault in one department’s subsystem can now become a major issue for the firm as a whole, since all parts of the company now have access to each other’s data, which can be misinterpreted. Stan- dardization of processes and schooling of employees is crucial. A high

3 2. ERP SYSTEMS level of responsibility and accountability is required from each and every person that comes into direct contact with the system. Application suites that provide broad-ranging integrated functionality and which are used outside the manufacturing market, for example by service providers, hospitals and general business ofﬁces are also marketed as ERP solutions [2].

2.1 History of ERP

The term “ERP” was first introduced by the Gartner Group [3] in the year 1990 as a replacement for its predecessor “MRP”, which means Material Requirements Planning, later Manufacturing Resource Planning. MRP is a software suite that big companies like IBM developed during 1950s to increase transparency and introduce automation within their warehouse and inventory management. Key components of the MRP were bill-of-material processors, inventory management programs and material requirements development programs. The MRP and its functionality is still being used as a core component in today’s ERP systems. After noticing the huge potential of MRP systems, additional functionality such as production and purchasing control, master planning and forecasting, financials and costing were introduced to incorporate a larger portion of the company’s infrastructure. This gave birth to a cross-functional, integrated application suite called the MR- PII, which allowed to consolidate and communicate information throughout the company and its operations [1]. The continued growth of MRP and MRPII systems into an all- comprising bundle of integrated applications which are built around a single relational database management system with analysis tools and an executive information system, also known as a dashboard, subsequently created the basis for today’s ERP systems. These were called the first generation ERP systems. Their popularity grew rapidly also due to the upcoming Y2K problem and the introduction of the euro disrupted legacy systems, which resulted in many companies migrating from their old systems to ERP [4]. First generation ERP systems focused mainly on functionality that did not affect the customer directly – so called Back Office functions.

4 2. ERP SYSTEMS

The increased spread of the World Wide Web introduced the possibility of implementing also customer oriented Front Ofﬁce functionality, such as:

• Supply Chain Management (SCM)

• Customer Relationship Management (CRM)

• Supplier Relationship Management (SRM)

• Business Intelligence (BI)

• E-Business systems (e–commerce, e–government, e–telecom, e–ﬁnance)

• Modern graphical user interface

• Role-based access control

• Easy integration with common business tools

By implementing these capabilities into at that time available ERP systems, they received a new name: ERP II or Extended ERP. This new generation of ERP emphasizes the role of web-based software providing real-time access to the modern ERP system to not only employees but also business partners. These are the ERP systems we recognize today [2].

5 2. ERP SYSTEMS 2.2 ERP classiﬁcation

Selecting the most suitable ERP system for a company is an important process and should underlie a wide analysis of the company’s current and future needs. Most ERP solutions provide similar functionality, features and capabilities varying in depth, complexity, technology, price and applicability to a speciﬁc industry. The most common division of ERP systems is by focusing on the availability of the four key business components [5]:

• Manufacturing

• Logistics

• Finance

• Human resources

According to the amount of the mentioned functionalities the ERP system provides, we can divide it into three groups [5]:

All-in-One

As the name suggests, this type of ERP covers all key business components and its aim is to be universal and ﬂexible. These systems can be applied in a variety of companies with a pleasing result. The disadvantage for some businesses may be that the system components do not provide very deep process functionality. This is mostly a tra- deoff for the universality of the system.

Best-of-Breed

Opposite to All-in-One systems, these ERPs do provide a high level of detailed functionality, but mostly focus only on a narrow spectrum of stages of business. Integrating it with other systems may pose a difﬁculty.

6 2. ERP SYSTEMS

Lite Lite ERP systems are particularly popular by small businesses mainly due to their reduced cost. A small company won’t utilize the huge variety of services or deep process functionality that the two other types mentioned above provide. It mostly has drawbacks as in not containing basic functionality in some areas where it would be expec- ted. Also it may prohibit the company in growth. In these cases, an updated version or a complete new ERP system is acquired by the ﬁrm.

2.3 Current trends in ERP

To follow the latest trends in business and information technology, ERP system providers try to adapt their software to the current situation in the market to make their products even more desirable and competitive. According to the electronic journal Enterprise Apps To- day [6], the below mentioned new trends affect the look of enterprise ERP software today:

Mobile ERP Mobility is a big trend mainly due to the rapid spread of mobile devices and their increased potential of providing enough computing power along with ease of use. The potential of having real-time access to information anywhere in the world is a great opportunity for businesses to quickly embrace mobile ERP, which now can be used not only for dashboards and reports, but also for conducting key business processes.

Cloud ERP The increasing popularity of Software as a Service (SaaS) licensing and delivery model has moved information technology closer to the cloud, where data is stored on remote centralized servers and is available from any device connected to the World Wide Web. ERP systems also follow this trend and are starting to provide cloud and SaaS solutions. Many users have shown signs of reluctance to place

7 2. ERP SYSTEMS their data in the cloud, but the advantages of cloud ERP are becoming more and more apparent and these reservations are gradually disappearing.

Social ERP

Social networks have caused us to rethink how we communicate in life and also in business. Major ERP providers disagree on the topic if and which social media functionality to include into their ERP software. The promise of this effort is to increase the productivity of interaction works. Enterprise Apps Today states that the year 2014 will determine whether social can be a true ERP accessory, providing tangible value to front-line employees and management alike [6].

Two-tier ERP

Efforts were made to create an all-encompassing ERP system that should provide functionality for every aspect of each of the organi- zational systems. This effort failed due to a substantial amount of unsolved issues which brought with them a change in strategy – the adoption of two tiers of ERP. This is an equivalent of running two ERP systems simultaneously. First level consists of the corporate sphere, which enables to provide a global overlook and management of the company as a whole. The second level is division based, where each subsidiary ERP system can execute its own business model, workﬂows, and processes.

2.4 Adapting ERP

Current ERP systems provide a certain level of ﬂexibility to the customer. The two main methods of adapting the ERP software to the company’s bussiness processes are conﬁguration and customization.

8 2. ERP SYSTEMS

Conﬁguration

ERP vendors provide their products not as a complex monolithic1 application, but as modular2 software with a high level of adaptation capability to the customer’s business needs. The modules used in the final system are chosen and respectively configured after preforming a detailed analysis of the target company’s nature of business. This provides an advantage in lower total cost and increased ease of integration of the final suite. However, only configuring the modules may not always be the final answer, since ERP vendors develop their software according to “Best practices”, which are standard metho- dologies used by a wide spectrum of organizations and show better results than other techniques [5].

Customization

Customization is a deeper level of configuration, which is applied when the customer shows interest in custom functionality that is not commonly provided by the standardly available modules, or is not able to be achieved by configuration. Depending on cost and time effectiveness, the customer company can choose to update its current business processes to better suit the proposed standard workflows or to have the ERP vendor implement the required customized functionality into the module on-demand. In extreme cases this can lead to the development of new, unique modules. It has been observed that customizations can make the software more unstable and harder to maintain. Companies tend to falsely assume that a change in the habits of employees is easier than customizing a software. It is difficult and time consuming to get people used to new tools and processes that come with them. Thus in many cases, customization is the better choice [2].

1. Software design model where functionally distinguishable aspects are not ar- chitecturally separate components 2. Software design approach that subdivides a system into smaller parts called modules

9 2. ERP SYSTEMS 2.5 Open-source ERP

Except traditional commercial ERP systems distributed by companies like SAP3, Microsoft4 and Oracle5, the ERP market provides also open-source variants. Before a company decides to adapt an open- source ERP system, it must carefully study its advantages and disadvantages [7].

Advantages

• License cost – Most of the open ERP systems come with one of the open-source licenses6 such as GNU GPL (General Public License) or BSD (Berkley Software Distribution) license, which are free. This substantially lowers the initial cost, which on the other hand shows not to be one of the most important criteria for a company during a process of choosing a right ERP system.

• Vendor independence – The ability to rely on internal teams dedicated to maintain the ERP system, which have a signiﬁ- cantly lower response time and in general are more cost effective than having the maintenance and support handled by the vendor.

• Data control – The company data is maintained and acces- sed only by the employees, which is particularly useful for controlling purposes. System adaptation – The company can modify and adapt the software to its needs dynamically and independently when required. For example when a company expands its business to a new sphere of inﬂuence, there is no need to buy new functionality which may not even be at dis- posal.

3. http://www.sap.com/solutions/business-suite/erp/index.epx 4. http://www.microsoft.com/en-us/dynamics/default.aspx 5. http://www.oracle.com/us/products/applications/ebusiness/index.html 6. Licenses that comply with the Open Source Deﬁnition — allow software to be freely used, modiﬁed, and shared

10 2. ERP SYSTEMS

• Custom ERP – When a company decides to create its own custom ERP system, it can use an already existing fully functional open-source system as the core, and fully focus on developing and implementing their own internal requirements.

Disadvantages

• The best and the brightest – This term is used to describe the effect, where companies dedicate their most experienced IT specialists who have the highest potential of quickly adapting to their new role of implementing, customizing and maintai- ning the ERP system. These employees often are accumulated from other parts of the company, where they are valued for their experience in their current field. The firm has to find suitable replacements for these employees. A different solution would be to create a team of specialists who have previous experience with the chosen ERP system. This is mostly not the most effective choice, since qualified people are hard to find and are expensive [8].

• Uncertainty of results – The functionality, effectiveness and return of investment potential of the open-source ERP system is not guaranteed by any major ERP vendor, nor is there any warranty. The company has to fully rely on the skillset if its own employees.

• Minimum support – A high portion of the value of a commercial ERP solution is made of support services. Big open- source projects have their own active communities which provide only limited support. The ERP development team will have to devote some of its resources to create an internal system support role.

11 3 iDempiere

3.1 About iDempiere iDempiere is one of the open-source ERP suites available on the market today. It is strongly community driven and distributed under the GNU General Public License. Its main application is in small to me- dium businesses as a low-cost ERP system with all the needed functionality a commercial product would provide:

• Customer Relationship Management (CRM) • Supply Chain Management (SCM) • Material requirements planning (MRP)

These and many more stock modules included in iDempiere provide a high level of conﬁguration potential, which allows the user to modify and adapt the ERP system to the company’s processes without the need of changing anything in the source code. The main advantage of iDempiere is its modularity which allows it to be easily extended and customized. This high level of modula- rization is provided by the OSGi (Open Service Gateway initiative) framework1 that iDempiere incorporates.

3.2 History

Although the ﬁrst ofﬁcial alpha version of iDempiere was released in the year 2012, its origin dates back all the way to 1999 when Jorg Janke founded the open-source ERP system called Compiere2. Its popularity grew rapidly over the next few years, becoming one of the most successful projects on the portal SourceForge.net3. The community backing the open-source ERP project was concerned about the

1. http://www.osgi.org 2. http://www.compiere.com 3. http://sourceforge.net - the most successful web-based source code repository for open-source applications

12 3. IDEMPIERE plans of Compiere Inc. utilizing its success and starting to provide its products under a commercial license. An open-source Commu- nity edition was still supported but it had limited functionality. The community refused to cooperate under these circumstances and in the year 2006, created its own truly open-source ERP system called ADempiere4 based on the source code of Compiere. Extensive development was conducted by the community to bring the ERP suite to present standards, which attracted great popularity on the Sour- ceForge.net portal and earned it a place in the top ﬁve most popular software downloads. In the year 2011, developers started a technology test initiative to incorporate the OSGi framework into ADem- piere, which was a success and brought with it the ability to modula- rize the whole ERP suite. This connection was subsequently named iDempiere [9].

3.3 Working with iDempiere

Before a user can start working with the ERP system, an installation process is needed to be executed. This process is documented on the iDempiere wiki pages5, where a detailed list of steps and prerequ- isites that are needed to complete the installation can be found. It is highly recommended to carefully read and follow the manual to ensure a ﬂuent and errorless installation process. After a successful installation, the user is able to interact with the system either by using the standard Java Swing client or via a Java Server Pages (JSP) based web-interface. The Swing client is available directly after installation on the same machine. Connectivity from other desktops or mobile devices is possible by using the JSP client which can be obtained by navigating to the iDempiere server address on the local network. No further installation is necessary. The functionality of both client types is very similar. The most apparent difference is in the graphical user interface, which is more user-oriented within the browser client. Some functions are inacces- sible in one type of the client and vice versa. For example, the Ea-

4. http://www.adempiere.com 5. http://wiki.idempiere.org/en/Windows_Installer

13 3. IDEMPIERE

syImport / ExportCSV6 module is available only in the JSP browser client. This is due to the fact that the ZK framework7 which is used for the web-application part of iDempiere offers somewhat different functionality than the regular Swing library. Needless to say, both clients are capable of providing the full ERP functionality.

3.4 Development of new features iDempiere is available also as a development version, which can be found in the ofﬁcial Bitbucket repository8 and downloaded via a clo- ning process into a local repository of the developer’s computer. The preferred development environment is the Eclipse IDE9 which when conﬁgured correctly will make the iDempiere installation a straightforward process. The installation instructions and list of prerequisi- tes are described in detail on the iDempiere wiki pages10. Upon the initial set-up completion, the developer is provided with the complete and up-to-date iDempiere ERP software in form of an Eclipse project containing dozens of Java packages which re- present individual modules of the system. Server and database connectivity can be established by running the install.app module as an Eclipse application and supplying it with the required settings data. After completing these steps, iDempiere will be ready to be executed, customized and tested.

Using OSGi functionality The OSGi framework which is embedded in iDempiere allows to introduce new functionality to the ERP system in the form of dynamic components called plug-ins. These components can be remotely installed, started, stopped, updated, and uninstalled without the need of rebooting the system. Developing new plug-ins in iDempiere is

6. http://wiki.idempiere.org/en/NF1.0_ImportCSV 7. http://www.zkoss.org 8. https://bitbucket.org/idempiere/idempiere 9. http://www.eclipse.org 10. http://wiki.idempiere.org/en/Installation_in_Eclipse

14 3. IDEMPIERE

a straightforward process and is described in detail in the iDempiere wiki pages11.

3.5 Community

The iDempiere project is backed by a dynamic decentralized community of developers and subject matter specialists that actively contri- bute to the effort of keeping iDempiere a modern and up-to-date ERP suite. Willingness and short response time are a major trait when in need of information on any section of the system. Whether the user needs solving an issue, gain deeper knowledge on the working of a particular module or just general tips and tricks, the community is the right place to visit. The main channels for connecting to the community are:

• Multi-language Project Wiki12

• Support Forum Google Group13

• Commit Announcements Google Group14

• IRC Channel15

11. http://wiki.idempiere.org/en/Developing_Plug-Ins_-_Get_your_Plug- In_running 12. http://wiki.idempiere.org/wiki/Main_Page 13. https://groups.google.com/forum/#!forum/idempiere 14. http://www.globalqss.com/wiki/index.php/IDempiere/Full_Meeting_Minutes 15. http://webchat.freenode.net/?channels=idempiere

15 4 E-Commerce

4.1 Introduction

This topic is best described and explained by deﬁning terms that are closely associated with it. Let us introduce the basic terminology: Commerce – From the Latin word commercium which means trade or exchange. According to Merriam-Webster’s electronic dictionary1, commerce is deﬁned as the exchange or buying and selling of com- modities on a large scale involving transportation from place to place. E-Commerce (or eCommerce) is the short form for Electronic Com- merce, which is essentially traditional commerce utilizing the World Wide Web for conducting one or more of its main components, which range from advertising, buying and selling up to payment transaction and delivery. It can also be described as a market strategy where the company may or may not have a physical presence. According to the way a company has adopted e-commerce, we can distinguish three types of electronic marketing entry strategies used today [10]:

• Bricks-and-clicks – Companies that have both physical and online stores. The most common procedure of adapting this strategy is for an existing ﬁrm to subsequently introduce an online site for e-commerce.

• Click-to-brick – This market entry strategy is carried out by companies, which in the beginning have only an online presence, mostly in the form of an electronic store. Subsequently physical locations are opened to supplement the company’s online efforts.

• Pure-click – The only way this type of company interacts with its customers is via an e-commerce solution, completely omit- ting physical stores. It may own a warehouse for storing and transporting goods, but it does not serve customer interaction.

1. http://www.merriam-webster.com/dictionary/commerce

16 4. E-COMMERCE

A similar term exists for companies which offer only physical stores with no e-commerce involved whatsoever. The term used is Brick- and-mortar.

4.2 History

Very early electronic commerce efforts were made available with the introduction of ARPANET, the predecessor of the Internet and the World Wide Web. It was essentially a non-organized communication between a buyer and a seller subject carried out in an electronic, di- gital form. The goods and payment were exchanged in person. In 1979 the predecessor of online shopping was created by an En- glish inventor called Michael Aldrich2, who connected a television set to a transaction processing computer with a telephone line. This invention has received the name “teleshopping”, which means shopping at a distance. The year 1992 was the year when the ﬁrst commercial sales website was created by the company Book Stacks Unlimited. This was made available by the World Wide Web service introduced two years earlier. The website was registered under the domain www.books.com and provided very basic e-shop functionality such as online products selling and credit card processing. The same year a book was published with the title Future Shop: How New Technologies Will Change the Way We Shop and What We Buy. It has predicted the coming electronic commerce revolution, displayed a vision of future consumer empowerment and created a path- way for new businesses that wanted to get involved in electronic commerce [11]. In 1995, the United States National Science Foundation3 lifted its former strict prohibition of commercial enterprises on the Internet, which caused a tidal wave of companies entering the online market. New and promising business ideas emerged rapidly, which lead to the launching of various and very successful e-commerce products

2. http://www.aldricharchive.com/inventors_story.html 3. http://www.nsf.gov

17 4. E-COMMERCE such as Amazon4 or eBay5. Today, each modern business is not complete without an online presence in the form of at least a simple website informing its customers about the focus of the company and/or the products that it is selling. The more advanced approach is to also incorporate electronic store software to be able to show and sell the products to customers easily and comfortably.

4.3 Online shopping

When we hear the word e-commerce, the first thing that comes to mind is online shopping, or electronic shopping. E-shopping is only a subset of all the available electronic commerce services, however it is the most significant one. It utilizes the Business-to-customer (B2C) model, which allows consumers to directly buy products or services from a vendor over the internet, using web-based applications called electronic stores, web-stores or e-shops. An electronic store application simulates the experience of visi- ting and buying products in a physical store, which should evoke the feeling of reliability and trust in the customer. A positive user experience, intuitive manipulation, attractive and user-friendly design and a straightforward transaction process are the most important factors of an e-shop, which can guarantee satis- fied customers that will return to use this service again, and ultimately attract new clientele from their surroundings.

4. http://www.amazon.com 5. http://www.ebay.com

18 5 Connecting ERP and e-commerce systems

5.1 Introduction

It is important to know that e-commerce tools and ERP suites are two very different systems, which serve different purposes and are aimed for a different group of users. Enterprise resource planning systems are very complex software products and were not intended for public consumption. The early assumption was that the only people who work with the ERP system are highly trained company employees, and the system was designed and built in this fashion. This is beginning to change as customers and suppliers are starting to get eager to also receive access to the information stored in the company’s database, ranging from more detailed product, manufacturer or billing information up to inventory levels and order statuses. Along with this demand comes the need to present the information in a form that the customer will understand and have easy access to. This creates a demand to introduce a new access channel in to the ERP system designated for customers – a business-to-customer model, which is often represented by incorporating an e-commerce tool into the internal business processes of the company. When integrating an electronic commerce solution, companies owning a commercial ERP system have to rely on their vendor who often can provide integration support for e-commerce tools, which are however limited to a certain type and brand of software. Open-source ERP systems can excel in this matter due to their high conﬁguration and extension potential. One of the aspects that should be kept in mind during the ERP and e-commerce integration is that the Internet never sleeps. This means that the e-commerce tool should be fully functional and con- tinuously available to the customers on the internet even when the ERP system is undergoing maintenance or is otherwise not available. While talking about e-commerce integration into ERP, it is important to understand that these two systems should remain two separate entities, but have to retain good communication potential. This comes from the difference in nature of each product. E-commerce is

19 5. CONNECTING ERP ANDE-COMMERCESYSTEMS

a web-based sales and marketing tool, while ERP is an operational execution and ﬁnancial reporting system. Both have their own spe- cial workﬂows and functionality, which are best to be contained within an appropriate group of subject matter specialists, e.g. marketing teams for e-commerce and operation teams for ERP.

5.2 Current situation

Many different approaches exist today on how to connect electronic commerce tools with ERP systems. The most distinguishing factors between them are the way how the two systems interact with each other and on what basis is this communication created. An important factor when choosing the correct communication and synchronization software is the size of your business and the estimated amount of web-transaction in a given timeframe made by the e-commerce tool. The currently available connection methods are described below along with the advantages and disadvantages they bring.

5.2.1 iDempiere export and import functionality The most basic way of extracting data from and loading data into the iDempiere database is to use the integrated very powerful but limited import and export functionality. iDempiere provides two main ways of bulk data export and import.

• Data Import Processes and replication processes1

• EasyImport / ExportCSV module2

These methods are particularly useful for data migration or irre- gular updates. Using one of these approaches would be very ineffec- tive and time consuming if done on a regular basis.

1. http://www.adempiere.com/Data_Import 2. http://wiki.idempiere.org/en/NF1.0_ImportCSV

20 5. CONNECTING ERP ANDE-COMMERCESYSTEMS

5.2.2 E-shop side data connectors Connectors are commercial or open-source applications that provide specific connectivity between two systems. The category of these products which is relevant to this document are the e-shop to ERP connectors. Their purpose is to allow one specific ERP system to communicate with one specific electronic store. The exchange of data is handled by an application layer or an application programming interface (API) either installed in the ERP system or in the electronic store. There are also products available that have both an ERP and an e-shop part, which take longer to install but it is easier to establish a correct communication link between the two parts.

Advantages • If the connector installed is capable enough, and the needs of the company are simple enough, the connection can be set up and deployed in a matter of days.

• Since the connector is designed for one particular electronic store, it can provide a lot more in terms of speciﬁc functionality than other means of connection.

Disadvantages • The speciﬁcity of the connector is also its weak side in terms of universability. If the company decides to deploy multiple e-shops, a new connector must be applied.

• A different connector is needed when the company needs to connect to a different electronic store, than originaly designed.

• Not all e-shops especially the open-source ones, have a connector available. The customer must choose the type of e-shop according to the availability of a suitable connector.

In general, data connectors are suitable for smaller businesses which don’t handle bulk data exchange frequently, and require spe- ciﬁc functionality for their speciﬁc processes.

21 5. CONNECTING ERP ANDE-COMMERCESYSTEMS

5.2.3 Middleware The second approach is to use middleware, most commonly ETL (extract, transform, and load) software. These applications provide functionality that is able to integrate data from multiple types of applications developed by different vendors and/or hosted on separate hardware. It is an analogy to a software translator, who takes information from the ERP system and converts it into a format that the e-commerce tool can process. This method also works in the opposite way.

ETL comprises of three different stages:

• Extract – The first phase is responsible for extraction of data into a single format which later can be used in the transfor- mation process. Typical data sources are relational databases which are present in both ERP and e-commerce systems. Some variants of the ETL software are able to collect data also from different sources like flat files or non-relational databases, however this functionality is not relevant to our initial effort, since both sides of our communication array use the same type of data source.

• Transform – This stage applies a set of rules to the extracted information which can ﬁlter out or even create the ﬁnal form of the data that will later be loaded into the target system. The most common rules applied are: translating or encoding co- ded values, selecting a subset from the data, joining data together to form a bigger part, sorting, aggregation, lookup, and many more.

• Load – The last phase of the ETL workﬂow is importing the translated data into the target database. This can mean inserting, updating or deleting the information, which is chosen depending on the settings of the database or table into which the data is imported.

Most known commercial distributions of ETL middleware are provi-

22 5. CONNECTING ERP ANDE-COMMERCESYSTEMS

ded by companies like Oracle3, IBM4 and Microsoft5. Open-source and non-commercial versions include popular software like Talend6, ETL-Tols7 and Jitterbit8.

Advantages • Middleware software is available as a commercial product backed by experienced vendor companies, and also has easily ob- tainable open-source instances • ETL can be run as an application on a local computer or network, or as a middleware service located on remote servers and communicate via Internet • Fluent data exchange once the conﬁguration process is done correctly • High performance even as remote services • Lower programming effort is needed, since the middleware application handles a reasonably high fraction of the communication automatically

Disadvantages • Increased maintenance due to involving a third party to the communication process • Difﬁcult and time consuming process of the initial conﬁgura- tion • Possible operational problems can occur when the software is not properly set-up or designed due to its complexity

3. http://www.oracle.com/us/products/middleware/data- integration/enterprise-edition/overview/index.html 4. http://www.ibm.com/developerworks/data/library/techarticle/dm- 0411simchuk 5. http://technet.microsoft.com/en-us/library/ms141026.aspx 6. http://www.talend.com/resource/free-etl.html 7. http://etl-tools.info 8. http://www.jitterbit.com/solutions/etl-data-integration

23 5. CONNECTING ERP ANDE-COMMERCESYSTEMS 5.3 Summary

There are many currently available ways to integrate electronic stores into their ERP systems. Companies can choose between commercial or open-source, ETL or data connectors. Each approach is very speci- ﬁc and has its own set advantages and disadvantages. One common disadvantage is the somewhat low level of universality the currently available tools and applications provide. This is why I have decided to develop eConnect, which brings a slightly different approach to this topic.

24 6 eConnect

The main goal of this diploma thesis is to design and implement extended functionality for iDempiere which will allow companies running the ERP system to actively utilize e-commerce solutions in the form of electronic stores for their business activities. A substantial focus is placed on the aspect of universality of the developed tool. eConnect is a tool that provides bi-directional communication capability between iDempiere and a variety of electronic stores. The main difference from other products that offer similar functionality is a level of e-shop platform independence. This is achieved by applying different data exchange rules to different e-shop connections. This ability is however not unlimited and relies on the target software’s functional capabilities. What requirements must the electronic store meet to be able to establish a full communication potential will be described in the later parts of this document. eConnect is realized as a plug-in for iDempiere. This form of implementation was chosen to comply with one of the basic requirements, which is ease of use. No necessity was present to involve an external third party tool, which would operate as a part of the e-shop software or as a standalone application. The company’s employees should be used to work with iDempiere on a regular basis, and an introduction to a different tool would be time consuming and contra productive. The main functional aspects of eConnect are initial data population and subsequent bi-directional data synchronization, which can be done on a regular basis.

6.1 Analysis of demands

For a client company to be able to use eConnect as an efﬁcient and universal connection and communication tool with e-commerce software, it has to incorporate functionality that will effectively allow iDempiere to manage most of the transaction aspects of the e-shop. The mentioned functionality consists of:

25 6. ECONNECT

• Synchronizing orders from the e-shop to iDempiere

• Synchronizing payments from the e-shop to iDempiere

• Synchronizing order updates from iDempiere to the e-shop

If the client company wants to use the services of an open-source e- shop for which no speciﬁc connector module is available and middleware solutions are not in its best interests, eConnect should provide a certain level of universality. This means that iDempiere should be capable of connecting and communicating with a range of e-shops by applying the correct connection and data exchange settings. In the case that the client company is starting to apply the Brick- to-click model and is ready to begin using one of the supported electronic stores, eConnect should provide functionality that allows bulk exporting of data into the e-shop. This would dramatically decrease the initial set-up time and also decrease the potential of introducing errors by manually populating the newly installed e-shop. The exported data should include all the information that is needed to start using the electronic store immediately, which are: Pro- ducts, Categories, Manufacturers, Countries and Currencies. Additi- onal settings such as sore information, modules and templates need to be set-up manually.

6.2 E-Commerce systems analysis

Connectors and middleware typically use procedures bound to only one pre-deﬁned target system. In the case of eConnect, the built-in functionality should allow communication with different electronic stores without the need of creating a new instance of eConnect for each connection. Various brands, types and in some cases even versions of e-shops are based on a different data architecture. This creates a complex problem as how to handle and mitigate these differences. The solution to this question is to focus on the similarities between the data architectures of iDempiere and the electronic stores, and utilize their potential.

26 6. ECONNECT

6.2.1 Data architecture similarities The main function of electronic stores is the capability to conduct and keep track of business transactions. The most crucial parts of the mentioned processes range from storing and presenting products to creating orders and invoices. This functionality must be present in every e-shop software, otherwise the application would not meet the requirements for a fully operational electronic store. On a data level, each of these processes is represented by a single or a set of physical tables in the database. Due to the presence of a high degree of standardization in this sector, attributes and variables used by these procedures are very similar and in the case of core elements, identical. ERP suites and in particular iDempiere provide the same functionality as mentioned above, but in the form of back-ofﬁce1 processes. Even when the purpose of the data processed in an ERP and an e-commerce system is fairly different, the data structure remains basically the same. These similarities in the data structure of both systems are the basic building blocks of the universality of eConnect.

6.2.2 Specific example To demonstrate the similarities in the data storage architectures of the ERP and e-shop systems, one particular example was chosen to describe how data can be exchanged between databases. The three diagrams in the picture 6.1 show how information about product categories is stored in the database of electronic stores os- Commerce and OpenCart, and in the ERP system iDempiere. Pro- duct categories are virtual aggregation mechanisms that relate products to each other into a group according to specific product or marketing properties. Each category has its own set of attributes that defines it within the system. The attributes of each table were re-arranged so that the similarities are made even more visible.

1. A part of most corporations where tasks dedicated to running the company itself take place. The opposite of front-ofﬁce, which incorporates sales and other customer-facing services

27 6. ECONNECT

Obr. 6.1: Diagram showing how the three systems store data related to product categories

This document will be representing the parent system, table and ﬁeld information in the standard SQL notation, which looks like this:

Similarities The tables oscommerce.categories and oscommerce.categories_description, opencart.category and opencart.category_description and the single idempiere.m_product_category have the same purposes in their respective systems and fully share these ﬁve attributes:

• Category identifier field, which is also the primary key of each of the tables. It defines a unique identification number for each category entry. idempiere.m_product_category.m_product_catego- ry_id corresponds to opencart.category.category_id and to oscom-

28 6. ECONNECT

merce.categories.categories_id. All the fields are represented as numeric or integer values. • Parent identifier field. This attribute points to the superior category and allows the categories to be represented as a tree structure. idempiere.m_product_category.parent_id corresponds to opencart.category.parent_id and to oscommerce.categories.parent_id. This is a very good example of a standard naming convention. All fields share the same attribute type, which is numeric or an integer. • Date when the category was created e.g. inserted into the database. idempiere.m_product_category.created corresponds to opencart.category.date_added and to oscommerce.categories.date_added. The data type is a timestamp, which is a sequence of characters containing a representation of the given date and time. The database engines are able to parse and interpret a high amount of different timestamp types, ranging from strings mar- king date and time values up to the numeric UNIX time2 representation. • Date when the category was last updated e.g. one or more attributes of the specific entry were changed. idempiere.m_product- _category.updated corresponds to opencart.category.date_modified and to oscommerce.categories.last_modified. The data type of all the columns is again a timestamp. • Name of the category. In other words, the text that will be associated and displayed together with the category reference. idempiere.m_product_category.name corresponds to opencart.cate- gory_description.name and to oscommerce.categories_description- .categories_name. All columns have the data type: variable character, which is essentially a string with a fixed maximal length. This may create consistency issues with tables where the imported string exceeds the given maximal length value. Also, to be able to effectively reach the data, the category and description tables have to be joined together using the category identifier column.

2. The number of seconds elapsed from the 1st of January 1970

29 6. ECONNECT

These are the ﬁve attributes that compose the minimal set of information that is needed to create a basic category representation in the osCommerce e-shop. These however are not sufﬁcient for OpenCart, which also requires the attribute status to be present.

Additional linkable attributes for OpenCart

• Description of the category e.g. additional information associated with the category that will be displayed to the user under certain circumstances. idempiere.m_product_category.description correlates to opencart.category_description.description. The same rules and possible issues apply as for the category name column.

• Availability status. This attribute is represented as a Boolean value where true indicates that the category is available and visible for the user, and false means the exact opposite. idempiere.m_product_category.isactive corresponds to opencart.category.status. The reason why the status attribute must be present is that when a new category is inserted and the status field remains empty, OpenCart’s database will push a pre-set default value into this field, which is 0, meaning false. This category would be present in the database, but would not be visible in the e-shop. The administrator would need to manually adjust this attribute through the administration interface. Another difficulty exists with the different interpretations of Boolean values in OpenCart and iDempiere. iDempiere values for true and false are characters ‘Y’ and ‘N’ while OpenCart uses numeric values 1 and 0. A basic translator process needs to be present to make these two representations compatible.

• Search keywords. These columns are dedicated to contain me- tadata which is used by the internal e-shops search functionality and also for search engine indexing. opencart.category_des- cription.meta_keywords and opencart.category_description.meta_- description both correspond to the column idempiere.m_product- _category.value which is used by iDempiere as the primary search value.

30 6. ECONNECT

Default values

There is one more attribute common to both osCommerce and Open- Cart which is not present in the iDempiere category table and thus will remain empty. In both cases the name of the mentioned attribute is sort_order. This attribute represents the order in which the categories will be displayed in the e-shop category tree, and is equivalent to the iDempiere attribute sequence which is not available for the idempiere.m_product_category table. In this case, the electronic stores database engine is configured to insert a default value into this field. Although this value will not serve the purpose of sorting the categories in the tree representation, but most of the electronic stores are programmed well enough to cope with this issue and use their default method of sorting, which is alphabetical sorting, or sorting by the date the category entry was added. If the e-shop administrator needs to change the sort order of the categories, it is made possible via manual configuration in the administration interface. Since the amount of categories is usually significantly smaller than the amount of products, the process is not overly time consuming. When it comes to attributes which do not have the corresponding value available, the possibility of inserting a common default value or the value of another column will not be beneficial in this particular case, but it can be useful in the case of OpenCart’s additional column from the table category_description, which is language_id. E-commerce solutions often provide multilingual support which is represented in the database by a specific language identifier located in a table dedicated to store different language types. To be able to choose the correct category name and description, the language_id must be set to an appropriate value corresponding to the required language. Since iDempiere does not support multiple languages for category entries, the correct solution would be to insert a default value which selects the correct language automatically for all categories.

Unused attributes

Upon populating all the needed attributes of the electronic stores, the idempiere.m_product_category table still has several columns that

31 6. ECONNECT are unused. These columns are not useful for the e-shop to function properly and thus do not need to be exported. In the case of an import process, the attributes will not be able to be populated by the data from the electronic store since it does not possess them. The use of a variable default value would be beneﬁcial, and in some cases mandatory.

Analysis results This example was chosen to illustrate the principle of connecting information throughout the systems due to the high similarity of the data tables and their attributes. Such a high commonality level is however not present in all cases. Product and order data for example are a lot more complex. Also different e-shops can have very compli- cated category representations. After analyzing the commonalities within the data storage architectures of numerous electronic stores, the resulting information was compared to iDempiere and the way it stores its data. As a result, a connection method was designed to establish a bi-directional ERP and e-shop communication based on data exchange settings created for speciﬁc system pairs. The data exchange settings consist of:

• E-shop and database parameters

• Appropriate communication data types

• Link between corresponding tables and columns

• Default values for cases when one side is not able to provide the needed data

• Primary key identiﬁers which allow entries to be u pdated

32 7 Implementation

The implementation process of eConnect can be divided into these four steps:

• Creation and custom tables in iDempiere which allow to store connection and data exchange settings

• Registration of custom tables into the Application Dictionary which allows the user to manage the data stored in the custom tables via the iDempiere client

• Introduction of a new plugin to iDempiere using the OSGi framework, which houses all the needed functionality

• Registration of processes into the Application Dictionary which allows the user to execute the processes via the iDempiere client

7.1 Custom tables creation

To be able to store settings information in iDempiere, a set of new tables have to be created to house this data. To distinguish the custom from built-in tables in the iDempiere database, the preﬁx cz_econnect_ was chosen to match with the plug-in package name.

The information that we want to store in our custom tables is:

• Global e-shop identiﬁer: This table stores the names of electronic stores switch the user is able to choose from when using the import, export and synchronization processes. It also holds an identiﬁer linking the e-shop to its database connection settings. This data is represented by the table cz_econnect_eshop.

33 7. IMPLEMENTATION

• Database connection settings: Each electronic store has its own database that it needs to connect to prior to be able to mani- pulate any data that is or will be stored in it. The name of the table which stores this information is cz_econnect_database. Except the database setting identiﬁcation name, this table houses the basic connection data such as:

– User login credentials: The user name and password that is used to authenticate and authorize the user in the database engine. – Database name: A database is essentially a set of tables which may or may not be in relation between each other. A database engine can have multiple databases. We need to pick the name of the correct database which contains the tables of the e-shop we want to connect to. – Database engine U.R.L: The Uniform Resource Locator, which is basically a web address that points to the place and port on which the database engine listens to inco- ming trafﬁc. – Database driver: There are many different database engine architectures which have their own set of connection rules and settings. For the Java program to be able to communicate with the database engine, a driver must be selected so that the application knows if it should connect to a MySQL, PostgreSQL, MSSQL or any other type of database server. This data is sufﬁcient to let Java establish a connection to the required database server.

• Data exchange settings: This set of data is responsible for let- ting the eConnect processes know which information to work with and where to ﬁnd it in the foreign (non-iDempiere) database. For better visibility, two tables are used to store the required data: cz_econnect_settings and cz_econnect_datatype. The data exchange settings consist of:

– E-shop identiﬁer: Each rule entry has to be bound to a spe- ciﬁc e-shop instance. This column is referenced from the table cz_econnect_eshop.

34 7. IMPLEMENTATION

– Data type: This column is used as a filter attribute for let- ting eConnect know to which process the rule entry is bound to. It is referenced from the table cz_econnect_data- type and stores implicit information such as the identifier and name of category, product or order data types. – E-shop table name: The name of the table located in the database of the electronic store, where the required information is kept. – E-shop column name: Similar to the above attribute, but defines the location of the information within the given table. – iDempiere column name: This attribute defines the posi- tion of the required information within a specific iDem- piere table. The name of the table is not needed, since eConnect derives it automatically from the data type attribute. – Primary key selector: States whether the given rule entry is the primary key of the foreign table. This is needed to enable eConnect to perform update operations. – Default value: The value that is inserted into the previously chosen target column instead of missing input information.

Except the custom attributes mentioned above, each iDempiere table must contain the iDempiere default columns, which are:

• ad_client_id - Client Identiﬁer

• ad_org_id - Organization Identiﬁer

• isactive - Flag to indicate whether the record is active

• created - Time when the record was created

• createdby - ID of the user who created the record

• updated - Time when the record was last updated

35 7. IMPLEMENTATION

• updatedby - ID of the user who last updated the record

These columns are necessary for the iDempiere Application Dicti- onary1 to recognize the custom tables and provide important functionality such as creating a new window for the tables where their content can be managed via the iDempiere client.

7.2 Custom tables registration

To enable the user to insert, change or delete data in the custom tables directly from the iDempiere client, the user must log into the ERP system as the system administrator and follow these steps:

• Register the tables in the iDempiere Application Dictionary. This is done by creating a new record for each of the tables in the menu Application Dictionary -> Table and Column and filling in the required fields, especially the table names which figure in the iDempiere database.

• Register the table columns. To do this, run the process Create columns from DB which is located in the Table and Column window under the menu item Process. This will automatically register columns to the table entry in the Application Dictionary. It is recommended to check if the process created the columns correctly, by browsing through them in the Column tab. Af- ter completing these two steps for each of the custom tables, iDempiere will now recognize the new tables and enable to create separate windows where the user can manage the con- tents of the tables. This is achieved by:

• Creating a new window for each of the tables. This is done by navigating to the menu Application Dictionary -> Window, Tab, & Field and creating a new record for all four custom tables.

1. The AD Engine is the base component of iDempiere, which allows most of its application to be conﬁgured directly in the dictionary without requiring compila- tion or re-building

36 7. IMPLEMENTATION

• After ﬁlling in the required window information, navigate to the Tab window, create a new record and again ﬁll in the required parameters along with the newly registered table instances.

• By clicking the button Create Fields, iDempiere will automatically populate the window with columns from the selected table.

After repeating these steps for each of the custom tables, iDempiere will enable their data management in a new dedicated user interface window. A more detailed guide on how to register tables can be found in the iDempiere wiki pages [9] or in the ADempiere 3.6 Cookbook [12].

7.3 iDempiere plug-in eConnect was designed as a plug-in module for iDempiere due to the advantages this form of implementation brings. Instead of using extension points, eConnect utilizes the newly available component deﬁnitions and IProcessFacroty services. The functionality components of eConnect are implemented in the bundle cz.segen.econnect and divided among three packages according to the general purpose of each individual functionality aspect:

• cz.segen.econnect.exports – Migrating data from iDempiere to electronic stores

• cz.segen.econnect.imports – Synchronizing data from electronic stores to iDempiere

• cz.segen.econnect.utils – General methods used by import and export processes

37 7. IMPLEMENTATION

7.3.1 Export processes package Every export class starts with the preﬁx EExport and migrates data from iDempiere to the designed electronic store. The type of data exported is shown by the sufﬁx of the class name, which is represented by one of the main data types: Product, Category, Manufacturer, Country or Currency. A global export is also present which migrates all the available data types in one continuous process. This is particularly useful when populating a new empty instance of an e-shop. Since the functionality of each export class follows the same princip- les, it will be closely described as a global entity.

Classes EExport The EExport classes extend the org.compiere.process.SvrProcess class in order to be able to execute their functionality as an iDempiere pro- cess2. They inherit two abstract methods which require implementation: prepare() and doIt().

Method prepare() - This method is allows reading and storing of parameters passed to it by the iDempiere process which is executed using the iDempiere client. Each export class may have a different amount of parameters depending on its speciﬁc functionality. The received parameters are stored as pre-deﬁned private attributes in the export class instance.

Method doIt() - As the name of the method may indicate, its purpose is to execute the functionality it contains after being called by the iDempiere process. In the case of the export classes, this method executes a series of individual tasks which can be divided into these main groups:

• Validating the received input parameters – The ﬁrst part of this subroutine validates the parameters received and stored by the prepare() method. It looks for incorrect or non-present values. When found, the whole doIt() method is stopped and

2. http://wiki.idempiere.org/en/Developing_Plug-Ins_-_Process

38 7. IMPLEMENTATION

an appropriate warning message is returned to the user interface.

• Fetching the correct e-shop, database connection and data exchange settings – This step calls three methods located in the static utility class cz.segen.econnect.utils.ModelMethods:

– getEShopSettingsByEShopID(...) – getEShopDatabaseConnectionSettingsByDatabaseID(...) – getEShopExportSettingsByType(...)

These methods return the required settings parameters which are subsequently stored in appropriate variables.

• Creating parameterized SQL queries corresponding to the selected data exchange settings – The third step uses the EE- xportStd class to parse the data exchange settings received in the previous step. After that, export rules contained in the settings are applied into an array of exportSQLBuilder objects, which are instances of the class cz.segen.econnect.utils.SQLBui- lder.

• Fetching the required information from iDempere and populating the parameterized SQL queries with the correct data – After creating and populating the SQLBuilder instances, the main process launches an appropriate method that fetches the required data from the iDempiere database, which is one of the getData(...) methods from the Mo- delMethods class. The SQLBuilder objects are then populated with the recovered data.

• Establishing connection to the selected e-shop database – In this next step, the process receives and stores the e-shop database connection from the method ModelMethods.getDBConnec- tion(...) by providing it with the database connection settings.

• Bulk execution of the ﬁnal SQL queries – After the connection to the electronic store database is established, the process checks if the object is already present in the target table by

39 7. IMPLEMENTATION

using the method ModelMethods.databaseContainsRow(...). Af- ter evaluating the presence of the object, corresponding update or insert SQL statements are created and fed into a batch sta- tement array, which in the end is executed. • Log query execution statistics – Finally, the process logs the amount of update and insert statements executed. This information is then logged by using the inherited addLog(...) method and subsequently displayed to the user in the iDempiere UI.

After the operations have been executed, the method returns a string object which is then displayed in the process result window in iDem- piere. The outcome of this process can be seen in the database of the electronic store, which is now populated with the most recent data from iDempiere.

7.3.2 Import processes package The same naming conventions as used in the export processes are applied to the Import classes. The preﬁx is EImport and the sufﬁx is one of the available import types: Order or Payment. The import processes use a similar approach as the export processes.

Classes EImport Same as the EExport classes, the EImport classes also extend the org.compiere.process.SvrProcess and inherit two abstract methods: prepare() and doIt(). There are only two differences in functionality between EExport and EImport classes:

• Parameterized SQL queries are populated with the information received from the electronic store • The data is inserted into native iDempiere import tables

After the import process is successfully executed, the respective iDempiere native import table is populated and the data is ready to

40 7. IMPLEMENTATION be inserted into the corresponding working table. This is the ﬁrst and automated phase of the import process.

Import into working table The second phase of the import process is completed by using the standard iDempiere import functionality, which can be found in the iDempiere client under the menu System admin -> Data -> Data Import -> Import Order or Import Payment. After opening one of the Import Order or Import Payment windows, it will already contain the data from the respective native import table. The user now needs to run the iDempiere Import Process and correct any issues found during validation. When completing these two import phases, the required data from the electronic store will be present in iDempiere and will be available for further internal processing.

Registering processes To enable the user to execute any of the previously implemented processes, they must ﬁrst be registered in the iDempiere Application Dictionary. This is achieved by following these steps:

• Logging into iDempiere with the System Administrator role.

• Navigating to the menu Application Dictionary -> Report & Pro- cess and creating a new record.

• Filling in the required ﬁelds, particularly the Classname column, where the full class name of the appropriate process class should be inserted.

• Adding the required amount of parameters in the Parameter tab and ﬁlling in the needed data.

• Repeating this procedure for each of the import and export processes that are required.

41 7. IMPLEMENTATION

7.3.3 Utility classes package The cz.segen.econnect.utils package contains two classes which provide support functionality for the import and export processes: Mo- delMethods and SQLBuilder.

Class ModelMethods This static class comprises of methods which are dedicated to database communication only. Their functionality ranges from returning the target database connection object to providing result sets of various queries used in the import and export processes.

Class SQLBuilder This class enables to create SQLBuilder objects which are capable of storing detailed query information and provides SQL query creation methods. The most important methods from this class are:

• Method getInsertSQL() – Constructs and returns a complete SQL query which upon execution inserts the stored parameters into the target database.

• Method getUpdateSQL() – Similar to the above method, but the returned SQL query is type UPDATE.

• Method populateParams(...) – Populates the parameterized SQL query with values from a given set of results. It also contains validation methods and simple translator functionality which makes sure that the correct type and value of the parameter is added. This method must be called before the getInsertSQL() or getUpdateSQL() methods, otherwise they would return a parameterized query which would insert only the column names instead of values into the database.

42 8 Testing and Deployment

8.1 Problems encountered during implementation and testing

Numerous difﬁculties were encountered during the implementation and testing phases. The resolution was not always easy and straightforward. In some cases, a workaround had to be implemented. The issues encountered and provided ﬁxes are described below.

Attribute count mismatch This issue was observed when synchronizing orders from OpenCart into iDempiere. The customer name in OpenCart is stored in two columns: ﬁrstname and lastname, while iDempiere has only one column name for the customer. The result was that only the ﬁrst name was synchronized within the order. To solution to this situation was to implement an additional condition into the method which parses data exchange settings, which concatenates the two parameters under these conditions:

• The parameter type is string or a variable character set

• Settings contain entries which have the same target column name but different source column names

This resolves not only the single case of Name and Surname but multiple other possible cases.

Not supported ON DUPLICATE KEY UPDATE clause in PostgreSQL PostgreSQL does not support the use of the ON DUPLICATE KEY UPDATE clause which other database engines do. The purpose of this clause is to make the database engine check the existence of an object during an insert SQL query and when the object exists, update the entry instead of throwing an exception.

43 8. TESTINGAND DEPLOYMENT

It is possible to simulate this behavior by adding conditions to the SQL, but this would not be practical for our automatically generated queries. The solution was to create a method which ﬁrst checks if the given entry is present in the target database. According to the result, either an INSERT or an UPDATE query is generated.

Scale parameter not defined correctly in iDempiere database If an attribute of a table is of the type NUMERIC, it can be represented in Java either as an integer or a floating point number. The standard way to distinguish between these two types would be to check for the Scale parameter of the numeric attribute to see if and how many numbers it should contain after the decimal point. In the iDempiere database, floating point numbers do not have the scale attribute defined at all. This causes the function that returns the scale value to return 0 in the case that it is set to 0 and also when it is not defined. This means that it is not possible to distinguish integers from floating point numbers by using the scale value. The workaround for this issue is not to check for the scale value, which would be the correct approach, but to check if the scale parameter is defined. This approach is not correct, but it returns the needed results in all the tested cases.

Data too long exception This exception is thrown by the database engine when the imported text data exceeds the maximum character limit of the target column. This issue occurred during the testing of currency export from iDem- piere to OpenCart. The currency description in OpenCart is set to a signiﬁcantly shorter maximum value than in iDempiere. The solution to this issue is to alter the mentioned e-shop table to accept longer column values when this type of exception is thrown.

8.2 Universality of eConnect

The data exchange settings approach of eConnect is able to establish communication between different electronic store databases, not

44 8. TESTINGAND DEPLOYMENT only the ones mentioned below. To be able to communicate with the target system, the database of the e-shop must ﬁt into certain data storage complexity requirements.

Tested e-shop software:

• OpenCart 1.4.9.3

• osCommerce 2.3.3.4

• VirtueMart 2.0.2.4

• Woocommerce 2.1.8

• Drupal 7.28

• Magento 1.9.0.0

Results

According of complexity of the tested electronic stores, they can be divided into three groups:

• Symplistic - The stores that fall into this category are Virtu- eMart and Woocommerce. Both are designed as plug-ins for a content management system (CMS). The database design of these two stores is very symplistic in order to be able to work together with the Joomla (VirtueMart) or Wordpress (Woocom- merce) CMS platform. Since iDempiere requires certain data to be available when performing data import or migration, such as country or currency identificators, none of the mentioned CMS plug-in based electronic stores provide this functionality and thus is not suported by the communication form which eConnect provides. It is however possible to artificially recreate the required missing data by assigning it a default value in the data exchange settings, but this would not be a flexible and thus suitable solution.

45 8. TESTINGAND DEPLOYMENT

• Complex - Electronic stores such as Drupal and Magento are assigned to the category of complex e-shop systems due to their high complexity of database architecture. This is due to the fact that both stores provide functionality that is beyond the scope of basic e-shop software, such as internal analytics, marketing, promotions and customization tools. In other e- commerce systems, these features are made available mostly by installing an external module which does not affect the already existing database structure. In the case of Magento and Drupal this functionality is already built-in, which has a dra- matic effect on the data tables.

As seen in the similarities example in the implementation chapter of this document, iDempiere uses one table to accomodate information regarding product categories. OpenCart and os- Commerce use two due to their localization capability. In the case of Magento, there are fourteen tables needed to deﬁne the complete product category data.

Magento and Drupal are categorized as too complex electronic stores to be able to correctly communicate via the eCon- nect data exchange format. However, it is possible to establish a communication link by using the data exchange settings approach, but this solution would be too complex and error prone to be regarded as useful.

• Suitable - OpenCart and osCommerce have the most suitable architecture which allows eConnect to utilize their similarity aspects to the maximum. The presence of relatively simple entity relations loweres the complexity of the system as a whole. On the other side, basic functionality, such as the availability of localization customization, adds required means to store data required by iDempiere.

OpenCart version 1.4.9.3 and osCommerce 2.3.3.4 were successfully tested and provide the full functionality potential for eConnect’s data exchange approach.

46 8. TESTINGAND DEPLOYMENT 8.3 Setting up eConnect

To be able to establish a connection between the two desired systems, connection and data exchange settings have to be set or imported correctly. This diploma thesis provides connection settings to two electronic stores that can be imported using the EasyImport functionality of the iDempiere web-client.

8.4 Functionality scenarios

After the connection and data exchange settings have been successfully imported, the eConnect application is ready to be deployed and utilized in one of the below scenarios.

Initial electronic store population To help speed up the process of setting up and populating a newly installed e-shop software, eConnect provides a process which automatically migrates the required data from iDempiere to the target e-shop. This is done by following these steps:

• Running the Populate Eshop process

• Choose one of the installed e-shop connection settings

• Pick the correct price list. This will impact the prices of the imported products

• Start the process

It is recommended to use a blank copy of the chosen electronic store, since many stores come pre-installed with sample data. This however will not create any collisions since eConnect will automatically update any information that does not correspond to the data stored in the iDempiere database. By starting the process, eConnect will execute all the needed subp- rocesses to import the initial data into the chosen e-shop. After the population process stops, statistical data will be displayed which shows the amount and type of operations conducted.

47 8. TESTINGAND DEPLOYMENT

The result is a fully populated e-shop database with objects which in most cases are initialized directly by running the electronic store software.

Data maintenance Most of the data maintenance is done via iDempiere which minima- lizes the interaction needed with the electronic store. If a product is changed or a new one is added, simply run the correct synchronization process which will import or update the new product in the e-shop database. The changes will be seen immediately by refreshing the store application. Synchronization processes are available for many other data types, such as categories, countries, orders or payments.

Order and payment synchronization Order and payment synchronization work similarly to the data maintenance processes. By running the order or payment synchronization tool, eConnect checks for new orders or payments in the electronic store and according to this information imports the new data into the respective iDempiere import table. After this is done, the user only needs to start the standard import and validation process which is described in the previous part of this document.

48 9 Conclusion

In the recent years, e-commerce systems are slowly but surely ma- king their way into close integration with ERP. A customer oriented firm is generally more successful than a profit oriented firm, and the business is well aware of that. This is why many companies started to deploy the Business-to-Customer strategy, and those who haven’t will most probably follow. There are many solutions on the market which provide electronic commerce and ERP connection capabilities. Each has its own list of advantages and disadvantages. Choosing the most suitable e-commerce system for a company is a similarly difficult process as the selection of the perfect ERP system. It depends on dozens of factors that need to be taken into account, and no solution is the best for everyone since each company is so very different from the other. The software which was developed as a result of this diploma thesis is not meant to be a breakthrough in its field. It is meant to provide an alternative solution for smaller companies that do not want or don’t have the means to implement one of the available commercial solutions. It also gives an opportunity to try out less known electronic store platforms which in other cases would not be supported by the popular ERP to e-commerce connection tools. As an open-source software, eConnect provides an opportunity for developers to adopt its concept and allows them to further extend it by adding their own ideas and experiences, which may one time result in a truly universal and popular solution to the business needs of many companies.

49 10 Electronic attachments

This thesis also includes electronic attachments that are uploaded into the MU IS. They consist of:

• diploma_thesis_jsegen.pdf - the electronic version of this document in PDF format

• econnect.zip - the source code of eConnect as a java package

• settings.zip - ZIP archive containing connection and data exchange settings for OpenCart and osCommerce e-shops

50 11 Literature

[1] BASL, Josef. Podnikové informaˇcní systémy: podnik v infor- maˇcníspoleˇcnosti.2. vyd. Praha : Grada, 2008. 283 s. ISBN 978- 80-247-2279-5.

[2] WAILGUM, Thomas. ERP Deﬁnition and Solutions [online]. Last modiﬁed on April 17, 2008. [cit. May 2014]. Available from World Wide Web: .

[3] Gartner Group. A Vision of Next Generation MRP II: Scenario S-300-339, April 12, 1990

[4] Thin Enterprise Resource Planning (Second ed.). Boston: Thom- son Course Technology. 2006. ISBN 0-619-21663-8.

[5] Krnáˇc,Igor. Adaptácia ERP systému ADempiere. Brno: bachelor thesis FI MU, 2011.

[6] DREW, Robb. Top 8 ERP Trends for 2014 [online]. Last mo- diﬁed on December 20, 2013. [cit. May 2014]. Available from World Wide Web: .

[7] GRUMANN, Galen. Is Open Source The Answer to ERP? [online]. Last modiﬁed on February 15, 2007. [cit. May 2014]. Available from World Wide Web: .

[8] CHEN, Injazz. Planning for ERP systems: analysis and future trends. Business Process Management Journal, Vol. 7 No. 5, 2011, pp. 374-86.

[9] ADempiere ERP Wiki pages [online]. Created 2006. [cit. May 2014]. Available from World Wide Web: .

51 11. LITERATURE

[10] CLARK, Patrick. Click-to-Brick: Why Online Retailers Want Stores in Real Life [online]. Last modiﬁed on July 10, 2013. [cit. May 2014]. Available from World Wide Web: .

[11] SNIDER, J.H. and ZYPORIN, Terra. Future Shop: How New Technologies Will Change The Way We Shop and What We Buy. 2008. ISBN 978-0-59550-363-6

[12] KUMAR, Ajit. ADempiere 3.6 Cookbook. Birmingham: Packt Publishing, 2011. 316 s. ISBN 978-1-84951-338-8