Baobaz Blog

"Magento DataFlow - Data Exchange Made Flexible" article introduced global concept of data exchange framework implemented in Magento. Today I would like to tell more about default adapters implemented in DataFlow module.

1. Adapter definition

   Adapters are responsible for pluging into an external data resource and fetching requested data or saving given data into data resource. For this purpose all adapters implement interface Mage_Dataflow_Model_Convert_Adapter_Interface which contains two methods: load() and save(). Data exchange concept introduced in DataFlow module use adapters in 3 contexts:

   • to load data from resource - using load() method
   • to save data to resource - using save() method
   • to process one parsed row - when defined as adapter/method pair of variables of parser

   For first two contexts adapter's xml definition looks like that:

   <action type="dataflow/convert_adapter_io" method="load">
       ...
   </action>

   Action tag has two parameters: type and method. Type tells as which adapter class is to be used in this action. It is defined using its alias. Method tells us which method of this adapter class action should call. As mentioned before, by default there are two available methods: load and save. Children of action tag define variables which are parameters used when executing adapter's method. Variables are defined like in the example below:

   <action type="dataflow/convert_adapter_io" method="load">
       <var name="type">file</var>
       <var name="path">var/import</var>
       <var name="filename"><![CDATA[products.csv]]></var>
       <var name="format"><![CDATA[csv]]></var>
   </action>

2. Magento DataFlow default adapters

   Magento DataFlow module contains few default adapter classes which you can find in app/code/core/Dataflow/Model/Convert/Adapter. Not all of them have yet implemented load() and save() methods.

   For common case of reading data from or saving data to local or remote file you will use dataflow/convert_adapter_io (Mage_Dataflow_Model_Convert_Adapter_Io).

   Following variables will allow you to define local/remote file as data source:

   • type - defines type of io source we want to process. Valid values: file, ftp
   • path - defines relative path to the file
   • filename - defines data source file's name
   • host - for ftp type it defines the ftp host
   • port - for ftp type it defines the ftp port; if not given, default value is 21
   • user - for ftp type it defines the ftp user, if not given default value is 'anonymous' and password then is 'anonymous@noserver.com'
   • password - for ftp type it defines the ftp user's password
   • timeout - for ftp type it defines connection timeout; default value is 90
   • file_mode - for ftp type it defines file mode; default value is FTP_BINARY
   • ssl - for ftp type if it is not empty, then ftp ssl connection is used
   • passive - for ftp type it defines connection mode; default value is false

3. Customer and Product adapters

   For most commonly exchanged entities - customer and product - Magento provides default adapters: customer/convert_adapter_customer (Mage_Customer_Model_Convert_Adapter_Customer) and catalog/convert_adapter_product (Mage_Catalog_Model_Convert_Adapter_Product). Both inherit from Mage_Eav_Model_Convert_Adapter_Entity.

   To simply load all customers data for selected store you can use the following xml:

   <action type="customer/convert_adapter_customer" method="load">
       <var name="store">default</var>
   </action>

   Sometimes you may want to not load all customers in database. To help you with this there are following variables valid:

   • filter/firstname - to load only customers with firstname starting with value of this variable
   • filter/lastname - to load only customers with lastname starting with value of this variable
   • filter/email - to load only customers with email starting with value of this variable
   • filter/group - to load only customers from group with id equal to value of this variable
   • filter/adressType - to export only selected addressType; valid values are: both, default_billing, default_shipping
   • filter/telephone - to load only customers with telephone starting with value of this variable
   • filter/postcode - to load only customers with postcode starting with value of this variable
   • filter/country - to load only customers with country iso code equal to value of this variable
   • filter/region - to load only customers with region equal to value of this variable (for US just 2-letter state names)
   • filter/created_at/from - to load only customers created after a date defined as value of this variable
   • filter/created_at/to - to load only customers created before a date defined as value of this variable

   For example:

   <action type="customer/convert_adapter_customer" method="load">
       <var name="store"><![CDATA[0]]></var>
       <var name="filter/firstname"><![CDATA[a]]></var>
       <var name="filter/lastname"><![CDATA[a]]></var>
       <var name="filter/email"><![CDATA[a]]></var>
       <var name="filter/group"><![CDATA[1]]></var>
       <var name="filter/adressType"><![CDATA[default_billing]]></var>
       <var name="filter/telephone"><![CDATA[1]]></var>
       <var name="filter/postcode"><![CDATA[7]]></var>
       <var name="filter/country"><![CDATA[BS]]></var>
       <var name="filter/region"><![CDATA[WA]]></var>
       <var name="filter/created_at/from"><![CDATA[09/22/09]]></var>
       <var name="filter/created_at/to"><![CDATA[09/24/09]]></var>
   </action>

   Same way you can load and filter products loaded from database with following variables:

   • filter/name - to load only products with name starting with value of this variable
   • filter/sku - to load only products with sku starting with value of this variable
   • filter/type - to load only products with type defined as value of this variable; valid values are: simple, configurable, grouped, bundle, virtual, downloadable
   • filter/attribute_set - to load only products with attribute set id equal to value of this variable
   • filter/price/from - to load only products with price starting from value of this variable
   • filter/price/to - to load only products with price up to value of this variable
   • filter/qty/from - to load only products with quantity starting from value of this variable
   • filter/qty/to - to load only products with quantity up to value of this variable
   • filter/visibility - to load only products with visibility id equal to value of this variable
   • filter/status - to load only products with status id equal to value of this variable

   Example:

   <action type="catalog/convert_adapter_product" method="load">
       <var name="store"><![CDATA[0]]></var>
       <var name="filter/name"><![CDATA[a]]></var>
       <var name="filter/sku"><![CDATA[1]]></var>
       <var name="filter/type"><![CDATA[simple]]></var>
       <var name="filter/attribute_set"><![CDATA[29]]></var>
       <var name="filter/price/from"><![CDATA[1]]></var>
       <var name="filter/price/to"><![CDATA[2]]></var>
       <var name="filter/qty/from"><![CDATA[1]]></var>
       <var name="filter/qty/to"><![CDATA[2]]></var>
       <var name="filter/visibility"><![CDATA[2]]></var>
       <var name="filter/status"><![CDATA[1]]></var>
   </action>

Seems a little bit frightening if you see all those id values you have to provide for filters. Fortunatelly for these two entities - customers and products - there is wizard like profile generator that allows you to define filter with simple select boxes.

In next part I will describe use of parsers and adapters in context of parsers.

Have you ever tried to read the complete list of features Magento comes with? Gosh, it's huge. And it's even more impressive when you look at Magento Connect and all the extensions available there. However, if you are reading this blog entry you are probably looking for something which is not available yet. Or maybe you are a developer who just loves to play with the code. Either way, for serious business reasons or just because of your hack-ish nature, developing a Magento module is fun.

Let's create something really really simple. The module. You might wonder what that nifty module will be doing. Well in this fist part it will does nothing but being declared in Magento!

Where to begin?

Take a loot at the directories structure of your Magento installation. In app > code > local, create a directory, which will be a kind of container for your modules, in Magento it's usually called code pool. If you wonder how to name it, I can say that company name you're working for is probably a good one. It is in my case ;)

So let's create it

Then, in this container we should create the module. For the purpose of this blog let's call it "Reader" (in next few parts the reason of using this name should clarify, if not... well.. it's still sounds good ;) )

So far, so good. The next step is a little bit more complex, but still simple. Let's change current directory...

Hm... where are we?

Now, you must tell Magento about the module. To do this you must create an xml file. Take your favorite editor and create a file called Baobaz_Reader.xml (as you probably guessed the name consists of <container_name>_<module_name>.xml

Now put this into your file

Save file. And.... Voilà! That's one small step for a man, one giant leap for man...gento. :-)

To be sure that Magento knows about our new module let's login to admin panel. Navigate to System > Configuration, choose Advanced, from Advances section (yes, it's on the left).

You should see Baobaz_Reader at the top (or somewhere there) of the "Disable modules output" list.

OK, OK, OK... I admit that our nifty module makes nothing so far but be patience, in a few next parts we will add some more useful features.

To be continued...

One of major features of e-commerce websites is the possibility to share data with offline sale management systems. Magento made data exchange flexible and quite easy with DataFlow module.

Magento DataFlow is a data exchange framework that use four types of components: adapter, parser, maper and validator. At current state of developement validators are not implemented, but are reserved for future use.

Dataflow of data exchange process is defined as XML structure and called profile. Magento provides simple wizard-like tool for generation of some basic import/export profiles operating on products or customers entities. Advanced profiles manager is also provided for advanced users able to create XML defining profile without wizard tool and with need to use more custom dataflow operations related also to other entities.

Adapters are responsible for pluging into an external data resource and fetching requested and filtered data. It can be used for example to get data from: local or remote file, web services, database and more.

For example to load data from csv file you can put in XML profile the following code:

To load data from remote FTP server you can use same adapter, but with these parameters:

Parsers are responsible for transforming one data format to another. It can be used for example to convert CSV file content to two-dimmensional array, or opposite.

To parse CSV file content into database product entities you can use this code in your profile:

Adapter defined within parser variables as <var name="adapter"> and adapter's method <var name="method"> are responsible for parsing loaded data. In this particular case parser converts data from CSV file content to two-dimmensional array and calls the adapter's method "parse" to process it.

The simplest way of import customization is creating own adapter given as variable within parser definition. In most cases you will need to overwrite one of existing adapters and modify or write your own parsing method (in most cases it will be overwrited saveRow() method)

Mappers are responsible for altering data values from one to another. These are useful for maping one field to another.

In example below source's 'reference' column is mapped into 'sku' column and variable '_only_specified' is set to true, so imported/exported will be only listed columns:

This is just the tip of the iceberg of possibilities Magento DataFlow module offers. Come back later to read more.

One of the frequent questions you ask when installing Magento is how to add Bestsellers block to your homepage

Indeed, Magento - in its standard version at least - does not provide this feature.

Nevertheless, even if the lack of this feature might be quite weird for an ecommerce platform, it is not a blocking issue. Quite the contrary, it allows you to realise how easy it is to create new blocks on Magento, or, at least, the flexibility of the architecture of Magento.

1. Get the data

   This could be the most difficult step since you must know perfectly the Magento core object model.
   As soon as you know where to collect data (in this case, sold products) and how to request them (in this case, by descending order of sales), getting required data is pretty easy, it just take one line (actually several lines here, but just to allow a better readability).
   

   $_productCollection = Mage::getResourceModel('reports/product_collection')
       ->addAttributeToSelect('*')
       ->addOrderedQty()
       ->setOrder('ordered_qty', 'desc');

2. Create your BestSellers block

   Remember to use MVC pattern: Zend framework, and therefore Magento, ensure that each object has its specific role.
   Creat your business object:
   In a file that could be app/code/local/ma_librairie/Bestseller/Model/Bestseller.php write your business rules (here: getting sold products by descending order of sales).
   Then in a file app/code/local/ma_librairie/Bestseller/Block/Bestseller.php extend class Mage_Core_Block_Template that will pass data to display block.
   At last, in file app/design/frontend/mon_theme/default/template/catalog/product/view/bestseller.phtml display block content.
   Of course these files have to be declared in the appropriate XML configuration files.

3. Display your block

   Set up your block on yout active layout.
   Either change the XML file of your layout, or add it on your homepage from Magento backoffice.

For more information about Magento Templates, you may read the article Creating templates for your Magento website.

Magento Top 10 things to know: Magento software has made quite a buzz since the version 1 release in 2008 and revolutionized the world of e-commerce solutions. The reasons for this enthusiasm about Magento websites are numerous, here are 10 of them.

1. Opensource Software, Solid Company

   Written in PHP, based on the Zend framework, Magento is an opensource solution.
   This means that source code is public: everyone has access to it, for free. Contributions from anyone are welcome and speed of integration of new features is increased, as it was the case for example for VAT management: the first official release was not adapted to European market but the openness of the code helped solving this problem within a few weeks.
   However, Magento was developed by a for-profit company, Varien, ensuring uniformity and consistency in functionality and continuity in developments, with over sixty persons working on it .

2. Turnkey Software

   Download Magento, unzip the archive on your Apache / PHP / MySql server, run the automatic installation script, go to the admin panel to customize Magento configuration and you're ready to sell!
   Having that done, updating the system when new releases comes out can be easily done by going back to the administration panel, clicking only twice on your mouse, no more.
   Of course, you would still have to customize the look of your store and integrate your catalog content.

3. Highly Customizable Data Model, Long Feature List

   Whether you are selling shoes, socks, clothes, curtains, wine, hotel rooms or virtual downloadable products, Magento data model is designed and thought to fit your needs, while offering the standard features that you are entitled to expect from an e-commerce website: product customizable attributes (sizes, colors ...), configurable products, Cross-Sell, Up-Sell, et cetera.
   Moreover, version 1.3 introduced the concept of flat catalog that provides an excellent compromise between performance and customizability.
   Among the many standard features available, you will find tools for managing price and promotions.

4. Advanced Management for Template Design

   Magento Design system allows you to manage the template of your Magento website.
   Supplied with a basic graphic design, Magento allows you to lay out your shop as you want to, while allowing you to benefit from future releases. (See Creating templates for your Magento website.)

5. Designed for Search Engine Optimization

   Allowing URL rewriting, meta-data fields, Google Site Map management, etcetera, Magento gives you all the tools you need for Search Engine Optimization.
   (See Magento et le référencement naturel)

6. Global Software

   Thanks to Magento your store is ready for global sales, with the possibility (but not the obligation) to define for each area or country:

   • its own currency,
   • its own language (many languages are available),
   • its own graphic design,
   • its own sub-catalog,
   • its own shipping rules,
   • et caetera .

7. Payment Modules

   Magento currently knows how to handle over 200 payment systems , including:

   • ATOS / Sips,
   • CyberMUT Paiement / Paiement CIC,
   • CyberP@iement,
   • FIAT-NET ReceiveAndpay,
   • Google Checkout,
   • Ogone,
   • Check,
   • Paybox,
   • PayPal,
   • SPPLUS,
   • et caetera .

   It is also easy to add new ones.

8. Shipping modules

   Magento currently knows how to handle over 60 shipping systems , including:

   • Free shipping management,
   • Chronospot,
   • Colissimo,
   • DHL,
   • FedEx,
   • TNT, relais colis,
   • UPS,
   • et caetera .

   It is also easy to customize a module to suit your logistical constraints and your pricing policy, or even create new ones.

9. Connecting to Your System

   Magento provides standard modules import / export that allows you to import files to Magento.
   These modules can be easily automated and customized to fit in your existing system. (See Importing files into Magento)

10. Robust Software, Reliable Solution

    With today hundreds of e-Store running on Magento , some of which containing over 300,000 records (three hundred thousand) Magento is now recognized as a reliable and trustworthy software, whatever your needs are.

Among the many standard features, one of them is the subject of many questions: importing files into Magento.

Indeed, except for a few pure players who can just rely on Magento back-office, writing interfaces connections with existing information system is a prerequisite when setting up an e-Commerce solution.

This might involve:

• updating your Magento catalog (SKU, description, pictures...) from an existing back office,
• updating stocks quantities from an existing back office,
• pushing new orders in batch mode (eg. once a day) or interactive mode (in real time) from Magento to logistic systems,
• pulling orders (order statuses, shipping tracking IDs ...), from logistic system and send e-mails to customers,
• pushing orders data to accounting systems,
• pushing customers data to CRM systems,
• et caetera.

Of course, these operations, their automation and especially their test running can not be built in a Magento module nor in the Magento configuration panel. However, Magento offers APIs and connectors that provide solutionq for writing connectors that can be easily and cleanly plugged to the core system. This allows to have less specific developments and insure compatibility with future versions. The most time consuming part will eventually be the specification of your business needs.