Language and development rules: Difference between revisions

Jump to navigation Jump to search
m
no edit summary
Tag: 2017 source edit
mNo edit summary
Tags: Visual edit Mobile edit Mobile web edit Advanced mobile edit
Line 289: Line 289:
</syntaxhighlight>
</syntaxhighlight>


Note, the problem of float numbers is the same on all languages and not only when inserting data into database. It occurs also with any language when you work on "real" numbers, so numbers must be, as soon as they are affected into variable, cleaned with function price2num with second parameter defined to :
Note, the problem of float numbers is the same on all languages and not only when inserting data into database. It occurs also with any language when you work on "real" numbers, so numbers must be, as soon as they are affected into variable, cleaned with function price2num with second parameter defined to<span> </span>:
'MU' (for unit prices), 'MT' (for total prices) or 'MS' (otherwise) depending on usage of number. (see function documentation)
'MU' (for unit prices), 'MT' (for total prices) or 'MS' (otherwise) depending on usage of number. (see function documentation)


* SQL Date functions NOW, SYSDATE, DATEDIFF or DATE are forbidden inside SQL requests. Other functions like (MONTH, YEAR) must also be avoided if possible. If you must use the current date into a field, value must come from the PHP and not from the database engine. This is for 3 reasons:
*SQL Date functions NOW, SYSDATE, DATEDIFF or DATE are forbidden inside SQL requests. Other functions like (MONTH, YEAR) must also be avoided if possible. If you must use the current date into a field, value must come from the PHP and not from the database engine. This is for 3 reasons:
** better portability of code
**better portability of code
** better management of TimeZone (reference timezone is the PHP timezone and timezone of database may differs so all dates functions must be on PHP side).
**better management of TimeZone (reference timezone is the PHP timezone and timezone of database may differs so all dates functions must be on PHP side).
** better performance (see example and final comment)
**better performance (see example and final comment)


For example, don't do:
For example, don't do:
Line 317: Line 317:
An other advantage of this rule, is that request benefits of index because we are making a compare of a field with a fixed value. When using datediff, you make an operation on field before comparison, this means database can't use the index on field, resulting on very bad performance compared to solution without the datediff.
An other advantage of this rule, is that request benefits of index because we are making a compare of a field with a fixed value. When using datediff, you make an operation on field before comparison, this means database can't use the index on field, resulting on very bad performance compared to solution without the datediff.


* The statement WITH ROLLUP is prohibited:
*The statement WITH ROLLUP is prohibited:


This statement should not be used: it is not handled the same way by different databases.
This statement should not be used: it is not handled the same way by different databases.
Also, using WITH ROLLUP breaks the purity of the returned data. The intermediate subtotal generation aggregation performed by this statement can easily be done using PHP and helps keep the returned data array clean (not corrupted by artificially inserted data).
Also, using WITH ROLLUP breaks the purity of the returned data. The intermediate subtotal generation aggregation performed by this statement can easily be done using PHP and helps keep the returned data array clean (not corrupted by artificially inserted data).


* The sql function GROUP_CONCAT can't be used. It is not portable and ask the database to make subrequests for line into main table, this can be done easier by doing a subrequest inside the PHP loop when building the output of each line.
*The sql function GROUP_CONCAT can't be used. It is not portable and ask the database to make subrequests for line into main table, this can be done easier by doing a subrequest inside the PHP loop when building the output of each line.


* Use $db->ifsql for SQL IF
*Use $db->ifsql for SQL IF


Do not include the IF into your forged SQL request. But use instead $db->ifsql() method to you will make a SQL IF that is compatible with all SQL databases.
Do not include the IF into your forged SQL request. But use instead $db->ifsql() method to you will make a SQL IF that is compatible with all SQL databases.


* No DELETE CASCADE and ON UPDATE CASCADE
*No DELETE CASCADE and ON UPDATE CASCADE


Such SQL instructions are forbidden because they bypass the business rules of the application bringing a lot of troubles out of control of the developers. For example, if there is a delete cascade between table A and B, when the application will execute the code to remove a record in A, the children into table B will also be removed (this is the goal). But if there was a Dolibarr PHP trigger (for example provided by an external module) on the deletion of record of B (for example to validate the deletion or to execute a complementary action), the DELETE CASCADE will be executed without having the Dolibarr PHP trigger on B deletion executed, missing the validation or actions of the PHP trigger of the module. All business rules must be implemented on the same side (so the server PHP side), this is the reason why business rules implemented on Database side (like with delete cascade) are not allowed (same conclusion than the next point about Database triggers).
Such SQL instructions are forbidden because they bypass the business rules of the application bringing a lot of troubles out of control of the developers. For example, if there is a delete cascade between table A and B, when the application will execute the code to remove a record in A, the children into table B will also be removed (this is the goal). But if there was a Dolibarr PHP trigger (for example provided by an external module) on the deletion of record of B (for example to validate the deletion or to execute a complementary action), the DELETE CASCADE will be executed without having the Dolibarr PHP trigger on B deletion executed, missing the validation or actions of the PHP trigger of the module. All business rules must be implemented on the same side (so the server PHP side), this is the reason why business rules implemented on Database side (like with delete cascade) are not allowed (same conclusion than the next point about Database triggers).
Line 335: Line 335:
==Using Database triggers==
==Using Database triggers==
Using Database triggers is not allowed into the code of Dolibarr, so you should have no troubles with Database triggers.  
Using Database triggers is not allowed into the code of Dolibarr, so you should have no troubles with Database triggers.  
* Database triggers are not portable. Compatibility may be broken when upgrading your version of your database
 
* Database triggers are often forgotten into backup tools
*Database triggers are not portable. Compatibility may be broken when upgrading your version of your database
* Database triggers need special permissions that you may not have (and should not have) on a common secured database
*Database triggers are often forgotten into backup tools
* Database triggers means also that you include some "business rules" of your application into the database where most other rules are managed by the code itself (this can create conflict or code that can't be debugged with a step by step debugger)
*Database triggers need special permissions that you may not have (and should not have) on a common secured database
* Database triggers execute code using the timezone, the default language of the database instead of the reference timezone of the application that may be different (generating offset into database) and the default language setup in application ...
*Database triggers means also that you include some "business rules" of your application into the database where most other rules are managed by the code itself (this can create conflict or code that can't be debugged with a step by step debugger)
* There is tons of reasons why we do not recommend to use database triggers in your own development.
*Database triggers execute code using the timezone, the default language of the database instead of the reference timezone of the application that may be different (generating offset into database) and the default language setup in application ...
*There is tons of reasons why we do not recommend to use database triggers in your own development.


Using Database triggers can save you time when you make development but is a very bad practice when quality, portability, security and sustainability are priority, so it is forbidden as it is not compatible with the high level of quality of code expected by Dolibarr team. External modules that uses Database triggers may also be refused on the market place since it won't work for more than 50% of users.
Using Database triggers can save you time when you make development but is a very bad practice when quality, portability, security and sustainability are priority, so it is forbidden as it is not compatible with the high level of quality of code expected by Dolibarr team. External modules that uses Database triggers may also be refused on the market place since it won't work for more than 50% of users.
Line 373: Line 374:
</syntaxhighlight>
</syntaxhighlight>


There is no native SQL requests "UPDATE FROM" in all Dolibarr core (only simple "UPDATE") so this difference of syntax is not a problem. But if you need to use one in your own code of your module, you should do :
There is no native SQL requests "UPDATE FROM" in all Dolibarr core (only simple "UPDATE") so this difference of syntax is not a problem. But if you need to use one in your own code of your module, you should do<span> </span>:
<syntaxhighlight lang="PHP">
<syntaxhighlight lang="PHP">
if ($this->db->type=='pgsql') {
if ($this->db->type=='pgsql') {
Line 393: Line 394:
For example:
For example:
<syntaxhighlight lang="PHP">
<syntaxhighlight lang="PHP">
print '<a href="'.dol_buildpath('/mydir/mypage.php').'">'.img_picto('Texte alt','namepictopng','').'</a>';
print '<a href="'.dol_buildpath('/mydir/mypage.php').'">'.img_picto('Texte alt','namepictopng','').'</a>';''
</syntaxhighlight>
</syntaxhighlight>


Line 455: Line 456:
Note that the skeletons are also used by the PHP Code generator ([[Module ModuleBuilder]]), which is described in the development chapter of Dolibarr modules, to speed up your development.
Note that the skeletons are also used by the PHP Code generator ([[Module ModuleBuilder]]), which is described in the development chapter of Dolibarr modules, to speed up your development.


== Global variables==
==Global variables==
There is some information or objects that is common and fixed for all the execution of a PHP page and need to be used nearly everywhere in the code. To avoid to propagate everywhere this
There is some information or objects that is common and fixed for all the execution of a PHP page and need to be used nearly everywhere in the code. To avoid to propagate everywhere this
information or objects, they are stored into global variables that you can access everywhere. This global variables are initialized at the begin of each PHP call (into the master.inc.php or master.inc.php or in top of the page). These objects are instantiated only once. The list of available global variables is defined here:
information or objects, they are stored into global variables that you can access everywhere. This global variables are initialized at the begin of each PHP call (into the master.inc.php or master.inc.php or in top of the page). These objects are instantiated only once. The list of available global variables is defined here:
{{Template:TemplateGlobalVariables}}
{{Template:TemplateGlobalVariables}}


* $conf is the object that contains the configuration (saved into database)
*$conf is the object that contains the configuration (saved into database)
* $user is the object of the current user
*$user is the object of the current user
* $mysoc is the object with all information about the current company
*$mysoc is the object with all information about the current company
* $langs is the object with the current language
*$langs is the object with the current language
* $db is the database handler of the current database connexion (inside classes, we should pass it in the constructor and we should get it with $this->db)
*$db is the database handler of the current database connexion (inside classes, we should pass it in the constructor and we should get it with $this->db)
* $hookmanager is a factory class to manage hooks
*$hookmanager is a factory class to manage hooks
* $extrafields is a factory class to manage extrafields
*$extrafields is a factory class to manage extrafields


==Dates and Timezones==
==Dates and Timezones==
Line 636: Line 637:
This is not a good idea as it is just technically not possible for the following reasons:
This is not a good idea as it is just technically not possible for the following reasons:


* First, you must understand that composer is a tool '''to deploy a frozen image of a package'''. It is a tool to '''deploy''', not a tool to '''manage dependencies''' (event if some features exists for that). Dolibarr is an application that has a lot of dependencies, but for each of them, the Dolibarr project has to apply some patches to have the dependencies working correclty (for different reasons, mainly because of bugs or architecture error into original project. See the file '''dev/dolibarr_changes.txt''' for full list). Each PHP dependency has at least one change to do to have it working correctly. So the image of a library L that Dolibarr needs is NOT the image provided into the composer package by the L project. After discussion with '''Nils Adermann''', the author of the composer package during the first OSFS congress (https://05f5.com) to get its point of view on this need for large projects, it appears that if a large project is in this situation, the good way to use composer is to build yourself a very large package called "dolibarr" with no composer dependencies and all external libraries embedded into the same composer project so we can deploy all in one composer command, confirming that composer is above all a good "fast deployment system" but a bad "dependency manager". The dependency system of composer project was simply designed to use an exact version of a library and not to use a patched version of this library. It is even designed to avoid this, making its use not possible in this context.
*First, you must understand that composer is a tool '''to deploy a frozen image of a package'''. It is a tool to '''deploy''', not a tool to '''manage dependencies''' (event if some features exists for that). Dolibarr is an application that has a lot of dependencies, but for each of them, the Dolibarr project has to apply some patches to have the dependencies working correclty (for different reasons, mainly because of bugs or architecture error into original project. See the file '''dev/dolibarr_changes.txt''' for full list we have to do on each dependency). Each PHP dependency has at least one change to do to have it working correctly. So the image of a library L that Dolibarr needs is NOT the image provided into the composer package by the L project. After discussion with '''Nils Adermann''', the author of the composer package during the first OSFS congress (https://05f5.com) to get its point of view on this need for large projects, it appears that if a large project is in this situation, the good way to use composer is to build yourself a very large package called "dolibarr" with no composer dependencies and all external libraries embedded into the same composer project so we can deploy all in one composer command, confirming that composer is above all a good "fast deployment system" but a bad "dependency manager". The dependency system of composer project was simply designed to use an exact version of a library and not to use a patched version of this library. It is even designed to avoid this, making its use not possible in this context.
* Second point is a different problem with the composer dependencies system: Each dependency on a library has itself a high number of sub-dependency. And all this sub-dependencies have also a high number of dependencies that nobody can't control (there is always sub-dependencies on the libraries you use that are managed by a lower level dependency project). The result is that when you use composer to install the main library you need, you install (and you can avoid it) a ton of other packages you don't need. For example, Dolibarr is a 80Mb project with all its PHP dependencies. If you install it with composer (we suppose that the first point is not a problem), then the project reach the size of 300 Mb. Size of project is multiplicated by 3.5 with no reason.
*Second point is a different problem with the composer dependencies system: Each dependency on a library has itself a high number of sub-dependency. And all this sub-dependencies have also a high number of dependencies that nobody can't control (there is always sub-dependencies on the libraries you use that are managed by a lower level dependency project). The result is that when you use composer to install the main library you need, you install (and you can avoid it) a ton of other packages you don't need. For example, Dolibarr is a 80Mb project with all its PHP dependencies. If you install it with composer (we suppose that the first point is not a problem), then the project reach the size of 300 Mb. Size of project is multiplicated by 3.5 with no reason.
* Third point is also problems with the dependency system: When you install an external library this library has dependency and each of this depency of depencies are constraints with each other and with PHP version. Imagine the required library L1 is compatible with PHP x to x+n and the library L2 is compatible with PHP y to y+m, etc... To have an application working correctly, you must find a PHP version that will be common to all the required packages, so here inside the intersection of the two ranges [x - x+n] and [y - y+m]. But when you are using a large number of high level libraries, it means you are depending on a very large number of low level libraries (because each high level libraries has a lot of medium level libraries as dependencies that has a high number of low level dependencies). And when this number is very large, the number of different required ranges become so large that you reach a situation where the number of PHP versions that are common to all of them (hte intersection of all ranges) is very low, and even does not exists at all (there is no combination of version that is compatible with ALL packages you need because each sub-project depreciate its PHP version in its own roadmap without taking care of which child or parent dependency use it). But Dolibarr must works for every body, whatever is other project roadmap, and on the as large as possible number of PHP versions and not just on one or two given major version of PHP. Compatibiliy must also be guaranted during a very long term (an ERP is not a software we can change every year), when low level libraries are changing requirement very often.
*Third point is also problems with the dependency system: When you install an external library this library has dependency and each of this depency of depencies are constraints with each other and with PHP version. Imagine the required library L1 is compatible with PHP x to x+n and the library L2 is compatible with PHP y to y+m, etc... To have an application working correctly, you must find a PHP version that will be common to all the required packages, so here inside the intersection of the two ranges [x - x+n] and [y - y+m]. But when you are using a large number of high level libraries, it means you are depending on a very large number of low level libraries (because each high level libraries has a lot of medium level libraries as dependencies that has a high number of low level dependencies). And when this number is very large, the number of different required ranges become so large that you reach a situation where the number of PHP versions that are common to all of them (hte intersection of all ranges) is very low, and even does not exists at all (there is no combination of version that is compatible with ALL packages you need because each sub-project depreciate its PHP version in its own roadmap without taking care of which child or parent dependency use it). But Dolibarr must works for every body, whatever is other project roadmap, and on the as large as possible number of PHP versions and not just on one or two given major version of PHP. Compatibiliy must also be guaranted during a very long term (an ERP is not a software we can change every year), when low level libraries are changing requirement very often.
* Last point is about security: As we view in the second point, when using composer, a very high number of PHP files are installed even if you don't need them. Dolibarr is a critical application that need a very strict security policy. Requirement for security is at a high level (See our page [[Security_information]]). When using composer, you install a lot of files and small sub-project and sub-project of sub-projects where security is not considered as important than what we need on the Dolibarr project. If we make an analysis on a given period, the number of vulnerability reported on the whole project Dolibarr + files of external libraries when installed by composer (that we mostly don't need) is multiplicated by 20 compared to the number of vulnerabilities that are reported only in the scope of Dolibarr + external libraries when they are embedded in their minimal version manually). Such an increase is simply not possible and some companies would simply not be allowed to use Dolibarr if we accept these due to a strict security requirements of their business.
*Last point is about security: As we view in the second point, when using composer, a very high number of PHP files are installed even if you don't need them. Dolibarr is a critical application that need a very strict security policy. Requirement for security is at a high level (See our page [[Security_information]]). When using composer, you install a lot of files and small sub-project and sub-project of sub-projects where security is not considered as important than what we need on the Dolibarr project. If we make an analysis on a given period, the number of vulnerability reported on the whole project Dolibarr + files of external libraries when installed by composer (that we mostly don't need) is multiplicated by 20 compared to the number of vulnerabilities that are reported only in the scope of Dolibarr + external libraries when they are embedded in their minimal version manually). Such an increase is simply not possible and some companies would simply not be allowed to use Dolibarr if we accept these due to a strict security requirements of their business.


As we can see, composer, even if it was largely adopted by a lot of small projects, is far away from the state of the art. For all this reasons, it is just not possible to use composer so we keep the composer.json file in the project so we can use it for some development tests or make upgrade tests of some libraries with it, but we have renamed it by adding the extension ".disabled" to be sure that end users won't try to install or update their Dolibarr with it (in which case, the application will become unsecured and unstable).
As we can see, composer, even if it was largely adopted by a lot of small projects, is far away from the state of the art. For all this reasons, it is just not possible to use composer so we keep the composer.json file in the project so we can use it for some development tests or make upgrade tests of some libraries with it, but we have renamed it by adding the extension ".disabled" to be sure that end users won't try to install or update their Dolibarr with it (in which case, the application will become unsecured and unstable).

Navigation menu