Dans les profondeurs - generating Ryzom Core sources from xml

[nSigma]

This all started with trying to understand changes to server/src/entities_game_service/database_guild/outpost/plc.cpp/h


Templated Code generation
There exist xslt transforms for generating many of the interfaces through the server and common src trees.  The respective cpp/hfiles actually contain "DO NOT EDIT" notices.
I've been working through understanding what is going on there, specifically starting from merging changes that all seemed to start with database.xml.
There is one readme in the shard_unifier_service that kicks things off... it points to xslt_proc and generate_module_interface.xslt.  This interface transform matches code generated across many of the server folders and in the common game_share.. each with an xml descriptor that is very easily revived. 

Herewith are my notes if anyone wants to look at it, or is already familiar with it!

-----

=== Generated files: xslt ===
Somes files are generated from XML sources,
eg:
  server/shard_unifier_services' nel_database_mapping.cpp/h  and database_mapping.cpp/h
 
Editing these is not recommended (noted in the source!)  These files can be HUGE..
Updates to the structures and code is indicated to be done through making the necessary updates to the .xml files and running the xml through xsltproc (with an appropriate conversion) to generate the updated source.

Tool:
  sudo apt-get install xsltproc

Generation notes:
  The origional xslt transforms (eg: common/src/game_share/generate_module_interface.xslt) is aruably a little outdated.. (note use of auto_ptr)
    (updating while reviewing to ensure generated output at the very least matches repo source)

  Notes on Generators:
    common/src/game_share/generate_module_interface.xslt
      This transform does interface definitions for types.. wrapping enums with string getters and setters, member gets/sets and general object guff
      Source xmls fed to this transform are in various folders for sub components.
    - supports : header cpp php
    - updated this to use the CUniquePtr from deprecated auto_ptr as per repo code
    - added (int32) casts as per repo code
    - added nilPtr rename as per repo code
    - updates to correct this transform for c++11 compatiblility (*done - verfied new generation matches origional repo source!)
    Supported params:
        --stringparam output <header|cpp|php>
        --stringparam filename {target filename minux extension}


    common/src/game_share/generate_client_db.xslt
      This transform seems to be very specifically for the server and client databases.  Output is HUGE
      Source xmls fed to this transform are in various folders for sub components.
    - supports : cpp-server header-server cpp-client header-client
    Supported params:
        --stringparam output <header|cpp|php>
        --stringparam side <client|server>
        --stringparam filename {target filename minux extension}
        --stringparam bank <PLC|GUILD|OUTPOST>

  NB+++ General Results:
    The scripts could simplify code maintenance.. and simplify mods just based on updates to key files.
    It may be well worth organising the code generation with these transforms with some scripts to automate their use.


    === SERVER ===
 
   : server/src/shard_unifier_service
      Files that are identified to be generated:
        nel_database_mapping.cpp/h >> TF:generate_module_interface.xslt IN:nel_database_mapping.xml (*confirmed)
        database_mapping.cpp/h >> TF:generate_module_interface.xslt IN:database_mapping.xml (*confirmed)

      HowTo: Run within the server/src/shard_unifier_service folder...

        Generation source : nel_database_mapping.xml

Code: [Sélectionner]

xsltproc -o nel_database_mapping.cpp --stringparam output cpp --stringparam filename nel_database_mapping ../../../common/src/game_share/generate_module_interface.xslt nel_database_mapping.xml
xsltproc -o nel_database_mapping.h --stringparam output header --stringparam filename nel_database_mapping ../../../common/src/game_share/generate_module_interface.xslt nel_database_mapping.xml
       
Generation source : database_mapping.xml

Code: [Sélectionner]

xsltproc -o database_mapping.cpp --stringparam output cpp --stringparam filename database_mapping ../../../common/src/game_share/generate_module_interface.xslt database_mapping.xml
xsltproc -o database_mapping.h --stringparam output header --stringparam filename database_mapping ../../../common/src/game_share/generate_module_interface.xslt database_mapping.xml

   : server/src/entities_game_service
      Files that are identified to be generated:
        database_guild.cpp/h >> TF:generate_client_db.xslt IN:common/data_common/database.xml
        database_outpost.cpp/h >> TF:generate_client_db.xslt IN:common/data_common/database.xml
        database_plr.cpp/h >> TF:generate_client_db.xslt IN:common/data_common/database.xml

      Sources:
        database.xml >> common/data_common/database.xml
        -- used for : database_ [plr, guild, outpost] generation!
       
      HowTo: Run within the server/src/entities_game_service folder...
        Generation source : nel_database_mapping.xml
 

Code: [Sélectionner]

xsltproc -o database_plr.cpp --stringparam output cpp --stringparam side server --stringparam filename database_plr --stringparam bank PLR ../../../common/src/game_share/generate_client_db.xslt ../../../common/data_common/database.xml
xsltproc -o database_plr.h --stringparam output header --stringparam side server --stringparam filename database_plr --stringparam bank PLR ../../../common/src/game_share/generate_client_db.xslt ../../../common/data_common/database.xml


Code: [Sélectionner]

xsltproc -o database_guild.cpp --stringparam output cpp --stringparam side server --stringparam filename database_guild --stringparam bank GUILD ../../../common/src/game_share/generate_client_db.xslt ../../../common/data_common/database.xml
xsltproc -o database_guild.h --stringparam output header --stringparam side server --stringparam filename database_guild --stringparam bank GUILD ../../../common/src/game_share/generate_client_db.xslt ../../../common/data_common/database.xml


Code: [Sélectionner]

xsltproc -o database_outpost.cpp --stringparam output cpp --stringparam side server --stringparam filename database_outpost --stringparam bank OUTPOST ../../../common/src/game_share/generate_client_db.xslt ../../../common/data_common/database.xml
xsltproc -o database_outpost.h --stringparam output header --stringparam side server --stringparam filename database_outpost --stringparam bank OUTPOST ../../../common/src/game_share/generate_client_db.xslt ../../../common/data_common/database.xml

          Note - client side does not seem to generate anything.
            There are no mode definitions for mode="header-client" or mode="cpp-client".
          eg:

Code: [Sélectionner]

xsltproc -o database_plr_client_test.cpp --stringparam output cpp --stringparam side client --stringparam filename database_plr_client_test --stringparam bank PLR ../../../common/src/game_share/generate_client_db.xslt ../../../common/data_common/database.xml
xsltproc -o database_plr_client_test.h --stringparam output header --stringparam side client --stringparam filename database_plr_client_test --stringparam bank PLR ../../../common/src/game_share/generate_client_db.xslt ../../../common/data_common/database.xml

          Notes:
            Updates in database.xml from generated code that is in repo
              set: TARGET/CONTEXT_MENU/COUNTER=I5 (from I4 - match repo)
              add: EXCHANGE,RECEIVE,TEMP,SHARE,TRADING/
                          CREATE_TIME:I32 (to match repo)
                          SERIAL,I32 (to match repo)
              add: PACK_ANIMAL
              check: GUILD/FAME/count == 7 (old was 6.. see pvp_clan.h)
              set: PACKANIMAL/BEAST/count=7 (from 4 - match git repo)
              add: PACKANIMAL/BEAST/NAME:I32 (to match repo)
              check: FAME/PLAYER/count == 7 (old was 6.. see pvp_clan.h)
         
          NB+++Result:
            The output of running these is correct and complete updated code for
              database_plr.cpp/h database_guild.cpp/h database_outpost.cpp/h
            based on changes to the common/data_common/database.xml...
         
         
   !TODO!
   : server/src/admin_modules
      Files that are identified to be generated:
        admin_modules_itf.cpp/h

   !TODO!
   : server/src/patchman_service
      Files that are identified to be generated:
        module_admin_itf.cpp/h
        re_module_itf.cpp/h
        rr_module_itf.cpp/h
        spa_module_itf.cpp/h
        spm_module_itf.cpp/h
        spt_module_itf.cpp/h

   !TODO!
   : server/src/server_share
      Files that are identified to be generated:
        backup_service_itf.cpp/h
        char_name_mapper_itf.cpp/h
        chat_unifier_itf.cpp/h
        command_executor_itf.cpp/h
        entity_locator_itf.cpp/h
        entity_locator_itf.cpp/h
        logger_service_itf.cpp/h
        login_service_itf.cpp/h/php
        mail_forum_itf.cpp/h
       
       
    === COMMON ===

   : common/src/game_share
      Files that are identified to be generated:
        --- lua
        item_special_effect.cpp/h >> lua.. TF:enum_generator.lua IN:item_special_effect.lua (*eyeballed... this seems to be the only use case thus far for the lua convertor.. if more instances are found then perhaps an xslt transform should be on the cards)
        Note: quite a few of the enums follow the same structure.. whether this is spawn from the original use of the script or just devs rolling through as the enum type is simple enough to not really require a generator..
        --- xslt
        character_sync_itf.cpp/h >> TF:generate_module_interface.xslt IN:character_sync_itf.xml (*confirmed h/cpp, however manual repo mods change ucstring for Fullname to const ucstring&... need to verify concern)
        msg_ais_egs_gen.h >> TF:generate_module_interface.xslt IN:msg_ais_egs_gen.xml (*confirmed h, can gen cpp but is has no functional content.  Manual repo edits for new member: Organization get/set methods.  Fixed by updating xml source.)
        r2_modules_itf.cpp/h >> TF:generate_module_interface.xslt IN:r2_modules_itf.xml (*confirmed cpp/h - minor compile cleanup mods in repo src)
        r2_share_itf.cpp/h >> TF:generate_module_interface.xslt IN:r2_share_itf.xml (*confirmed cpp/h - minor compile cleanup mods in repo src)
        ring_session_manager_itf.cpp/h >> TF:generate_module_interface.xslt IN:ring_session_manager_itf.xml (*confirmed cpp/h)
        ring_session_manager_itf.php >> TF:(??) IN:(??)

      HowTo: Run within the common/src/game_share folder...
        Generation source : item_special_effect.lua
          (?? Lua script)

        Generation source : character_sync_itf.xml
          Note: fixed pathing

Code: [Sélectionner]

xsltproc -o character_sync_itf.cpp --stringparam output cpp --stringparam filename character_sync_itf ./generate_module_interface.xslt character_sync_itf.xml
xsltproc -o character_sync_itf.h --stringparam output header --stringparam filename character_sync_itf ./generate_module_interface.xslt character_sync_itf.xml

        Generation source : msg_ais_egs_gen.xml
          Note: fixed pathing

Code: [Sélectionner]

xsltproc -o msg_ais_egs_gen.h --stringparam output header --stringparam filename msg_ais_egs_gen ./generate_module_interface.xslt msg_ais_egs_gen.xml

        Generation source : r2_modules_itf.xml

Code: [Sélectionner]

xsltproc -o r2_modules_itf.cpp --stringparam output cpp --stringparam filename r2_modules_itf ./generate_module_interface.xslt r2_modules_itf.xml
xsltproc -o r2_modules_itf.h --stringparam output header --stringparam filename r2_modules_itf ./generate_module_interface.xslt r2_modules_itf.xml

        Generation source : r2_share_itf.xml

Code: [Sélectionner]

xsltproc -o r2_share_itf.cpp --stringparam output cpp --stringparam filename r2_share_itf ./generate_module_interface.xslt r2_share_itf.xml
xsltproc -o r2_share_itf.h --stringparam output header --stringparam filename r2_share_itf ./generate_module_interface.xslt r2_share_itf.xml

        Generation source : ring_session_manager_itf.xml
          Note: fixed pathing, shard count enum

Code: [Sélectionner]

xsltproc -o ring_session_manager_itf.cpp --stringparam output cpp --stringparam filename ring_session_manager_itf ./generate_module_interface.xslt ring_session_manager_itf.xml
xsltproc -o ring_session_manager_itf.h --stringparam output header --stringparam filename ring_session_manager_itf ./generate_module_interface.xslt ring_session_manager_itf.xml

-------


So, after all that, it builds... lol, a little more automation and perhaps a little less work down the line for some mods

~/n/Sigma