Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

Software on Silicon Blog

JamesT_21
Moderator
Moderator 100 likes received 50 likes received 25 likes received
Moderator

Faster Leaner More Flexible – ModusToolbox v2.2

Ladies and gentlemen, geeks of all kinds… I haven’t been here for a while, because we were busy getting our bouncing baby ModusToolbox v2.2 out the door. “We” is a huge team​ (see page iii for the v2.1 team). This tool is faster, leaner, and more flexible. (My editor would not let me say "flexibler." Go figure.) In some ways it is 4x faster and 100x leaner.

Bring it on.png

I don’t expect you to take my word for it. Here’s how that happened. ModusToolbox v2.2 introduces a new way to handle libraries. I love libraries – both the book kind and the software kind. One takes me of to an entirely new universe. The other does my work for me – I get to use someone else’s code. These are good things.

For those who love great books, this is all explained in the Library Manager User Guide. OK, maybe not a great book, the plot is a bit thin, but it is really useful. CJ (who is the tip of that particular spear) explains all this. When you start working for real with v2.2, go read it. That’s where I went when I was figuring out why the tool got a whole lot faster.

Version 2.1: the make system reads a URL (in a .lib file) and fetches that library, which may include more .lib files. The process proceeds recursively until all libraries are found and added to the application’s folder. Each time you create an application you may get yet another copy of the same library. This duplication takes time (slower), every application has every library (fatter), and that’s how it works (no choice).

Version 2.2 introduces shared libraries. For this it uses a .mtb file rather than a .lib file. So we call it the MTB flow rather than the older LIB flow. The mtb file adds a neat twist we’ll talk about in a moment. CJ tells you all the details in the documentation. But let’s talk.

You know your application needs a particular library, say a BSP. The deps folder in the application has the .mtb file for that library. If the BSP requires other libraries, those are represented by .mtb files in that library’s deps folder, and they are processed recursively. So far, this is how .lib files worked. So what’s different?

Here’s what changed.

The .mtb file has both the URL for the library I want, and a pointer to a shared location where the library might already exist. The default location is the mtb_shared folder. If the same version of the same library is already in the shared location, WOOHOO the make system doesn’t need to download it again. So you end up with faster (less to download) and leaner (a single small file that points to the library, instead of maybe 100 megs and 4,000 source files in your application folder). If the library isn’t there or it’s a different version, then the make system fetches it, just like before.

The make system also creates an mtb.mk file that keeps track of which library is used in the application, and where it is. You don’t need to deal with any of this, because it all just works. However, knowing a bit about the secret sauce is always a good thing.

OK, that’s faster and leaner. What about flexible? By default we share libraries. But you decide what’s shared and what’s specific to an individual application. If you want to treat it as app-specific, then your app gets the entire library, and you can do with it as you please without affecting any other application in your universe. You can still lock on any released version of any library. Talk about flexible!

LibManager.png

So, why do this? Well, for one we eat our own dogfood around here. The process took too long. More importantly we listen to customers. You didn’t like it either.

You may want to get ModusToolbox v2.2 sooner rather than later. Keep in mind you still need to get the library the first time, so that it is in the shared folder. Some older code examples won’t use the .mtb mechanism until we update them, so you may not see the improvement all the time. But when you do…

I did a little informal measurement on my machine, with my home wireless network access. Your mileage will vary of course. In the end, with shared libraries already in place from other apps, ModusToolbox v2.2 created Hello World four times faster, and the application occupied 1/100th of the disk space. I’ll take a 4X speed and 100X space optimization any day.

Finally, depending on how you work, your company’s rules, and all that good stuff, you may not use our make system at all. A common real-world workflow is to learn how this is all put together, what the dependencies are (handled automagically in our make system) and then fetch the libraries and versions you want from GitHub. Install them on your company’s server(s) and you have the “blessed” known good code in your own system. Point your development project at that code using whatever environment you want, and go! It is a testament to the elegance of the ModusToolbox design that you can completely disregard the fundamental underpinning of the entire structure, and it still works.

If you are new to ModusToolbox, or you want to dive into some aspect of the tool that you aren’t familiar with, check out our ModusToolbox 101 training material on GitHub. This material will update for version 2.2 within a day or two of release. The table of contents for this course is awesome. So is the content.

Cheers, and happy coding!

7 Comments
Len_CONSULTRON
Level 9
1000 replies posted 750 replies posted 500 likes received
Level 9

James,

You are correct.  An export of a project is at least 100x smaller!  Hooray!

I have a potential issue though.

The design.modus file that defines the Project's BSP device configuration file is also in the shared directory.

How do I define a unique Device configuration per Project per BSP without modifying the shared design.modus source for other projects?

Len

JamesT_21
Moderator
Moderator 100 likes received 50 likes received 25 likes received
Moderator

I know the engineering team is actually pondering how to make that specific use case more transparent.

In the meantime, this will work - make the BSP library application-specific (i.e. local to the project). Then make the changes in the local copy. On the BSP tab of the Library manager, turn OFF the Shared box for the BSP. When you update, a copy of the BSP shows up inside your application's folder. The one in the shared folder remains unaffected. So if you have an app that uses the "default" you're good. If you have an app where you want to make changes to the default, make it local and then change.

If you want to create an entirely custom BSP, you can do that too, but if all you're doing is making a few tweaks, that's a lot of work for not a lot of gain. If that's of interest, see Make Your Own BSP for Fun and Profit

Len_CONSULTRON
Level 9
1000 replies posted 750 replies posted 500 likes received
Level 9

James,

Thank you for the reply and the link.

I posted this basic question on another discussion board and received similar responses. Re: In MTB 2.2, how do I create a Project and BSP unique .modus design file since the design.modus f...

As stated in the link, my assumption is that the design.modus file is the near equivalent to the TopDesign file in Creator.   Therefore, the design.modus is potentially mostly unique to the project and BSP targeted.   Using a common shared design.modus file appears to be potentially problematic if the shared file is modified for purposes of resources and IO configuration on a project-by-project basis. 

Other projects projects sharing the newly modified design.modus file could easily no longer work when re-built.

It is possible that I don't fully understand the new eco-system operation of the MTB 2.2.   Maybe this use-case has been thought out elegantly and is more seamless than the recommended "bells and whistles" of creating a custom BSP.

I agree with you that the shared lib concept espoused by the new 2.2 is more efficient when it comes to benefitting from static library code (within a release).   I rather not go back to the 2.1-style of downloading the entire lib structure into the project.

Len

0 Likes
JamesT_21
Moderator
Moderator 100 likes received 50 likes received 25 likes received
Moderator

Your assumption about the role of design.modus is spot on. IMHO you clearly understand how MTB2.2 works. Sharing the design.modus file within a BSP does not make sense if you want to tweak that design, and most people at some point want to tweak the design. Duh! And that risks breaking anything else that shares the design.

I'm going to get your insight into the engineering team, just so they see. This is in fact something under discussion. You are not alone. Thank you for the feedback. Much appreciated.

0 Likes
Len_CONSULTRON
Level 9
1000 replies posted 750 replies posted 500 likes received
Level 9

James,

It's a common practice in 'C' coding to 'local' variable in a function with the same name as a global.  At compile-time, the local variable is used in the function as a first priority over the global.   I realize this can lead to confusion when debugging is needed.

What if project/BSP-specific files default to the global (shared location).   Once a modification occurs (for example Device Configurator changes the design.modus) it gets placed in the local project/BSP directory.

When the build occurs, it looks for local copies of the design files first.  If not present, it defaults to the global copies.   An export would include only local project/BSP copies.

Just a thought.

Len

0 Likes
JamesT_21
Moderator
Moderator 100 likes received 50 likes received 25 likes received
Moderator

Thanks Len!. The "standard practice" is evil code writing But a slick trick. The BSP suggestion, passed specifically into the team. Thanks again!

0 Likes
GregoryL_86
Moderator
Moderator 5 sign-ins First comment on blog First solution authored
Moderator

Hi Len,

There is a method in ModusToolbox to exclude the design.modus from the BSP so that it can be replaced with one that is application specific. It uses a feature of the make system whereby a directory that starts with COMPONENT can be included or excluded from a build via a make variable. A short summary:

1. In each BSP, the design configuration information is in a folder called COMPONENT_BSP_DESIGN_MODUS. Note that this contains the deisgn.modus file and may include other configurator files such as design.capsesnse.

2. In the applications makefile, add the following variable which will exclude anything from that directory:

DISABLE_COMPONENTS += BSP_DESIGN_MODUS.

3. Copy all of the configurator files that you need from that directory to your application. If you are not using something (e.g. capsense) then you don't need to copy that file.

4. Once you do this, the application will find the local design.modus file since the other one has been excluded. Note that if you make these changes inside the Eclipse IDE you may need to refresh the quick panel before the "device configurator" button will open the correct file. I would recommend that you always look at the banner at the top of the device configurator to make sure it opened the correct file.

The COMPONENT mechanism can and is used for lots of other things in ModusToolbox and is available to the user if you want to be able to control what gets included/excluded from different builds.

We have some online training material that you might find useful at this site:

GitHub - cypresssemiconductorco/CypressAcademy_MTB101_Files

Chapter 5a, section 5a.8.3 discusses what I described above but there may be useful info in the other chapters/sections as well. I hope this is helpful.

Greg

About the Author
Been there, done that. Mostly. For software tools and developer support.