Chapter 1: Plug-in programing from past to the future
It is the best time. Although the internal API of Android not allowed to be modified by google play, but in China, we download App in the domestic markets, and Google Play is rarely used. The major application markets are allow this. Therefore, each major companies of domestic internet industry has their own plug-in frameworks. With this open environment, we can do many things with Android plug-in framework.
It is the worst time. With the popularity of plug-in technology in China, for the interviews with major internet companies, plug-in technology will be general topic to be asked about. It is difficult for the Android developers not to be familiar with ‘reflection’ and ‘newProxyInstance’, but to master the principles of ‘ClassLoader’ and ‘Resources’.
1.1What is the plug-in？
A game center, such as Lianzhong in China, supports hundreds of games such as Chess, Bridges, and Texas hold 'em poker. The game platform is made of all games which have hundreds of megabytes. It takes a long time to download them all, but users tend to play only one or two of them. It is improper for users to download hundreds of games that they do not play.
In addition, any game update or release requires re-downloading hundreds of megabytes of installation packages, which is also a maddening event for users.
Therefore, the game platform usually adopts plug-in technology.
The common practice is to allow users to download only installation package with a dozen of megabyte, which includes the game center and the most popular games, such as Texas Hold 'em Poker. Users enter the game center, and get the game list, click on item named "Chess" to download Chess game plug-in, it is called "on-demand download".
This is plug-in programming, and based on the game platform for the personal computer, such plug-in is an executable file with extension file name as ".exe" or ".dll".
In the Android world, there is no such file as exe or dll. All files are compressed into an apk file and downloaded locally. App installation process of Android platform, in fact, just put the apk file into a local directory, then, use PMS to read the permission information and the information of four major components. So plug-in programming in the Android is not the same as that of software application on a computer.
In fact, for the game in the Android world, the usage of the plug-in technology is not a real plug-in framework, it is just the dynamic delivery of the script from the server, according to the information in the script, modify the character attributes, add props and maps of the game.
Android plug-in technology, mainly used in news, e-commerce, reading, traveling, video, music and other applications. For example, a traveling App, it involves the order and payment, and can be regarded as a branch of e-commerce, it can be split into independent plug-ins such as hotels, air tickets and train tickets, etc.
1.2 Why need pluggable?
When Android programmers write new features, bugs, or even crashes will exits in their App.
Once a traveling App has a bug, which will cause the user can not to put an order. With the crash, the user even can not get the order page. When such happens, we have to fix these issues as fast as we can, and re-publish to all kinds of Android market, it has been too late, the loss of business in every second, this is why use the Android plug-in programing to fix such issues. Users do not need to download full App again, and the updated versions of plug-in can be enjoyed in several minutes.
On the other hand, if we want to expand the market and compete with others, the sooner the new feature released, the higher rate of the coverage of market and user. If you publish a new release to the Android market frequently, the user will be annoyed. If the release cycle is fixed to half a month, which will lead to a long backlogs of new features, half a month later, the competitor may also release the same features. At such situation, if there is a plug-in technology support, once the new feature can be finished, the use will use it immediately, it is a best way for competition.
1.3 History of Android Plug-in Programing
In July 27, 2012 , it was the first milestone in Android plug-in technology. Yimin Tu(mmin18 on Github), who worked for dianping.com , released the first Android plug-in open source project——AndroidDynamicLoader, and the App of dianping.com was based on this framework. It is a plug-in framework based on Fragment. Through the dynamic loading of the Fragments in the plug-in, the page switching was implemented in the APP, but there is only one Activity which as the container of Fragments. In the open source project, it is the first time for us to know how to use the addAssetPath method of AssetManager to handle the resources in the plug-in.
In 2013, 23Code appeared. 23Code provides a shell where plug-ins can be dynamically downloaded and run. We can write a variety of controls outside and run them in the framework. It is the Android plug-in framework. For about this project and related open source resources, I don’t know very clear,if anyone knows the author, please contact with me.
On March 27, 2013, BoKui, a developer of Taobao client, provide a technical sharing about Taobao's Atlas plug-in framework in the Alibaba Technology Salon No.16, including several classes of Hook, incremental update, downgrade, and compatibility such as ActivityThread and so on. Such video shows the plug-in technology from macroscopic view, and not to say how to implement it, not to mentioned as open source projects. This thinking is very advanced in 2013, but at that time, I’m still a student for Android programing.
At 8:20 on March 30, 2014, it was the second milestone of Android plug-in technology. Mr. Ren Yugang published an Android plug-in project named dynamic-load-apk, which is not the same as many other plug-in projects. It does not modify with the internal methods of the Android system, but to solve the problem from the application layer of the App by creating a class named ProxyActivity to distribute and start the plug-in Activity. Because Ren Yugang invented a keyword called "that” in this framework, so I call it as "that" framework in this book. In fact, the author does not like the nickname, and named it as DL. When he develop this framework, there were so many difficulties, because there was no enough information on plug-in technology that could be referred to, especially before 2014.
"That" framework only have the implementation of the plug-in activity at beginning. With the contribution of Xiao Tian and Siyu Song, the implementation of Service was available later. In April 2015, "that" framework was stabilized. At that time, I was the leader of App Technology at Tuniu. When I ever saw this framework, decided to introduce it into the Tuniu App resolutely (actually there was no other choice at that time). The developers of the specific operation were Liangliang Wang and Zhengbin Wei. They took the framework and transplanted it into the Tuniu App. The framework has withstood the verification of a million-activiation app. It is the first market App for plug-in technology.
At the same time, Tao Zhang was also contribute to the implementation of plug-in technology. In May 2014, after reading all the source codes of DL, he released his first plug-in framework——CJFrameForAndroid. This design is similar to "that", except that he called ProxyActivity and ProxyService as hosting sites. In addition, the CJFrameForAndroid framework also presents the Activity's LaunchMode solution, which is a very important contribution to the plug-in framework and can be ported directly to "that" framework.
In November 2014, houkx released a plug-in project named android-pluginmgr on GitHub. This framework first proposed to register a StubActivity in the AndroidManifest file to deceive AMS, but actually opened ActivityA in the plug-in. But he did not use the technology to modify with Instrumentation and ActivityThread, but through the dexmaker.jar tool dynamically generated StubActivity, StubActivity class inherited from the plug-in ActivityA.
Today, it seems that the idea of such dynamically generated classes does not a good solution for plug-in framework, but it was not easy to acquire this point at that time. Everyone interprets their roles well in the stage of plug-in and plays a specific role at a particular time.
At the same time, hookx also found that the permissions requested in the plug-in framework can not take effect, so you must apply for all permissions in the host app in advance.
Android-pluginmgr has two branches, the author's idea of plug-in is located in the dev branch. Then, Lody, a high school student, took part in the open source project, and modified android-pluginmgr with Instrumentation thoughts, implemented in the master branch, but this story was happened in November 2015.
On December 8, 2014, there was a good news, Android Studio V1.0 was available. Android developers began to abandon Eclipse gradually, and try to use Android Studio. Android Studio is compiled and packaged with Gradle, which makes the design of the plug-in framework much simpler, eliminating the inconvenience of using Eclipse to run the Android SDK.
2015 was coming.
Lody, the high school student, he was still a sophomore at the moment. He started studying the Android system source codes from junior school. His first well-known open source project was TurboDex, which can quickly load dex with high speed. This is very useful in plug-in frameworks because it takes a long time to load all the plug-ins for the initialization.
At the end of March 2015, Lody released the plug-in project: Direct-Load-apk. This framework combines Yugang Ren's static proxy thoughts of “that“ framework, Houkx's pluginmgr framework whose idea is deception AMS, and modified Instrumnetation. Unfortunately, Lody was a student at that time, and did not spend much time to talk about this framework, so that too many people did not know the existence of the framework.
The legend of Lody hasn't finished yet, he spend much time into the project of VirtualApp. It is an App just like virtual machine on the Android system, which is a deeper technical topic, and we'll talk it later.
In May 2015, limpoxe released plug-in framework: Android-Plugin-Framework.
In July 2015, kaedea released plug-in framework: android-dynamical-loading.
In August 27, 2015, it was the third milestone of Android plug-in technology, and Yong Zhang's DroidPlugin came out. Yong Zhang was a developer of the 360 Mobile Assistant App team, and DroidPlugin was the plug-in framework used by the mobile assistant App. The magic of this framework is that any App can be loaded into the host. You can write a host App based on this framework, and then you can load App written by others as plug-ins.
DroidPlugin puts the Hook to the highest.
DroidPlugin has a powerful features, but the cost is to modify with many internal code of Android. Due to no more documentation for the DroidPlugin project, this framework is difficult to be understood. There are many articles about DroidPlugin, but the best one is written by WeiShu Tian. At that time, he also worked for the same company with Yong Zhang, he wrote out a series of articles about DroidPlugin, including the principles of Binder and AIDL, the Hook mechanism, and the plug-in mechanism for the four major components.
2015 was the first year for Android plug-in development. Not only “that” framework and DroidPlugin, but also many plug-in frameworks were born at that time.
OpenAtlas, the project was released on Github in May and was renamed ACDD later. It proposes to modify and regenerate the AAPT command so that the resource id of the plug-in apk is no longer only fixed 0x7f, which can be modified to a value of 0x71. This solves the problem of resource ids conflicting after consolidating plug-in resources into HostApp resources in the host.
OpenAtlas is also based on ‘execStartActivity’ method, which was modified, in ‘Instrumentation’ which is belong to the internal code of Android. It can implementation the Activity as plug-in.
In addition, OpenAltas also modified the ContextWrapper, rewrites the getResource method. Because the Activity is the subclass of the ContextWrapper, the plug-in Activity will inherit the getResource methods to get the resources in the plug-in——this method is not be used now , we create a base class called BasePluginActivity for the plug-in Activity and override the getResource method to fetch plug-in resource.
Ctrip open source their plug-in framework DynamicAPK in October 2015, which is based on the OpenAltas framework and binds with Ctrip's own business logic.
At the end of December 2015, Guangliang Lin released the Small framework. At that time, he worked for a car trading platform at Fujian. This framework was tailored to the App for the car trading platform.
In 2015, Android hot fix technology and React Native appeared in the developer's world, provided the same advantages as Android plug-in technology. Android plug-in technology is no longer the only choice for developers.
Since 2016, domestic Internet companies have gradually opened their own plug-in framework. No more technology has been adopted at this time, because all the solutions of the plug-in technology had been given by individual developers in 2015. Internet companies are the best platform to verify the feasibility of these plug-in technologies because their apps are used by millions of users.
Let us enumerate them in chronological order:
In August 2016, Palmeras launched Zeus.
In March 2017, Alibaba launched Atlas.
June 26, 2017, 360 mobile phone launched RePlugin.
On June 29, 2017, Didi launched VisualApk.
After reading the source code of these frameworks carefully, we can find that such open source frameworks are focus on:
With the mature of plug-in framework, many authors of plug-in technology have begun to change their focus. Some of them are still in the area of Android. For example, based on the DroidPlugin framework, Yong Zhang is doing his startup project——lightning boxes based on DroidPlugin framework; some for the Blockchain, they focus on writing smart contracts with GO language every day.
Thanks to those who have contributed in the field of plug-in technology, including the authors of the open source project, as well as authors who wrote the article on preaching. If some people may have some frameworks that may not be mentioned, any suggestions are welcome.
1.4 The usage of plug-in
Once upon a time, we once thought naively that Android plug-in was intended to add new features or add a complete module.
It took a lot of time and effort. After the project was implemented with the plug-in framework, we discovered that 80% usage of the plug-in is to fix online bugs. At this point, it has the same capabilities as hot fix tools such as Tinker and Robust, and even better than hot fix tools.
The App released a new version every two weeks. Generally, the new feature goes online at this point. On the other hand, the release strategy of many companies' Android App are affected by the new release of the iOS platform. New features need to wait for these two versions to be released. It is only when the Apple Store put the version of the iOS online, then the App of Android can be released. It is really not so busy to do.
In the past without plug-in framework, we did all kinds of development. We were scared to write bugs, if serious bug appeared, we must re-issue the upgraded version. With the plug-in framework, developers have no worries about the quality of the code ——anyway, if wrong, we can send a plug-in package to fix it quickly. After the app released, each plug-in will have one or two new releases every day.
The Android plug-in framework has become to a bug-fixing tool. This is the scene that we do not want to see.
In fact, the plug-in framework is more suitable for the games. For example, the King Glory, there will always be new skins, or a new hero role available every few days, even adjusting the roles attributes, all of these do not need to be released as a new version.
There is another use case for plug-in technology, that is ABTest, but it is not used widely. When the product manager want to determine which one will be selected for two styles of design, there will be two strategies which are made into the two plug-in packages, so that 50% of the users download the strategy A, the other 50% of the users download the strategy B. Checking the result after a week, such as the page conversion rate , it will tell you which strategy is better. That is Data-driven product.
In the previous section, it was mentioned about the componentization of Android. That is, with the independence of the business unit, the Android and iOS teams are split into their own business and have their own organization relationships. Therefore, it is necessary to split the different services of hotel ticket and train tickets into different modules. In componentization of Android, modules are still dependent on AAR, we can use Maven to manage these AAR.
This componentized model of Android is only applicable to the development stage. Once there is a bug in the on-line release or new features are to be released, all modules must be repackaged together to deploy the new release.
If we want to go further, then plug-in technology appeared. At this point, each service module is no longer AAR, but a packaged apk file, placed in the assets directory of the host App. In this way, after a release version, some module is updated, only the code of this module is repackaged, an incremental package is generated, and it is put on the server for the user to download.
It is the value of Android plug-in in enterprise application development. The small company only used the Android componentization and did not go into plug-in technology, such benefit was not acquired because of high development costs and low input-output ratio.
1.5 A better alternative thing: React Native
In 2015, React Native (RN) was born. At that time, few people paid attention to it because it was still immature with even few basic controls. Then, with the iteration of the RN project, the functions became more and more perfect. Although there has not been a release version 1.0 so far, we are delighted to find that it is the plug-in technology of Android and iOS.
Different person has different thinking.
When we put a lot of energy to study how to modify with the Android operation system source codes, they are taking another roadmap, it is to do the mapping, so that each control in Android or iOS, can find the corresponding control in the RN.
The RN is written based on JavaScript, packaged and put on the server for Android and iOS apps to download and use.
What makes RN better than Android plugins is that it also supports iOS, which maximizes cross-platform implementation. When we thought it was easy to use Android plug-in, and how to release new features to iOS, RN is a better choice.
For the performance, the difference between these two patterns is not significant. RN is very smooth on iOS and Android. Don't worry about this.
For small-sized or medium-sized companies and startups, they have no human resources and financial resources to develop a plug-in framework, and generally adopt a relatively stable, open source, and continuous updated plug-in framework in their country. However, it seems that iOS has no technical framework, especially after the JsPatch hot fix was forbidden by the AppStore. Their best choice is RN. Once the JavaScript is recruited, it will be able to quickly iterate and release. I do the code programming before, and do training now, there are hundreds of companies in China, including large state-owned enterprises, second-tier, third-tier Internet companies, and traditional industries. Found that 90% of companies in the country are in such situation: the demand for RN is far greater than Android plug-in technology.
For about RN, there are already some books available. This book mainly introduces Android plug-in technology. This section shows some points that Android plug-in is not as good as RN.
1.6 Why only in China?
Android plug-in is in full swing in China, why is it silent in foreign countries?
The applications of those unicorns in Silicon Valley did not find plug-in.
On the one hand, foreigners use Google Play. This official market does not allow plug-ins to exist. If there are plug-in frame in the Apps, it will not pass the review. This is similar to the Apple Store.
On the other hand, there may be no such requirements for foreigners.
When you find a foreign App display data is wrong, or somehow crashed, even if you feed back to them, you will get a graceful reply, such as the next version and then fix it. When is the next version? It will be several weeks later.
This is different from the situation in China. In a first or second-tier Internet company, any data display errors, or crashes, will result in a decrease in the number of orders, affecting the revenue directly. Therefore, the engineers often wake up at midnight to fix bugs, and then quickly release new versions of the plug-in package to avoid further loss of orders.
For first-tier and second-tier Internet companies in China will spend a lot of resource to support a team who are doing plug-in frameworks. After the framework is designed, they will not have heave work to do. When Android releases a new version every year, they will be very busy to study what Android system source code changes in the new version, and does it will have any impact on the plug-in framework of the company. In the long term, the company spends such resources on a cost-effective basis, which is almost equal to the amount of orders lost without plug-ins.
For the small or start-up companies in China, they do not have the enough financial resources to do their own plug-in framework, and generally adopt a relatively stable, open source, and continuous updated plug-in framework in China. Then, the RN was born, they switched to the RN's family.
Excellent engineers in China have launched their own Android plug-in framework. How about the foreign engineers doing at that time?
Foreign engineers are more concerned with the user experience. So you can find material design is popular in foreign countries, and in China, the designer is only to design a sample of iOS, for Android, to do the same is enough.
Foreign engineers are more concerned with functional programming. They are pursuing elegance, practicality, robustness, and reuse of codes. Unlike apps in China, they often rely on human resources to pile up codes and even bring bugs to the market. When refactoring, and the man who wrote the codes at the time may have left.
Therefore, when engineers in Silicon Valley have released ButterKnife, Dagger, AspectJ, OKHttp, Retrofit, and RxJava, engineers in China can only come up with a variety of plug-in frameworks, hot fix frameworks, and double-opening (open 2 application instances on a mobile) technologies.
1.7 All components require plug-in?
In Android, Activity, Service, ContentProvider, and BroadcastReceiver are called as four major components.
Do all four components need convert to plug-ins? Over the years, I have been working on plug-in technology with this question.
Several companies I worked for are OTA (Online Travelling) industry. This kind of app is similar to e-commerce, including a complete set of payment processes, Activity is used mostly, two or three hundred of Activities are not surprising, and Service and Receiver rarely used, ContentProvider is not in use.
Most apps in China are like this.
According to the technology stack, the App industry is divided into:
1.8 Double-Opening and Virtual Machine
Since plug-in will be replaced by RN as following years, what is the future of plug-in? the answer is virtual machine technology.
Some engineers already have experience of installing a virtual machine on a PC. As long as the computer's memory is large enough, you can open multiple virtual machines at the same time. On each virtual machine, install QQ software, log in with a different account, and then chat with yourself. Of course, chat with yourself is doesn’t make any sense.
Can it also support installing one or more virtual machines on an Android system? Someone in China is already doing such work, as what I known, the one who is a high school student ——Lody, when this book is written, he should be already a college student. He has a very famous open source project VirtualApp. This project is in commercial operation now; the other is Yong Zhang, the author of DroidPlugin, he is now full-time entrepreneurs to do this, based on the DroidPlugin, developed a lightning box, you can quickly load a variety of apk through it.
With such a virtual machine system, we can open two QQ application with different account on the phone and chat with ourselves.
The multiple instances technology that opens an app at the same time called Double-Opening. Some mobile phone systems in China now support double-opening technology. You can see this option in the settings of the Android phone.
Regarding to the technology of double-opening and virtual machines, there is only a brief about them. After all, it is out of this book scope.
1.9 From Native to HTML5
The mobile technology has become more and more mature, and it has evolved from a ridiculous situation in 2012 to now. For the Chinese engineers, we are more concerned about such a few points, hotfix, performance, development efficiency, App size and data-driven products. These aspects have already had very good solutions at present, but also emerged some excellent framework like RxJava, CanaryLeak. This topic is very big, and also out of this book scope.
By the immense prosperity of App technology, I recall my first job in 2004, the IT industry was transitioning from CS to BS. CS is a Client/Server architecture. For example, if you install a client of Lianzhong game center on Windows system, you can get the client of Texas Hold'em poker. After the technology of the Internet grew up, you start to move the original system into the website. This is BS, the full name is Browser/Server architecture.
To compare with CS, BS was a thin client, many features were not supported by BS. It came up with the concept of a SmartClient, which is a smart client. Outlook is a good sample. You can read and write E-mail offline without network, when the network is available, it send a written e-mail automatically.
Then, Flash was become hot. This was a tool for three webmasters originally, but it became the originator of the web-rich client. Based on Flash, there will be Flex, and now some companies are adopted Flex. At this time, Microsoft also came in and created a Silverlight, which is also similar to Flash on the web. At the same time, JavaScript is also working hard and gradually replacing the former and becoming the final winner of the rich client. A book was very popular at that time, it is called "JavaScript Design Patterns."
JavaScript was only used to do web visual effects in 2004. Ten years later, JavaScript has experienced with ajax, jQuery, ECMAScript from 1 to 6, Webpack packaging, and so on. The frameworks of Angular, React, Vue have become extremely powerful and have been packaged due to javascript as a "object-oriented" language.
Compared with web technology, the App is also taking the same development path. First, it mute for a few years. Many of the technologies on the website are moved to the App, just like what we do today. The next stage is the transition from CS to BS. The Hybird technology is the above-mentioned BS, but there are many defects, especially the bad performance of Web browser, then there are React Native, HTML 5 is also slow, but you can translate HTML 5 into Native code . It is not clear for me to make such kind of progress, but the future is very clear. Android and iOS technology will not die; on the other hand, HTML 5 will become the main method of App development in following years.
1.10 Summery
In this chapter, we review the history of Android plug-in technology which is basically divided into two parts, static agents and dynamic replacement. All plug-in frameworks are based on them. After the history review, you will find that plug-in technology is not accomplished at one stroke, and it has also experienced a process of gradual improvement from scratch.
Plug-in technology is not only used to fix bugs and dynamically release new features. In the process of researching plug-in technology, we have developed the Android virtual machine and double-opening technology. This is a new technology area that can get rid of the Android native system limitation, and run the app more faster.
React Native is also mentioned, which also fixes bugs and dynamically releases new features, similar to Android plug-in technology. Which technology to be selected depends on whether the R&D team is based on H5 or Android, and depends on whether it will released to Google Play.
It is the best time. Although the internal API of Android not allowed to be modified by google play, but in China, we download App in the domestic markets, and Google Play is rarely used. The major application markets are allow this. Therefore, each major companies of domestic internet industry has their own plug-in frameworks. With this open environment, we can do many things with Android plug-in framework.
It is the worst time. With the popularity of plug-in technology in China, for the interviews with major internet companies, plug-in technology will be general topic to be asked about. It is difficult for the Android developers not to be familiar with ‘reflection’ and ‘newProxyInstance’, but to master the principles of ‘ClassLoader’ and ‘Resources’.
1.1What is the plug-in？
A game center, such as Lianzhong in China, supports hundreds of games such as Chess, Bridges, and Texas hold 'em poker. The game platform is made of all games which have hundreds of megabytes. It takes a long time to download them all, but users tend to play only one or two of them. It is improper for users to download hundreds of games that they do not play.
In addition, any game update or release requires re-downloading hundreds of megabytes of installation packages, which is also a maddening event for users.
Therefore, the game platform usually adopts plug-in technology.
The common practice is to allow users to download only installation package with a dozen of megabyte, which includes the game center and the most popular games, such as Texas Hold 'em Poker. Users enter the game center, and get the game list, click on item named "Chess" to download Chess game plug-in, it is called "on-demand download".
This is plug-in programming, and based on the game platform for the personal computer, such plug-in is an executable file with extension file name as ".exe" or ".dll".
In the Android world, there is no such file as exe or dll. All files are compressed into an apk file and downloaded locally. App installation process of Android platform, in fact, just put the apk file into a local directory, then, use PMS to read the permission information and the information of four major components. So plug-in programming in the Android is not the same as that of software application on a computer.
In fact, for the game in the Android world, the usage of the plug-in technology is not a real plug-in framework, it is just the dynamic delivery of the script from the server, according to the information in the script, modify the character attributes, add props and maps of the game.
Android plug-in technology, mainly used in news, e-commerce, reading, traveling, video, music and other applications. For example, a traveling App, it involves the order and payment, and can be regarded as a branch of e-commerce, it can be split into independent plug-ins such as hotels, air tickets and train tickets, etc.
1.2 Why need pluggable?
When Android programmers write new features, bugs, or even crashes will exits in their App.
Once a traveling App has a bug, which will cause the user can not to put an order. With the crash, the user even can not get the order page. When such happens, we have to fix these issues as fast as we can, and re-publish to all kinds of Android market, it has been too late, the loss of business in every second, this is why use the Android plug-in programing to fix such issues. Users do not need to download full App again, and the updated versions of plug-in can be enjoyed in several minutes.
On the other hand, if we want to expand the market and compete with others, the sooner the new feature released, the higher rate of the coverage of market and user. If you publish a new release to the Android market frequently, the user will be annoyed. If the release cycle is fixed to half a month, which will lead to a long backlogs of new features, half a month later, the competitor may also release the same features. At such situation, if there is a plug-in technology support, once the new feature can be finished, the use will use it immediately, it is a best way for competition.
1.3 History of Android Plug-in Programing
In July 27, 2012 , it was the first milestone in Android plug-in technology. Yimin Tu(mmin18 on Github), who worked for dianping.com , released the first Android plug-in open source project——AndroidDynamicLoader, and the App of dianping.com was based on this framework. It is a plug-in framework based on Fragment. Through the dynamic loading of the Fragments in the plug-in, the page switching was implemented in the APP, but there is only one Activity which as the container of Fragments. In the open source project, it is the first time for us to know how to use the addAssetPath method of AssetManager to handle the resources in the plug-in.
In 2013, 23Code appeared. 23Code provides a shell where plug-ins can be dynamically downloaded and run. We can write a variety of controls outside and run them in the framework. It is the Android plug-in framework. For about this project and related open source resources, I don’t know very clear,if anyone knows the author, please contact with me.
On March 27, 2013, BoKui, a developer of Taobao client, provide a technical sharing about Taobao's Atlas plug-in framework in the Alibaba Technology Salon No.16, including several classes of Hook, incremental update, downgrade, and compatibility such as ActivityThread and so on. Such video shows the plug-in technology from macroscopic view, and not to say how to implement it, not to mentioned as open source projects. This thinking is very advanced in 2013, but at that time, I’m still a student for Android programing.
At 8:20 on March 30, 2014, it was the second milestone of Android plug-in technology. Mr. Ren Yugang published an Android plug-in project named dynamic-load-apk, which is not the same as many other plug-in projects. It does not modify with the internal methods of the Android system, but to solve the problem from the application layer of the App by creating a class named ProxyActivity to distribute and start the plug-in Activity. Because Ren Yugang invented a keyword called "that” in this framework, so I call it as "that" framework in this book. In fact, the author does not like the nickname, and named it as DL. When he develop this framework, there were so many difficulties, because there was no enough information on plug-in technology that could be referred to, especially before 2014.
"That" framework only have the implementation of the plug-in activity at beginning. With the contribution of Xiao Tian and Siyu Song, the implementation of Service was available later. In April 2015, "that" framework was stabilized. At that time, I was the leader of App Technology at Tuniu. When I ever saw this framework, decided to introduce it into the Tuniu App resolutely (actually there was no other choice at that time). The developers of the specific operation were Liangliang Wang and Zhengbin Wei. They took the framework and transplanted it into the Tuniu App. The framework has withstood the verification of a million-activiation app. It is the first market App for plug-in technology.
At the same time, Tao Zhang was also contribute to the implementation of plug-in technology. In May 2014, after reading all the source codes of DL, he released his first plug-in framework——CJFrameForAndroid. This design is similar to "that", except that he called ProxyActivity and ProxyService as hosting sites. In addition, the CJFrameForAndroid framework also presents the Activity's LaunchMode solution, which is a very important contribution to the plug-in framework and can be ported directly to "that" framework.
In November 2014, houkx released a plug-in project named android-pluginmgr on GitHub. This framework first proposed to register a StubActivity in the AndroidManifest file to deceive AMS, but actually opened ActivityA in the plug-in. But he did not use the technology to modify with Instrumentation and ActivityThread, but through the dexmaker.jar tool dynamically generated StubActivity, StubActivity class inherited from the plug-in ActivityA.
Today, it seems that the idea of such dynamically generated classes does not a good solution for plug-in framework, but it was not easy to acquire this point at that time. Everyone interprets their roles well in the stage of plug-in and plays a specific role at a particular time.
At the same time, hookx also found that the permissions requested in the plug-in framework can not take effect, so you must apply for all permissions in the host app in advance.
Android-pluginmgr has two branches, the author's idea of plug-in is located in the dev branch. Then, Lody, a high school student, took part in the open source project, and modified android-pluginmgr with Instrumentation thoughts, implemented in the master branch, but this story was happened in November 2015.
On December 8, 2014, there was a good news, Android Studio V1.0 was available. Android developers began to abandon Eclipse gradually, and try to use Android Studio. Android Studio is compiled and packaged with Gradle, which makes the design of the plug-in framework much simpler, eliminating the inconvenience of using Eclipse to run the Android SDK.
2015 was coming.
Lody, the high school student, he was still a sophomore at the moment. He started studying the Android system source codes from junior school. His first well-known open source project was TurboDex, which can quickly load dex with high speed. This is very useful in plug-in frameworks because it takes a long time to load all the plug-ins for the initialization.
At the end of March 2015, Lody released the plug-in project: Direct-Load-apk. This framework combines Yugang Ren's static proxy thoughts of “that“ framework, Houkx's pluginmgr framework whose idea is deception AMS, and modified Instrumnetation. Unfortunately, Lody was a student at that time, and did not spend much time to talk about this framework, so that too many people did not know the existence of the framework.
The legend of Lody hasn't finished yet, he spend much time into the project of VirtualApp. It is an App just like virtual machine on the Android system, which is a deeper technical topic, and we'll talk it later.
In May 2015, limpoxe released plug-in framework: Android-Plugin-Framework.
In July 2015, kaedea released plug-in framework: android-dynamical-loading.
In August 27, 2015, it was the third milestone of Android plug-in technology, and Yong Zhang's DroidPlugin came out. Yong Zhang was a developer of the 360 Mobile Assistant App team, and DroidPlugin was the plug-in framework used by the mobile assistant App. The magic of this framework is that any App can be loaded into the host. You can write a host App based on this framework, and then you can load App written by others as plug-ins.
DroidPlugin puts the Hook to the highest.
DroidPlugin has a powerful features, but the cost is to modify with many internal code of Android. Due to no more documentation for the DroidPlugin project, this framework is difficult to be understood. There are many articles about DroidPlugin, but the best one is written by WeiShu Tian. At that time, he also worked for the same company with Yong Zhang, he wrote out a series of articles about DroidPlugin, including the principles of Binder and AIDL, the Hook mechanism, and the plug-in mechanism for the four major components.
2015 was the first year for Android plug-in development. Not only “that” framework and DroidPlugin, but also many plug-in frameworks were born at that time.
OpenAtlas, the project was released on Github in May and was renamed ACDD later. It proposes to modify and regenerate the AAPT command so that the resource id of the plug-in apk is no longer only fixed 0x7f, which can be modified to a value of 0x71. This solves the problem of resource ids conflicting after consolidating plug-in resources into HostApp resources in the host.
OpenAtlas is also based on ‘execStartActivity’ method, which was modified, in ‘Instrumentation’ which is belong to the internal code of Android. It can implementation the Activity as plug-in.
In addition, OpenAltas also modified the ContextWrapper, rewrites the getResource method. Because the Activity is the subclass of the ContextWrapper, the plug-in Activity will inherit the getResource methods to get the resources in the plug-in——this method is not be used now , we create a base class called BasePluginActivity for the plug-in Activity and override the getResource method to fetch plug-in resource.
Ctrip open source their plug-in framework DynamicAPK in October 2015, which is based on the OpenAltas framework and binds with Ctrip's own business logic.
At the end of December 2015, Guangliang Lin released the Small framework. At that time, he worked for a car trading platform at Fujian. This framework was tailored to the App for the car trading platform.
- Corresponding to the plug-in's ClassLoader, the Small framework inserted the dex file into the Host App's ClassLoader , so that Host App can load any class of any plug-in.
- The Small framework is an Activity, which starts the plug-ins through Hook Instrumentation. It is the same as the DroidPlugin, a StubActivity is declared in the AndroidManifest to trick the AMS.
- To support for the other three components in the Small framework is to declare the plug-in's Service, Receiver, and ContentProvider in HostApp's AndroidManifest.
- For the resources, the Small's solution has its own method. Using AssetManager's addAssetPath() method, all the plug-in resources are merged into the host's resources, if the resource id conflict occurs, Small did not use Atlas to modify the AAPT, but in the plug-in R.java and resource. After these two files generation, just change the prefix of all resources in the plug-in R.java from 0x7f to 0x71, and also change 0x7f to 0x71 in resource.arsc.
In 2015, Android hot fix technology and React Native appeared in the developer's world, provided the same advantages as Android plug-in technology. Android plug-in technology is no longer the only choice for developers.
Since 2016, domestic Internet companies have gradually opened their own plug-in framework. No more technology has been adopted at this time, because all the solutions of the plug-in technology had been given by individual developers in 2015. Internet companies are the best platform to verify the feasibility of these plug-in technologies because their apps are used by millions of users.
Let us enumerate them in chronological order:
In August 2016, Palmeras launched Zeus.
In March 2017, Alibaba launched Atlas.
June 26, 2017, 360 mobile phone launched RePlugin.
On June 29, 2017, Didi launched VisualApk.
After reading the source code of these frameworks carefully, we can find that such open source frameworks are focus on:
- Plug-in compatibility, including the impact of the upgrade of the Android system on the plug-in framework, and the impact of different mobile phone ROMs.
- Plug-in stability, for example, different kinds of unknown crashes.
- Management of plug-ins, including installation and uninstallation.
With the mature of plug-in framework, many authors of plug-in technology have begun to change their focus. Some of them are still in the area of Android. For example, based on the DroidPlugin framework, Yong Zhang is doing his startup project——lightning boxes based on DroidPlugin framework; some for the Blockchain, they focus on writing smart contracts with GO language every day.
Thanks to those who have contributed in the field of plug-in technology, including the authors of the open source project, as well as authors who wrote the article on preaching. If some people may have some frameworks that may not be mentioned, any suggestions are welcome.
1.4 The usage of plug-in
Once upon a time, we once thought naively that Android plug-in was intended to add new features or add a complete module.
It took a lot of time and effort. After the project was implemented with the plug-in framework, we discovered that 80% usage of the plug-in is to fix online bugs. At this point, it has the same capabilities as hot fix tools such as Tinker and Robust, and even better than hot fix tools.
The App released a new version every two weeks. Generally, the new feature goes online at this point. On the other hand, the release strategy of many companies' Android App are affected by the new release of the iOS platform. New features need to wait for these two versions to be released. It is only when the Apple Store put the version of the iOS online, then the App of Android can be released. It is really not so busy to do.
In the past without plug-in framework, we did all kinds of development. We were scared to write bugs, if serious bug appeared, we must re-issue the upgraded version. With the plug-in framework, developers have no worries about the quality of the code ——anyway, if wrong, we can send a plug-in package to fix it quickly. After the app released, each plug-in will have one or two new releases every day.
The Android plug-in framework has become to a bug-fixing tool. This is the scene that we do not want to see.
In fact, the plug-in framework is more suitable for the games. For example, the King Glory, there will always be new skins, or a new hero role available every few days, even adjusting the roles attributes, all of these do not need to be released as a new version.
There is another use case for plug-in technology, that is ABTest, but it is not used widely. When the product manager want to determine which one will be selected for two styles of design, there will be two strategies which are made into the two plug-in packages, so that 50% of the users download the strategy A, the other 50% of the users download the strategy B. Checking the result after a week, such as the page conversion rate , it will tell you which strategy is better. That is Data-driven product.
In the previous section, it was mentioned about the componentization of Android. That is, with the independence of the business unit, the Android and iOS teams are split into their own business and have their own organization relationships. Therefore, it is necessary to split the different services of hotel ticket and train tickets into different modules. In componentization of Android, modules are still dependent on AAR, we can use Maven to manage these AAR.
This componentized model of Android is only applicable to the development stage. Once there is a bug in the on-line release or new features are to be released, all modules must be repackaged together to deploy the new release.
If we want to go further, then plug-in technology appeared. At this point, each service module is no longer AAR, but a packaged apk file, placed in the assets directory of the host App. In this way, after a release version, some module is updated, only the code of this module is repackaged, an incremental package is generated, and it is put on the server for the user to download.
It is the value of Android plug-in in enterprise application development. The small company only used the Android componentization and did not go into plug-in technology, such benefit was not acquired because of high development costs and low input-output ratio.
1.5 A better alternative thing: React Native
In 2015, React Native (RN) was born. At that time, few people paid attention to it because it was still immature with even few basic controls. Then, with the iteration of the RN project, the functions became more and more perfect. Although there has not been a release version 1.0 so far, we are delighted to find that it is the plug-in technology of Android and iOS.
Different person has different thinking.
When we put a lot of energy to study how to modify with the Android operation system source codes, they are taking another roadmap, it is to do the mapping, so that each control in Android or iOS, can find the corresponding control in the RN.
The RN is written based on JavaScript, packaged and put on the server for Android and iOS apps to download and use.
What makes RN better than Android plugins is that it also supports iOS, which maximizes cross-platform implementation. When we thought it was easy to use Android plug-in, and how to release new features to iOS, RN is a better choice.
For the performance, the difference between these two patterns is not significant. RN is very smooth on iOS and Android. Don't worry about this.
For small-sized or medium-sized companies and startups, they have no human resources and financial resources to develop a plug-in framework, and generally adopt a relatively stable, open source, and continuous updated plug-in framework in their country. However, it seems that iOS has no technical framework, especially after the JsPatch hot fix was forbidden by the AppStore. Their best choice is RN. Once the JavaScript is recruited, it will be able to quickly iterate and release. I do the code programming before, and do training now, there are hundreds of companies in China, including large state-owned enterprises, second-tier, third-tier Internet companies, and traditional industries. Found that 90% of companies in the country are in such situation: the demand for RN is far greater than Android plug-in technology.
For about RN, there are already some books available. This book mainly introduces Android plug-in technology. This section shows some points that Android plug-in is not as good as RN.
1.6 Why only in China?
Android plug-in is in full swing in China, why is it silent in foreign countries?
The applications of those unicorns in Silicon Valley did not find plug-in.
On the one hand, foreigners use Google Play. This official market does not allow plug-ins to exist. If there are plug-in frame in the Apps, it will not pass the review. This is similar to the Apple Store.
On the other hand, there may be no such requirements for foreigners.
When you find a foreign App display data is wrong, or somehow crashed, even if you feed back to them, you will get a graceful reply, such as the next version and then fix it. When is the next version? It will be several weeks later.
This is different from the situation in China. In a first or second-tier Internet company, any data display errors, or crashes, will result in a decrease in the number of orders, affecting the revenue directly. Therefore, the engineers often wake up at midnight to fix bugs, and then quickly release new versions of the plug-in package to avoid further loss of orders.
For first-tier and second-tier Internet companies in China will spend a lot of resource to support a team who are doing plug-in frameworks. After the framework is designed, they will not have heave work to do. When Android releases a new version every year, they will be very busy to study what Android system source code changes in the new version, and does it will have any impact on the plug-in framework of the company. In the long term, the company spends such resources on a cost-effective basis, which is almost equal to the amount of orders lost without plug-ins.
For the small or start-up companies in China, they do not have the enough financial resources to do their own plug-in framework, and generally adopt a relatively stable, open source, and continuous updated plug-in framework in China. Then, the RN was born, they switched to the RN's family.
Excellent engineers in China have launched their own Android plug-in framework. How about the foreign engineers doing at that time?
Foreign engineers are more concerned with the user experience. So you can find material design is popular in foreign countries, and in China, the designer is only to design a sample of iOS, for Android, to do the same is enough.
Foreign engineers are more concerned with functional programming. They are pursuing elegance, practicality, robustness, and reuse of codes. Unlike apps in China, they often rely on human resources to pile up codes and even bring bugs to the market. When refactoring, and the man who wrote the codes at the time may have left.
Therefore, when engineers in Silicon Valley have released ButterKnife, Dagger, AspectJ, OKHttp, Retrofit, and RxJava, engineers in China can only come up with a variety of plug-in frameworks, hot fix frameworks, and double-opening (open 2 application instances on a mobile) technologies.
1.7 All components require plug-in?
In Android, Activity, Service, ContentProvider, and BroadcastReceiver are called as four major components.
Do all four components need convert to plug-ins? Over the years, I have been working on plug-in technology with this question.
Several companies I worked for are OTA (Online Travelling) industry. This kind of app is similar to e-commerce, including a complete set of payment processes, Activity is used mostly, two or three hundred of Activities are not surprising, and Service and Receiver rarely used, ContentProvider is not in use.
Most apps in China are like this.
According to the technology stack, the App industry is divided into:
- Game App. People have their own online update process. Many of them use scripts like Lua.
- Mobile assistants, mobile phone guards, and the use of such applications for Service, Receiver, and ContentProvider. Therefore, the plug-in of the four major components must be implemented.
- Music, video, and live video applications are very dependent on Service and Receiver, in addition to more activities.
- The e-commerce, social, news, and reading apps are basically Activities. The use of other three major components is not enough, and only the support for Activity and Service plug-ins can be considered.
1.8 Double-Opening and Virtual Machine
Since plug-in will be replaced by RN as following years, what is the future of plug-in? the answer is virtual machine technology.
Some engineers already have experience of installing a virtual machine on a PC. As long as the computer's memory is large enough, you can open multiple virtual machines at the same time. On each virtual machine, install QQ software, log in with a different account, and then chat with yourself. Of course, chat with yourself is doesn’t make any sense.
Can it also support installing one or more virtual machines on an Android system? Someone in China is already doing such work, as what I known, the one who is a high school student ——Lody, when this book is written, he should be already a college student. He has a very famous open source project VirtualApp. This project is in commercial operation now; the other is Yong Zhang, the author of DroidPlugin, he is now full-time entrepreneurs to do this, based on the DroidPlugin, developed a lightning box, you can quickly load a variety of apk through it.
With such a virtual machine system, we can open two QQ application with different account on the phone and chat with ourselves.
The multiple instances technology that opens an app at the same time called Double-Opening. Some mobile phone systems in China now support double-opening technology. You can see this option in the settings of the Android phone.
Regarding to the technology of double-opening and virtual machines, there is only a brief about them. After all, it is out of this book scope.
1.9 From Native to HTML5
The mobile technology has become more and more mature, and it has evolved from a ridiculous situation in 2012 to now. For the Chinese engineers, we are more concerned about such a few points, hotfix, performance, development efficiency, App size and data-driven products. These aspects have already had very good solutions at present, but also emerged some excellent framework like RxJava, CanaryLeak. This topic is very big, and also out of this book scope.
By the immense prosperity of App technology, I recall my first job in 2004, the IT industry was transitioning from CS to BS. CS is a Client/Server architecture. For example, if you install a client of Lianzhong game center on Windows system, you can get the client of Texas Hold'em poker. After the technology of the Internet grew up, you start to move the original system into the website. This is BS, the full name is Browser/Server architecture.
To compare with CS, BS was a thin client, many features were not supported by BS. It came up with the concept of a SmartClient, which is a smart client. Outlook is a good sample. You can read and write E-mail offline without network, when the network is available, it send a written e-mail automatically.
Then, Flash was become hot. This was a tool for three webmasters originally, but it became the originator of the web-rich client. Based on Flash, there will be Flex, and now some companies are adopted Flex. At this time, Microsoft also came in and created a Silverlight, which is also similar to Flash on the web. At the same time, JavaScript is also working hard and gradually replacing the former and becoming the final winner of the rich client. A book was very popular at that time, it is called "JavaScript Design Patterns."
JavaScript was only used to do web visual effects in 2004. Ten years later, JavaScript has experienced with ajax, jQuery, ECMAScript from 1 to 6, Webpack packaging, and so on. The frameworks of Angular, React, Vue have become extremely powerful and have been packaged due to javascript as a "object-oriented" language.
Compared with web technology, the App is also taking the same development path. First, it mute for a few years. Many of the technologies on the website are moved to the App, just like what we do today. The next stage is the transition from CS to BS. The Hybird technology is the above-mentioned BS, but there are many defects, especially the bad performance of Web browser, then there are React Native, HTML 5 is also slow, but you can translate HTML 5 into Native code . It is not clear for me to make such kind of progress, but the future is very clear. Android and iOS technology will not die; on the other hand, HTML 5 will become the main method of App development in following years.
1.10 Summery
In this chapter, we review the history of Android plug-in technology which is basically divided into two parts, static agents and dynamic replacement. All plug-in frameworks are based on them. After the history review, you will find that plug-in technology is not accomplished at one stroke, and it has also experienced a process of gradual improvement from scratch.
Plug-in technology is not only used to fix bugs and dynamically release new features. In the process of researching plug-in technology, we have developed the Android virtual machine and double-opening technology. This is a new technology area that can get rid of the Android native system limitation, and run the app more faster.
React Native is also mentioned, which also fixes bugs and dynamically releases new features, similar to Android plug-in technology. Which technology to be selected depends on whether the R&D team is based on H5 or Android, and depends on whether it will released to Google Play.
