It's pretty clear that Android development is a lot of people's first exposure to the world of Linux and programming. I'm pretty impressed with what people like dsmryder and Bsydz have been able to pull of with their limited experience. Hopefully I can help show you guys and anyone else who is interested a couple tips to make things a hell of a lot easier.
Please note this is not intended to be a complete newbie developer's guide. I assume you have the mtdev cm9 and possibly cm7 repos checked out, and that you can build it. Also, you should know the basics of Linux commands, which I assume you know since you've managed to get the build going.
Command Line Overview:
The command line in Linux is a lot more capable than a lot of people realize. The number one tip I can give you for the command line is learn to use and love tab completion. Commands and filenames can all be tab completed if you use the bash shell. This means rather than typing out ~/src/andoird/android-cm-7/ you simply start typing the first few letters, then press the tab key and the shell will attempt to complete it for you as best it can. If there are multiple options, it will stop, but then touble-tap the tab key to have the shell list the available expansions. Keep filling it out and tab expanding, and you will be typing a lot fewer letters to get the same commands very easily.
Probably the second biggest tip is that your shell keeps a history of your commands. Press up and down to go back through your command history. If you keep using a command, or a very similar command, just go back to it and edit the portion you need. You can scroll through the command line horizontally more quickly by holding control plus the arrow key to move by word. You can also search your command history by pressing control+r, then typing a few characters to search for. Press control-r again to search further back in your history. I know this sounds a bit complicated, but after using it for a while it will become second nature, and you'll wonder how you ever accomplished anything without it.
Now I need to explain how commands can be strung together. It's more complicated, but it's very useful and you can learn the simpler forms pretty easily. You string commands together with the pipe character, '|' (vertical bar, shift-backslash, usually above enter). What this does is take the output from one command, and use it as the input for another command.
Which brings me to the next big tip: less. The 'less' command is called a pager, it lets you easily move through and search through command output. For example, type "ls -lh /usr/lib". There will be a ton of output that floods your terminal. Now you can scroll through and read this all like a schmuck, or you can pipe it through less, and search it quickly to find what you need. Try "ls -lh /usr/lib | less" (remember the pipe key). Now you can use the page-up and page-down keys to move through the output. Use the '/' key to search forward (this is case sensitive), and the '?' key to search backward. When you are done, press 'q' to quit.
Finding files:
With a huge project like android, it can be difficult to find where files are. Linux has a find command, but it can be a little difficult to use, so I'll explain the more common aspects.
If you just type 'find' in the directory you are in, you'll get every file in that directory and all subdirectories. That's somewhat useful, but what we want to do is prune it down even further. For example, let's find QualcommCameraHardware.cpp in the mtdev cm9 repo: "find . -name QualcommCameraHardware.cpp". This will take a bit to run the first time, but things get cached into RAM if you have enough and should be faster next time.
That second parameter, the "." above, is where to start the find from. The "." means your current directory. There is also a ".." which means "one directory higher than I am right now". So if you are in android-cm9 and you and to quickly find something in android-cm7, you can use use "../android-cm7" rather than switching directories. You can find a specific directory if you know what you are looking for will be there, and it will be quicker. For example we could do "find hardware -name QualcommCameraHardware.cpp" and it would just look in the hardware directory, or "find ../android-cm7/hardware -name QualcommCameraHardware.cpp" for the cm7 repo. Also note, the "-name" is case sensitive. We'll get to case-insensitive later, it's a little difficult with find.
Searching for strings:
"grep" is the command to use to search for strings in files or command output. It is pretty advanced, but we don't need many of its features at all. grep will work on a file if you give it as the last parameter, but works on other commands' output by default. So, if you just type "grep searchforme", the command sits there waiting for input, and you will need to ctrl-c out of it. Try it with a file: "grep stopPreview hardware/qcom/camerahal/QualcommCameraHardware.cpp". Note that by default it is case sensitive, make it case insensitive with "-i": "grep -i stoppreview hardware/qcom/camerahal/QualcommCameraHardware.cpp" (you are remembering to use tab completion, right?).
You can also search through a directory and all it's content with the "-r" option: "grep -ri stoppreview hardware". If you really get desperate, grep for that string through the whole damn repo (remember you can use "." as the current directory), but it's will take a few minutes.
If you are dealing with binary files, the use the 'strings' command, and pipe the output to grep: "strings out/target/product/triumph/system/lib/liboemcamera.so |grep CAMERA_STOP". Remember too, you can pipe this string of commands or any other to less if the output is too long: "strings out/target/product/triumph/system/lib/liboemcamera.so |grep CAMERA_STOP | less".
Combining finding and searching:
If you know where you need to be searching for strings, combining find and grep can make searching very quick. First, let's learn another shell term: the wildcard (*). The wildcard matches any string. For example, try: "ls hardware/qcom/camerahal". Now try: "ls hardware/qcom/camerahal/*.cpp". It's as if you gave the ls command each individual string that ended with ".cpp". Well, let's use that with find: "find hardware/qcom/camerahal -name "*.cpp"". (Note that when using wildcards, you need to put your find term in quotes.) That's kind of a lame example since it's not any different than ls. But, let's find every C++ source file in the repo: "find . -name "*.cpp"". You can also put a wildcard in the middle: "find . -name "Camera*.cpp"
So now we've identified a set of files we are interested in, and we would like to search for something in them. Time to learn another command: xargs. This useful little guy runs a different program on the output of a given command. Sound complicated? It kind of is... but what if we would take the output of find, and run grep on each of those files? Pretty cool, right? Let's do it: "find . -name "Camera*.cpp" | xargs grep getPictureSize".
Honestly, this is a big chunk of what I use day to day in trying to quickly find things in a big project. There are of course ways to refine this further, there are other tools that can help tweak command output even further... but this is kind of a good base layer of knowledge, and you can always add to it as you learn more.
Finding differences between files:
To quickly tell if two files are the same, I like to use md5sum: "md5sum hardware/qcom/camerahal/QualcommCameraHardware.cpp ../android-cm7/hardware/qcom/camerahal/QualcommCameraHardware.cpp". This will give you a hash value for each file that will match if the files are bit for bit identical.
If the files don't match, and are source code files, you can then use the diff command to see what the differences are: "diff -u hardware/qcom/camerhal/QualcommCameraHardware.cpp ../android-cm7/hardware/msm7k/libcamera/QualcommCameraHardware.cpp". (Note the -u, this is for unified diff format. I recommend always using -u, it is much easier to read). Pipe this to less, or if you want to save to a file, use the redirect operator ">": "diff -u hardware/qcom/camerhal/QualcommCameraHardware.cpp ../android-cm7/hardware/msm7k/libcamera/QualcommCameraHardware.cpp > qualcomm.diff". (In fact, you can use this redirect operator to save the output of any command). Diff shows only the portions which are different, the header at the top describes the output. To make the two files the same, you would take out all the lines that start with '-', and add all the lines that start with '+'.
In fact, if you save this diff file, you can use it to convert one file to the other version, but that is slightly outside the scope of this guide. It's not hard to do though, I'm sure you can figure it out, just search about the patch command.
The diff command will also work recursively through subdirectories with "-r". Another useful option is "-N", which won't show the diff of files that aren't there (otherwise you get a + for every line of the new file). So maybe you use: "diff -rNu cm9-kernel/drivers/media/video/msm cm7-kernel/drivers/media/video/msm".
Dealing with Android:
You certainly know by now what a beast Android is. The good news is, you don't have to do a full rebuild for most changes, and you don't need to to a full flash for some changes. After you run "adb shell" to get a prompt on your phone, run: "mount -o remount,rw /system". This will remount the /system directory as read-write, so you can make changes on your live phone.
At this point, you can use "adb push" to copy files from your machine directly to your phone. For example, "adb push android-triumph/out/target/product/triumph/system/lib/libcamera.so /system/lib" will copy the local libcamera.so onto your camera's /system/lib.
When you run "brunch triumph" to do your build, pay attention to which files are actually being rebuilt. For example when working on the camera stuff, often there were just two files that would change on me: libcamera.so and camera.msm7x30.so. You will see the files that change scroll by in blue during the build. Cherry-picking these from the out directory saved me a lot of time. This same rule applies for the .apks that get built... if only one or a couple change it might be quicker to copy them individually rather than reflashing.
Note that you will not want to just pull the battery if you remount read-write, there may be a chance of corruption. Always do a software reboot to make sure things get written out properly. Or, if you adb shell in, you can run the "sync" command to force a write flush.
One other tip I have is for logcat. Rather than just running plain logcat, try "adb logcat -v time *" to get timestamps and extra verbose information. Remember, you can of course redirect to a file with our friend ">".
Text Editor:
Do yourself a favor and learn to use a real text editor. I strongly recommend either Emacs or vim, I'm an Emacs man myself. It probably has an easier initial learning curve, but I'm not going to cover it here there are plenty of newbie guides for text editors. Don't close your editor when you're done with a file, leave all those files loaded and it will make switching between them a lot easier.
Version Control:
I need to learn git myself to get the camera stuff checked in =]
One general comment: always always always put as much detailed information in your commit messages as possible. You (or someone else) will thank you later when you are trying to figure out why code was changed months ago.
KEEP TRACK OF EVERYTING
I cannot stress this enough. Ideally we could replay every keystroke but obviously that isn't manageable. Try to only change one thing at a time, and keep track of exactly what you are changing. Sometimes I keep a little text file log of ideas and things I have tried.
Please note this is not intended to be a complete newbie developer's guide. I assume you have the mtdev cm9 and possibly cm7 repos checked out, and that you can build it. Also, you should know the basics of Linux commands, which I assume you know since you've managed to get the build going.
Command Line Overview:
The command line in Linux is a lot more capable than a lot of people realize. The number one tip I can give you for the command line is learn to use and love tab completion. Commands and filenames can all be tab completed if you use the bash shell. This means rather than typing out ~/src/andoird/android-cm-7/ you simply start typing the first few letters, then press the tab key and the shell will attempt to complete it for you as best it can. If there are multiple options, it will stop, but then touble-tap the tab key to have the shell list the available expansions. Keep filling it out and tab expanding, and you will be typing a lot fewer letters to get the same commands very easily.
Probably the second biggest tip is that your shell keeps a history of your commands. Press up and down to go back through your command history. If you keep using a command, or a very similar command, just go back to it and edit the portion you need. You can scroll through the command line horizontally more quickly by holding control plus the arrow key to move by word. You can also search your command history by pressing control+r, then typing a few characters to search for. Press control-r again to search further back in your history. I know this sounds a bit complicated, but after using it for a while it will become second nature, and you'll wonder how you ever accomplished anything without it.
Now I need to explain how commands can be strung together. It's more complicated, but it's very useful and you can learn the simpler forms pretty easily. You string commands together with the pipe character, '|' (vertical bar, shift-backslash, usually above enter). What this does is take the output from one command, and use it as the input for another command.
Which brings me to the next big tip: less. The 'less' command is called a pager, it lets you easily move through and search through command output. For example, type "ls -lh /usr/lib". There will be a ton of output that floods your terminal. Now you can scroll through and read this all like a schmuck, or you can pipe it through less, and search it quickly to find what you need. Try "ls -lh /usr/lib | less" (remember the pipe key). Now you can use the page-up and page-down keys to move through the output. Use the '/' key to search forward (this is case sensitive), and the '?' key to search backward. When you are done, press 'q' to quit.
Finding files:
With a huge project like android, it can be difficult to find where files are. Linux has a find command, but it can be a little difficult to use, so I'll explain the more common aspects.
If you just type 'find' in the directory you are in, you'll get every file in that directory and all subdirectories. That's somewhat useful, but what we want to do is prune it down even further. For example, let's find QualcommCameraHardware.cpp in the mtdev cm9 repo: "find . -name QualcommCameraHardware.cpp". This will take a bit to run the first time, but things get cached into RAM if you have enough and should be faster next time.
That second parameter, the "." above, is where to start the find from. The "." means your current directory. There is also a ".." which means "one directory higher than I am right now". So if you are in android-cm9 and you and to quickly find something in android-cm7, you can use use "../android-cm7" rather than switching directories. You can find a specific directory if you know what you are looking for will be there, and it will be quicker. For example we could do "find hardware -name QualcommCameraHardware.cpp" and it would just look in the hardware directory, or "find ../android-cm7/hardware -name QualcommCameraHardware.cpp" for the cm7 repo. Also note, the "-name" is case sensitive. We'll get to case-insensitive later, it's a little difficult with find.
Searching for strings:
"grep" is the command to use to search for strings in files or command output. It is pretty advanced, but we don't need many of its features at all. grep will work on a file if you give it as the last parameter, but works on other commands' output by default. So, if you just type "grep searchforme", the command sits there waiting for input, and you will need to ctrl-c out of it. Try it with a file: "grep stopPreview hardware/qcom/camerahal/QualcommCameraHardware.cpp". Note that by default it is case sensitive, make it case insensitive with "-i": "grep -i stoppreview hardware/qcom/camerahal/QualcommCameraHardware.cpp" (you are remembering to use tab completion, right?).
You can also search through a directory and all it's content with the "-r" option: "grep -ri stoppreview hardware". If you really get desperate, grep for that string through the whole damn repo (remember you can use "." as the current directory), but it's will take a few minutes.
If you are dealing with binary files, the use the 'strings' command, and pipe the output to grep: "strings out/target/product/triumph/system/lib/liboemcamera.so |grep CAMERA_STOP". Remember too, you can pipe this string of commands or any other to less if the output is too long: "strings out/target/product/triumph/system/lib/liboemcamera.so |grep CAMERA_STOP | less".
Combining finding and searching:
If you know where you need to be searching for strings, combining find and grep can make searching very quick. First, let's learn another shell term: the wildcard (*). The wildcard matches any string. For example, try: "ls hardware/qcom/camerahal". Now try: "ls hardware/qcom/camerahal/*.cpp". It's as if you gave the ls command each individual string that ended with ".cpp". Well, let's use that with find: "find hardware/qcom/camerahal -name "*.cpp"". (Note that when using wildcards, you need to put your find term in quotes.) That's kind of a lame example since it's not any different than ls. But, let's find every C++ source file in the repo: "find . -name "*.cpp"". You can also put a wildcard in the middle: "find . -name "Camera*.cpp"
So now we've identified a set of files we are interested in, and we would like to search for something in them. Time to learn another command: xargs. This useful little guy runs a different program on the output of a given command. Sound complicated? It kind of is... but what if we would take the output of find, and run grep on each of those files? Pretty cool, right? Let's do it: "find . -name "Camera*.cpp" | xargs grep getPictureSize".
Honestly, this is a big chunk of what I use day to day in trying to quickly find things in a big project. There are of course ways to refine this further, there are other tools that can help tweak command output even further... but this is kind of a good base layer of knowledge, and you can always add to it as you learn more.
Finding differences between files:
To quickly tell if two files are the same, I like to use md5sum: "md5sum hardware/qcom/camerahal/QualcommCameraHardware.cpp ../android-cm7/hardware/qcom/camerahal/QualcommCameraHardware.cpp". This will give you a hash value for each file that will match if the files are bit for bit identical.
If the files don't match, and are source code files, you can then use the diff command to see what the differences are: "diff -u hardware/qcom/camerhal/QualcommCameraHardware.cpp ../android-cm7/hardware/msm7k/libcamera/QualcommCameraHardware.cpp". (Note the -u, this is for unified diff format. I recommend always using -u, it is much easier to read). Pipe this to less, or if you want to save to a file, use the redirect operator ">": "diff -u hardware/qcom/camerhal/QualcommCameraHardware.cpp ../android-cm7/hardware/msm7k/libcamera/QualcommCameraHardware.cpp > qualcomm.diff". (In fact, you can use this redirect operator to save the output of any command). Diff shows only the portions which are different, the header at the top describes the output. To make the two files the same, you would take out all the lines that start with '-', and add all the lines that start with '+'.
In fact, if you save this diff file, you can use it to convert one file to the other version, but that is slightly outside the scope of this guide. It's not hard to do though, I'm sure you can figure it out, just search about the patch command.
The diff command will also work recursively through subdirectories with "-r". Another useful option is "-N", which won't show the diff of files that aren't there (otherwise you get a + for every line of the new file). So maybe you use: "diff -rNu cm9-kernel/drivers/media/video/msm cm7-kernel/drivers/media/video/msm".
Dealing with Android:
You certainly know by now what a beast Android is. The good news is, you don't have to do a full rebuild for most changes, and you don't need to to a full flash for some changes. After you run "adb shell" to get a prompt on your phone, run: "mount -o remount,rw /system". This will remount the /system directory as read-write, so you can make changes on your live phone.
At this point, you can use "adb push" to copy files from your machine directly to your phone. For example, "adb push android-triumph/out/target/product/triumph/system/lib/libcamera.so /system/lib" will copy the local libcamera.so onto your camera's /system/lib.
When you run "brunch triumph" to do your build, pay attention to which files are actually being rebuilt. For example when working on the camera stuff, often there were just two files that would change on me: libcamera.so and camera.msm7x30.so. You will see the files that change scroll by in blue during the build. Cherry-picking these from the out directory saved me a lot of time. This same rule applies for the .apks that get built... if only one or a couple change it might be quicker to copy them individually rather than reflashing.
Note that you will not want to just pull the battery if you remount read-write, there may be a chance of corruption. Always do a software reboot to make sure things get written out properly. Or, if you adb shell in, you can run the "sync" command to force a write flush.
One other tip I have is for logcat. Rather than just running plain logcat, try "adb logcat -v time *" to get timestamps and extra verbose information. Remember, you can of course redirect to a file with our friend ">".
Text Editor:
Do yourself a favor and learn to use a real text editor. I strongly recommend either Emacs or vim, I'm an Emacs man myself. It probably has an easier initial learning curve, but I'm not going to cover it here there are plenty of newbie guides for text editors. Don't close your editor when you're done with a file, leave all those files loaded and it will make switching between them a lot easier.
Version Control:
I need to learn git myself to get the camera stuff checked in =]
One general comment: always always always put as much detailed information in your commit messages as possible. You (or someone else) will thank you later when you are trying to figure out why code was changed months ago.
KEEP TRACK OF EVERYTING
I cannot stress this enough. Ideally we could replay every keystroke but obviously that isn't manageable. Try to only change one thing at a time, and keep track of exactly what you are changing. Sometimes I keep a little text file log of ideas and things I have tried.
