It has been a few days since I’ve made my last post, and that’s for good reason. I now have a new approach to testing the OC kernels! A very kind developer from droidforums.net, Kill_One, has helped me out by providing a stripped down version of the Sapphire ROM. This ROM contains only what is needed to boot up android and it even blocks all wireless connections right out of the box! Using this ROM will allow me to remove as many variables as possible when benchmarking. The goal is to have less than a .5% variation between 10 runs of a benchmarking app. That said, there may be variables in which I cannot control (think hardware). Also, due to Quadrant’s requirement for an internet connection and my lack there of (due to forced airplane mode), there will be no Quadrant benchmarks performed using the “Clean Room” Sapphire ROM.
Here is the testing procedure:
- Install the “Clean Room” ROM via nandroid.
- Restore APK’s from SD card.
- Benchmark PI
- CPU Benchmark
- System Panel Lite
- Battery stats
- Shutdown phone, remove MicroSD card, reboot.
- Kill all running apps with system panel.
- Verify phone is near “braindead” with aLogcat and system panel.
- Kill system panel & aLogcat via system/applications/manage applications/running applications. Kill the above two apps and finally the system app itself.
- Run the first benchmark, then repeat steps 4-6 before running the next benchmark.
The results should be pretty consistent through 10 consecutive runs of the benchmark. Although I have noticed that Linpack gives the most consistant results. Let me know if you see something you would like added!!!