Android CTS測試

一什麼是CTS

CTS簡介：Compatibility Test suite系列兼容測試
google定義了一個兼容性規范(Compatible Definition),而CTS就是為了確保某個測試符合該規范。從而基於Android的應用程序能在基於同一個api版本的設備上面運行。通過CTS測試的device可以獲得android的商標，並且享受android market的權限。

例如手機、平板電腦、車載娛樂系統；嚴格來說需要下載第三方應用的Android系統都要通過CTS測試，才能達到兼容性。

CTS的測試包可以到：http://source.android.com/compatibility/downloads.html去下載，選擇對應Android版本的CDD文檔（介紹CTS測試），CTS測試包下載（測試包會不斷更新），如：android-cts-verifier-4.0.3_r1-linux_x86-arm.zip，android-cts-media-1.0.zip；將下載的CTS測試包解壓到相應文件夾裡面應包含：android-cts文件夾裡面包含Docs、Repository、Tools三個文件夾。需要配置好SDK相關的環境變量，最好是在Ubuntu下進行，搭建好Android的開發環境就可以了。關於環境變量如何配置可以去找下相關的資料。

測試前需要安裝apk：adb install android-cts/repository/testcases/CtsDeviceAdmin.apk

會在系統設置－》安全－》設備管理器中生成下圖，把這三項選中

二、運行CTS<喎?/kf/ware/vc/" target="_blank" class="keylink">vcD4KPHA+PHByZSBjbGFzcz0="brush:java;">aaa@aaa-All-Series:~/D/android-cts-4.4_r3-linux_x86-arm/android-cts/tools$ ls cts-tradefed cts-tradefed.jar ddmlib-prebuilt.jar hosttestlib.jar junit.jar README tradefed-prebuilt.jar 首先在android-cts-4.4_r3-linux_x86-arm/android-cts/tools目錄下運行./cts-tradefed
如果沒有運行權限需要執行chmod 777 cts-tradefed

aaa@aaa-All-Series:~/D/android-cts-4.4_r3-linux_x86-arm/android-cts/tools$ ./cts-tradefed

此時會出現
cts-tf >

cts-tf > run cts --help
'cts' configuration: Runs a CTS plan from a pre-existing CTS installation

Printing help for only the important options. To see help for all options, use the --help-all flag

  cmd_options options:
    --[no-]help          display the help text for the most important/critical options. Default: false.
    --[no-]help-all      display the full help text for all options. Default: false.
    --[no-]dry-run       build but don't actually run the command.  Intended as a quick check to ensure that a command is runnable. Default: false.
    --[no-]noisy-dry-run build but don't actually run the command.  This version prints the command to the console.  Intended for cmdfile debugging. Default: false.
    --[no-]loop          keep running continuously. Default: false.

  test options:
    --plan               the test plan to run.
    -p, --package        the test packages(s) to run.
    -c, --class          run a specific test class.
    -m, --method         run a specific test method, from given --class.
    --continue-session   continue a previous test session.

  'file' logger options:
    -l, --log-level-display
                         the minimum log level to display on stdout. Default: ERROR. Valid values: [VERBOSE, DEBUG, INFO, WARN, ERROR, ASSERT]

eg:
執行一個類
cts-tf > run cts --class libcore.javax.net.ssl.SSLSocketTest
執行一個類中的方法
cts-tf > run cts -c libcore.javax.net.ssl.SSLSocketTest -m test_SSLSocket_getSupportedCipherSuites_names
測試的結果

cts-tf > run cts -c libcore.javax.net.ssl.SSLSocketTest -m test_SSLSocket_getSupportedCipherSuites_names
12-02 20:01:51 I/TestInvocation: Starting invocation for 'cts' on build '4.4_r3' on device FIH6WK5PFAOFO7MB
12-02 20:01:51 I/FIH6WK5PFAOFO7MB: Created result dir 2014.12.02_20.01.51
12-02 20:02:05 I/FIH6WK5PFAOFO7MB: Collecting device info
12-02 20:02:07 I/FIH6WK5PFAOFO7MB: -----------------------------------------
12-02 20:02:07 I/FIH6WK5PFAOFO7MB: Test package android.core.tests.libcore.package.libcore started
12-02 20:02:07 I/FIH6WK5PFAOFO7MB: -----------------------------------------
12-02 20:02:15 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getSupportedCipherSuites_names FAIL 
junit.framework.AssertionFailedError: Unknown cipher suites expected:<[]> but was:<[TLS_FALLBACK_SCSV]>
at libcore.java.security.StandardNames.assertValidCipherSuites(StandardNames.java:842)
at libcore.java.security.StandardNames.assertSupportedCipherSuites(StandardNames.java:852)
at libcore.javax.net.ssl.SSLSocketTest.test_SSLSocket_getSupportedCipherSuites_names(SSLSocketTest.java:68)
at java.lang.reflect.Method.invokeNative(Native Method)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1710)

12-02 20:02:18 I/FIH6WK5PFAOFO7MB: Saved log device_logcat_5252779086477145420.zip
12-02 20:02:19 I/FIH6WK5PFAOFO7MB: Saved log host_log_8790846039917450950.zip
12-02 20:02:19 I/FIH6WK5PFAOFO7MB: android.core.tests.libcore.package.libcore package complete: Passed 0, Failed 1, Not Executed 0
12-02 20:02:19 I/FIH6WK5PFAOFO7MB: Created xml report file at file:///home/dzt/D/dzt/xxxxxxxxx/xuna/android-cts-4.4_r3-linux_x86-arm/android-cts/tools/./../../android-cts/repository/results/2014.12.02_20.01.51/testResult.xml
12-02 20:02:19 I/FIH6WK5PFAOFO7MB: XML test result file generated at 2014.12.02_20.01.51. Passed 0, Failed 1, Not Executed 0
12-02 20:02:19 I/FIH6WK5PFAOFO7MB: Time: 0s
cts-tf > 
cts-tf > 
cts-tf > 
cts-tf > 
cts-tf > 
cts-tf > 
cts-tf > 
cts-tf > run cts --class android.widget.cts.AbsListViewTest
12-02 20:02:32 I/TestInvocation: Starting invocation for 'cts' on build '4.4_r3' on device FIH6WK5PFAOFO7MB
12-02 20:02:32 I/FIH6WK5PFAOFO7MB: Created result dir 2014.12.02_20.02.32
12-02 20:02:42 I/FIH6WK5PFAOFO7MB: Collecting device info
12-02 20:02:44 I/FIH6WK5PFAOFO7MB: -----------------------------------------
12-02 20:02:44 I/FIH6WK5PFAOFO7MB: Test package android.widget started
12-02 20:02:44 I/FIH6WK5PFAOFO7MB: -----------------------------------------
12-02 20:02:47 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessCacheColorHint PASS 
12-02 20:02:47 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessFastScrollEnabled PASS 
12-02 20:02:48 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessListPadding PASS 
12-02 20:02:49 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessScrollingCacheEnabled PASS 
12-02 20:02:49 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessSelectedItem PASS 
12-02 20:02:50 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessSelector PASS 
12-02 20:02:50 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessSmoothScrollbarEnabled PASS 
12-02 20:02:51 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessStackFromBottom PASS 
12-02 20:02:51 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessTranscriptMode PASS 
12-02 20:02:52 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAddTouchables PASS 
12-02 20:02:52 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testBeforeAndAfterTextChanged PASS 
12-02 20:02:52 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testCheckLayoutParams PASS 
12-02 20:02:53 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testComputeVerticalScrollValues PASS 
12-02 20:02:53 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testConstructor PASS 
12-02 20:02:54 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testDraw PASS 
12-02 20:02:54 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testFoo PASS 
12-02 20:02:55 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testGenerateLayoutParams PASS 
12-02 20:02:56 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testGetContextMenuInfo PASS 
12-02 20:02:56 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testGetFocusedRect PASS 
12-02 20:02:57 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testGetTopBottomFadingEdgeStrength PASS 
12-02 20:02:57 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testHandleDataChanged PASS 
12-02 20:02:58 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testInvalidateViews PASS 
12-02 20:02:58 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testLayoutChildren PASS 
12-02 20:02:58 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testPointToPosition PASS 
12-02 20:02:59 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testSetFilterText PASS 
12-02 20:03:03 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testSetOnScrollListener PASS 
12-02 20:03:03 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testSetRecyclerListener PASS 
12-02 20:03:04 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testSetScrollIndicators PASS 
12-02 20:03:04 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testShowContextMenuForChild PASS 
12-02 20:03:09 I/FIH6WK5PFAOFO7MB: Saved log device_logcat_5736317118032050398.zip
12-02 20:03:11 I/FIH6WK5PFAOFO7MB: Saved log host_log_5498353359267291055.zip
12-02 20:03:11 I/FIH6WK5PFAOFO7MB: android.widget package complete: Passed 29, Failed 0, Not Executed 0
12-02 20:03:11 I/FIH6WK5PFAOFO7MB: Created xml report file at file:///home/dzt/D/dzt/xxxxxxxxx/xuna/android-cts-4.4_r3-linux_x86-arm/android-cts/tools/./../../android-cts/repository/results/2014.12.02_20.02.32/testResult.xml
12-02 20:03:11 I/FIH6WK5PFAOFO7MB: XML test result file generated at 2014.12.02_20.02.32. Passed 29, Failed 0, Not Executed 0
12-02 20:03:11 I/FIH6WK5PFAOFO7MB: Time: 0s

cts-tf > run cts --class libcore.javax.net.ssl.SSLSocketTest
12-02 20:34:51 I/TestInvocation: Starting invocation for 'cts' on build '4.4_r3' on device FIH6WK5PFAOFO7MB
12-02 20:34:51 I/FIH6WK5PFAOFO7MB: Created result dir 2014.12.02_20.34.51
12-02 20:35:06 I/FIH6WK5PFAOFO7MB: Collecting device info
12-02 20:35:07 I/FIH6WK5PFAOFO7MB: -----------------------------------------
12-02 20:35:07 I/FIH6WK5PFAOFO7MB: Test package android.core.tests.libcore.package.libcore started
12-02 20:35:07 I/FIH6WK5PFAOFO7MB: -----------------------------------------
12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_HandshakeCompletedListener PASS 
12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_HandshakeCompletedListener_RuntimeException PASS 
12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_TrustManagerRuntimeException PASS 
12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_clientAuth PASS 
12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_clientAuth_bogusAlias PASS 
12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_close PASS 
12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_confirmSessionReuse PASS 
12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getEnableSessionCreation PASS 
12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getEnabledCipherSuites PASS 
12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getEnabledProtocols PASS 
12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getSSLParameters PASS 
12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getSession PASS 
12-02 20:36:15 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getSupportedCipherSuites_connect FAIL 
java.lang.Exception: One or more problems in test_SSLSocket_getSupportedCipherSuites_connect:
Problem trying to connect cipher suite TLS_FALLBACK_SCSV client=AndroidOpenSSL server=AndroidOpenSSL

at libcore.javax.net.ssl.SSLSocketTest.test_SSLSocket_getSupportedCipherSuites_connect(SSLSocketTest.java:114)
at java.lang.reflect.Method.invokeNative(Native Method)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1710)

12-02 20:36:15 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getSupportedCipherSuites_names FAIL 
junit.framework.AssertionFailedError: Unknown cipher suites expected:<[]> but was:<[TLS_FALLBACK_SCSV]>
at libcore.java.security.StandardNames.assertValidCipherSuites(StandardNames.java:842)
at libcore.java.security.StandardNames.assertSupportedCipherSuites(StandardNames.java:852)
at libcore.javax.net.ssl.SSLSocketTest.test_SSLSocket_getSupportedCipherSuites_names(SSLSocketTest.java:68)
at java.lang.reflect.Method.invokeNative(Native Method)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1710)

12-02 20:36:15 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getSupportedProtocols PASS 
12-02 20:36:15 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getUseClientMode PASS 
12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_interrupt PASS 
12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_interrupt_read PASS 
12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_reusedNpnSocket PASS 
12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setEnableSessionCreation_client PASS 
12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setEnableSessionCreation_server PASS 
12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setEnabledCipherSuites PASS 
12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setEnabledProtocols PASS 
12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setSSLParameters PASS 
12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setSoTimeout_basic PASS 
12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setSoTimeout_wrapper PASS 
12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setSoWriteTimeout PASS 
12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setUseClientMode PASS 
12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setUseClientMode_afterHandshake PASS 
12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_shutdownCloseOnClosedPeer PASS 
12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_startHandshake PASS 
12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_startHandshake_noClientCertificate PASS 
12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_startHandshake_noKeyStore PASS 
12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_untrustedServer PASS 
12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_TestSSLSocketPair_create PASS 
12-02 20:36:24 I/FIH6WK5PFAOFO7MB: Saved log device_logcat_9056351731602008844.zip
12-02 20:36:25 I/FIH6WK5PFAOFO7MB: Saved log host_log_1029983955603589903.zip
12-02 20:36:25 I/FIH6WK5PFAOFO7MB: android.core.tests.libcore.package.libcore package complete: Passed 33, Failed 2, Not Executed 0
12-02 20:36:25 I/FIH6WK5PFAOFO7MB: Created xml report file at file:///home/dzt/D/dzt/xxxxxxxxx/xuna/android-cts-4.4_r3-linux_x86-arm/android-cts/tools/./../../android-cts/repository/results/2014.12.02_20.34.51/testResult.xml
12-02 20:36:25 I/FIH6WK5PFAOFO7MB: XML test result file generated at 2014.12.02_20.34.51. Passed 33, Failed 2, Not Executed 0
12-02 20:36:25 I/FIH6WK5PFAOFO7MB: Time: 0s