Commit 6439c7f
committed
Add ScalaCheck specs to test suite
Adds 26 specifications in ScalaCheck, with 154 properties, of 18 are
proofs by example, and the rest are 136 tests using randomly-generated
values. ScalaCheck requires at least 100 generated tests to pass, by
default. So that gives 13,618 new tests!
The suite should be a real benefit for future maintenance endeavors,
rewrites, documentation, or finishing a Scala-based parser to avoid
depending on the the Xerces/SAX Java library.
These integrate well with the existing suite of JUnit 4 tests. If you
want to run both the ScalaCheck and JUnit tests from SBT,
> test
...
[info] Passed: Total 145, Failed 0, Errors 0, Passed 145
[success] Total time: 43 s, completed Sep 30, 2016 2:19:56 PM
If you want to run a single spec,
> testOnly scala.xml.XMLSpec
> testOnly scala.xml.XMLSpec
[info] + XML.write: OK, passed 100 tests.
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[success] Total time: 3 s, completed Sep 30, 2016 2:20:28 PM
If you want to run all the ScalaCheck specs,
> testOnly *Spec
...
[info] Passed: Total 26, Failed 0, Errors 0, Passed 26
[success] Total time: 30 s, completed Sep 30, 2016 2:22:58 PM
If you want to run the specs under a namespace,
> testOnly scala.xml.dtd.*Spec
...
[info] + dtd.DTD.toString: OK, passed 100 tests.
[info] + dtd.DocType.new(name): OK, passed 100 tests.
[info] + dtd.DocType.new(name, extId, intSubset): OK, passed 100 tests.
[info] + dtd.DocType.new(name, extId, emptyInt): OK, passed 100 tests.
[info] + dtd.DocType.toString: OK, passed 100 tests.
[info] Passed: Total 4, Failed 0, Errors 0, Passed 4
[success] Total time: 6 s, completed Sep 30, 2016 3:14:05 PM
If you want to generate 1000 tests instead of 100,
> testOnly *Spec -- -s 1000
...
[info] Passed: Total 26, Failed 0, Errors 0, Passed 26
[success] Total time: 217 s, completed Sep 30, 2016 2:27:31 PM
If you want to generate 10 tests, instead of 100,
> testOnly *Spec -- -s 10
...
[info] Passed: Total 26, Failed 0, Errors 0, Passed 26
[success] Total time: 10 s, completed Sep 30, 2016 2:23:49 PM
If you want to generate full back traces on errors and display the elapsed time
for each property,
> testOnly *Spec -- -verbosity 2
...
[info] + parsing.XhtmlParser.initialize: OK, passed 100 tests.
[info] Elapsed time: 0.046 sec
[info] + parsing.XhtmlParser.prolog: OK, passed 100 tests.
[info] Elapsed time: 0.022 sec
[info] + parsing.XhtmlParser.document: OK, passed 100 tests.
[info] Elapsed time: 0.015 sec
[info] Passed: Total 26, Failed 0, Errors 0, Passed 26
[success] Total time: 30 s, completed Sep 30, 2016 2:29:24 PM
The default verbosity is 0,
> testOnly *Spec -- -verbosity 0
...
[info] + parsing.XhtmlParser.initialize: OK, passed 100 tests.
[info] + parsing.XhtmlParser.prolog: OK, passed 100 tests.
[info] + parsing.XhtmlParser.document: OK, passed 100 tests.
[info] Passed: Total 26, Failed 0, Errors 0, Passed 26
[success] Total time: 26 s, completed Sep 30, 2016 2:30:26 PM
To run only the failed tests,
> testQuick
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for test:testQuick
[success] Total time: 1 s, completed Sep 30, 2016 2:32:50 PM
Currently, these tests in ScalaCheck will cover only 25% of the source
code, and 20% of branches.
Not surprisingly, there are some defects in scala-xml for certain
types and classes. I am forced to comment out and disable those tests
and generators so that the suite passes. Those issues should be taken
up separately and in subsequent pull requests. When those tests and
generators are re-enabled for those types, the code coverage would
most likely increase.
There are no shrink heuristics defined written to help ScalaCheck
identify the minimum value to falsify a property of an XML string or
data structure. This will be a nice to have, but was too advanced for
me to take on at this time.
Right now, the generators equally weight the Node types among those
that are allowed. It would probably be worthwhile to have the
frequencies be more realistic. For example, have a a greater emphasis
on Elem and EntityRef elements over Comment, ProcInstr, nulls,
empty strings and empty lists. The frequencies should be closer in
line with what a typical XML file would be, but also more broadly
cover the code and sooner using fewer generated values.
Similarly, there is not an appropriate sizing for lists. Presently,
ScalaCheck randomly selects a number, and then the generators I wrote
will try to approach termination by recursively halving and
square-rooting at each level of descent.1 parent 8c872c6 commit 6439c7f
File tree
62 files changed
+2958
-0
lines changed- src/test/scala/scala/xml
- dtd
- impl
- factory
- parsing
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
62 files changed
+2958
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
0 commit comments