| 1 | ---
|
| 2 | css_files: web/base.css web/toc.css web/release-index.css
|
| 3 | all_docs_url: -
|
| 4 | version_url: -
|
| 5 | ---
|
| 6 |
|
| 7 | Oils 0.22.0 Quality
|
| 8 | ===================
|
| 9 |
|
| 10 | <!-- NOTE: This file is published to /release/$VERSION/quality.html -->
|
| 11 |
|
| 12 | <span class="date">
|
| 13 | <!-- REPLACE_WITH_DATE -->
|
| 14 | </span>
|
| 15 |
|
| 16 | This is a supplement to the [main release page](index.html).
|
| 17 |
|
| 18 | <div id="toc">
|
| 19 | </div>
|
| 20 |
|
| 21 | ## Test Results
|
| 22 |
|
| 23 | ### Spec Tests
|
| 24 |
|
| 25 | - [OSH Survey](test/spec.wwz/osh-py/index.html). Test OSH with existing shells,
|
| 26 | and compare their behavior.
|
| 27 | - [OSH in C++](test/spec.wwz/osh-cpp/compare.html). The progress of the C++
|
| 28 | translation.
|
| 29 | - [YSH](test/spec.wwz/ysh-py/index.html). The legacy-free language.
|
| 30 | - [YSH in C++](test/spec.wwz/ysh-cpp/compare.html). The progress of the C++
|
| 31 | translation.
|
| 32 | - [Stateful Tests](test/spec.wwz/stateful/index.html). Tests that use
|
| 33 | [pexpect]($xref).
|
| 34 |
|
| 35 | ### Primary Test Suites
|
| 36 |
|
| 37 | - [Gold Tests](more-tests.wwz/suite-logs/gold.txt). Compare OSH with bash
|
| 38 | (using implicit assertions, not golden output.)
|
| 39 | - [Wild Tests](test/wild.wwz/). Parse and translate thousands of shell scripts
|
| 40 | with OSH.
|
| 41 | - [Python Unit Tests](more-tests.wwz/unit/).
|
| 42 | - [C++ Test Coverage](test/coverage.wwz/unified/html/index.html) measured by
|
| 43 | Clang.
|
| 44 | - [Log Files](test/coverage.wwz/log-files.html)
|
| 45 | - [Process Table](more-tests.wwz/process-table/). Are child processes in the
|
| 46 | right state for job control?
|
| 47 |
|
| 48 | ### More Tests
|
| 49 |
|
| 50 | - [Smoosh][] test suite (from [mgree/smoosh][]):
|
| 51 | - [smoosh](test/spec.wwz/smoosh/smoosh.html)
|
| 52 | | [smoosh-hang](test/spec.wwz/smoosh/smoosh-hang.html)
|
| 53 | - [parse-errors](more-tests.wwz/suite-logs/parse-errors.txt). A list of all parse errors.
|
| 54 | - [parse-errors-osh-cpp](more-tests.wwz/suite-logs/parse-errors-osh-cpp.txt).
|
| 55 | With the native binary.
|
| 56 | - [runtime-errors](more-tests.wwz/suite-logs/runtime-errors.txt). A list of shell runtime
|
| 57 | errors.
|
| 58 | - [ysh-runtime-errors](more-tests.wwz/suite-logs/ysh-runtime-errors.txt). YSH
|
| 59 | runtime errors.
|
| 60 | - [ysh-parse-errors](more-tests.wwz/suite-logs/ysh-parse-errors.txt). YSH
|
| 61 | parse errors.
|
| 62 | - [ysh-every-string](more-tests.wwz/suite-logs/ysh-every-string.txt). String
|
| 63 | literal stress test.
|
| 64 | - [ysh-large](more-tests.wwz/suite-logs/ysh-large.txt)
|
| 65 | - [lossless](more-tests.wwz/suite-logs/lossless.txt). Test an invariant of the parser.
|
| 66 | - [osh-usage](more-tests.wwz/suite-logs/osh-usage.txt). Misc tests of the `osh` binary.
|
| 67 | - [tools-deps](more-tests.wwz/suite-logs/tools-deps.txt). Tests for a subcommand in
|
| 68 | progress.
|
| 69 | - [syscall](more-tests.wwz/syscall/-wwz-index) How many syscalls do we make,
|
| 70 | and how many processes do we start?
|
| 71 | - [ysh-ify Tests](more-tests.wwz/suite-logs/ysh-ify.txt). Test OSH to YSH
|
| 72 | translation.
|
| 73 |
|
| 74 | [Smoosh]: http://shell.cs.pomona.edu/
|
| 75 |
|
| 76 | [mgree/smoosh]: https://github.com/mgree/smoosh/tree/master/tests/shell
|
| 77 |
|
| 78 | ## Benchmarks
|
| 79 |
|
| 80 | - [Parser](benchmarks.wwz/osh-parser/). How fast does OSH
|
| 81 | parse compared to other shells?
|
| 82 | - [Runtime](benchmarks.wwz/osh-runtime/). How fast does OSH run shell
|
| 83 | scripts?
|
| 84 | - [Compute](benchmarks.wwz/compute/). How fast does OSH run small programs
|
| 85 | without I/O?
|
| 86 | - [Build](benchmarks.wwz/ovm-build/). How long does it take for end users to
|
| 87 | build Oils? How big is the resulting binary?
|
| 88 | - [Virtual Memory Baseline](benchmarks.wwz/vm-baseline/). How much memory do
|
| 89 | shells use at startup?
|
| 90 | - [mycpp](benchmarks.wwz/mycpp-examples/). Compares Python and generated C++
|
| 91 | on small examples.
|
| 92 | - Memory Management Overhead. How much time do we spend managing memory,
|
| 93 | compared with the shell interpreter?
|
| 94 | - [benchmarks/gc](benchmarks.wwz/gc/). Stats from the OS and our GC runtime.
|
| 95 | - [benchmarks/gc-cachegrind](benchmarks.wwz/gc-cachegrind/). Stable
|
| 96 | measurements.
|
| 97 | - [uftrace](benchmarks.wwz/uftrace/). Stable measurements for parsing and
|
| 98 | runtime.
|
| 99 |
|
| 100 | ## Metrics
|
| 101 |
|
| 102 | - Lines of source, counted in different ways:
|
| 103 | - [overview](pub/metrics.wwz/line-counts/overview.html). The whole Oils
|
| 104 | repo, organized by type of source file.
|
| 105 | - [for-translation](pub/metrics.wwz/line-counts/for-translation.html). An
|
| 106 | overview of the "compiler engineer" project.
|
| 107 | - [cloc-report](pub/metrics.wwz/line-counts/cloc-report.txt). Significant
|
| 108 | lines of code, as measured by the [cloc][] tool.
|
| 109 | - Generated C++ code
|
| 110 | - [oils-cpp](pub/metrics.wwz/line-counts/oils-cpp.txt). The C++ code in the
|
| 111 | `oils-for-unix` tarball.
|
| 112 | - [preprocessed](pub/metrics.wwz/preprocessed/-wwz-index). How much code is
|
| 113 | passed to the compiler?
|
| 114 | - [cxx-dbg](pub/metrics.wwz/preprocessed/cxx-dbg.txt),
|
| 115 | [cxx-opt](pub/metrics.wwz/preprocessed/cxx-opt.txt)
|
| 116 | - [Binary code size](pub/metrics.wwz/oils-for-unix/-wwz-index) reported by
|
| 117 | [Bloaty][]. How much code is output by the compiler?
|
| 118 | - [overview](pub/metrics.wwz/oils-for-unix/overview.txt),
|
| 119 | [symbols](pub/metrics.wwz/oils-for-unix/symbols.txt)
|
| 120 | - [Doc Metrics](doc/metrics.txt)
|
| 121 |
|
| 122 | [cloc]: https://github.com/AlDanial/cloc
|
| 123 | [Bloaty]: https://github.com/google/bloaty
|
| 124 | [OVM]: //www.oilshell.org/cross-ref.html?tag=OVM#OVM
|
| 125 |
|
| 126 | ## Source Code
|
| 127 |
|
| 128 | These files may help you understand how Oils is implemented, i.e. with
|
| 129 | domain-specific languages and code generation.
|
| 130 |
|
| 131 | - [_gen/frontend/id_kind.asdl_c.h](pub/src-tree.wwz/_gen/frontend/id_kind.asdl_c.h.html).
|
| 132 | A list of language elements, used in the lexer and in multiple parsers and
|
| 133 | evaluators.
|
| 134 | - The regex-based lexer uses two stages of code generation:
|
| 135 | - [frontend/lexer_def.py](pub/src-tree.wwz/frontend/lexer_def.py.html)
|
| 136 | | [_gen/_tmp/match.re2c-input.h](pub/src-tree.wwz/_gen/_tmp/match.re2c-input.h.html)
|
| 137 | | [_gen/frontend/match.re2c.h](pub/src-tree.wwz/_gen/frontend/match.re2c.h)
|
| 138 | - [frontend/syntax.asdl](pub/src-tree.wwz/frontend/syntax.asdl.html). The syntax tree
|
| 139 | for OSH and YSH.
|
| 140 | - [ysh/grammar.pgen2](pub/src-tree.wwz/ysh/grammar.pgen2.html). The
|
| 141 | expression grammar for YSH. In contrast, the OSH parsers are hand-written.
|
| 142 |
|
| 143 | Also see the [oilshell/oil](https://github.com/oilshell/oil) repository.
|
| 144 |
|
| 145 | ## Old
|
| 146 |
|
| 147 | These links describe the CPython / "[OVM]($xref)" build, which should become
|
| 148 | the "experimental" version of Oils.
|
| 149 |
|
| 150 | #### OPy / OVM Metrics
|
| 151 |
|
| 152 | - Lines of dependencies:
|
| 153 | - [pydeps](pub/metrics.wwz/line-counts/pydeps.txt). Oils code plus the Python
|
| 154 | standard library.
|
| 155 | - [nativedeps](pub/metrics.wwz/line-counts/nativedeps.txt). Oils code plus a
|
| 156 | slice of CPython.
|
| 157 | - Bytecode Metrics
|
| 158 | - [overview](pub/metrics.wwz/bytecode/overview.txt) - Compare OPy vs. CPython.
|
| 159 | - [oil-with-opy](pub/metrics.wwz/bytecode/oil-with-opy.txt) - Oils compiled with
|
| 160 | OPy.
|
| 161 | - [oil-with-cpython](pub/metrics.wwz/bytecode/oil-with-cpython.txt) - Oils
|
| 162 | compiled with CPython (for comparison).
|
| 163 | - [src-bin-ratio-with-opy](pub/metrics.wwz/bytecode/src-bin-ratio-with-opy.txt) -
|
| 164 | How big is the compiled output?
|
| 165 | - OVM / CPython
|
| 166 | - [overview](pub/metrics.wwz/ovm/overview.txt) - An analysis of GCC's
|
| 167 | compilation of [OVM][] (a subset of CPython). [Bloaty][] provides the
|
| 168 | underlying data.
|
| 169 | - [cpython-defs/overview](pub/metrics.wwz/cpython-defs/overview.txt) - We try to
|
| 170 | ship as little of CPython as possible, and this is what's left.
|