The Witness

---

For those folks into technical posts, I'll discuss how we used our new programming language to create an automated framework to test the language itself. Jon wanted to convert his demos into proper language tests, and add in new ones. Testing becomes increasingly important as we approach a closed-group release.

I've also disliked test frameworks in general, which added to the problem. In the interest of feeling joyful and productive, I decided to sit and think about the specific reasons why I've been unhappy with frameworks, and to use the language to write one I'd enjoy using.

I like to ask personally interesting questions which the tests ought to help me answer. Often, they relate to the behavior of software at the threshold of cross-cutting concerns. For the jai compiler, they could be questions such as:. Don't worry if the questions seem bizarre the language is new and hasn't been released yet. The common theme is they all require very different language features to interact with each other i. For example, the first item could be handled polymorphically with the modify directive, enums, a switch statement, and some random bit generator.

The answer to the second item is that nothing bad happens; in fact, this opens the door for multiple structs to use the same polymorphic procedure as their constructor. This observation naturally leads to questions three and four. When I wrote the tests for some of these, compiler bugs were uncovered naturally and without much effort. Note how these questions are answered by just writing programs. They are the language tests. Anything not helping me towards that feels like noise, and that explains why I haven't been happy with many testing systems.

Here's the criteria I want a framework to meet to feel good about writing tests:. By a minimal report I mean showing me which programs ran and when, how many test asserts were called and where to find them if they failed. The report should look good so I can help my brain enjoy the overall experience.

To showcase the framework we ended up creating, let's go through an example, transforming a program into a language test. Here's the program that tackles one of the simpler inquiries from earlier: What if a polymorphic procedure is used as a constructor? While we're at it we'll ask the same of destructors.

As I had mentioned, this means multiple structs can use the procedure as their respective constructors. We can explore the possibility space even more, define any number of procedures, declare hundreds of complex structs, import 3rd-party modules—you name it. However, let's pretend we're done answering our polymorphic constructor questions. To turn this into a language test, let's make two straightforward modifications we'll add a couple of asserts while we're at it:.

Notice I changed the name from main to something else and added the simple note TestProcedure. Notes are essentially comments recognized by the compiler that do nothing by default. Ah, the reader will notice I have a file conveniently called tests. The reader might ask why we have 82 more asserts than what we wrote, plus another test procedure from a String file. More on that soon. That's a lot of useful output just by compiling a source file! In addition, we can have multiple test procedures on the same test file:.

Finally, if invariances in our tests don't hold i. Let's add a deliberate failure on the first test procedure:. If the test writer heads to the test files section, the procedure that failed would be made apparent. So how do we explain that weird String file from the reports earlier? We didn't write that!

Screenshots and Videos

Well, Ignacio found it useful to use this framework for the language's system modules. System modules are special. The test file from earlier imported Basic a catch-all module we use to experiment new features. Basic happens to indirectly import the String module. When I opened up String , I noticed Ignacio had written this somewhere on the file:. Which accounted for the 82 additional asserts.

That's because the framework detects tests written in external modules, outside of our own test files. Now anyone on the team writing modules can write test procedures inside their module, tag them, and they're done. The actual tests can live right next to the module code, outside of our folder, and we register them for free just as we automatically register the test files inside the folder for free.

And fret not, a module being imported multiple times will still have its test recognized only once. I'd like to note there are special settings and configurations the user doesn't know about good! Documentation amounts to a few well-placed comments. Upon release if we have time to make it happen , I'd like people to write their numerous test procedures, compile tests.

A visualization of a growing tree of test files as branches, where each test procedure is a leaf from the branch. A common theme back at Kennedy Space Center was that we wanted to automate most of the "bookkeeping" so the engineer's time and effort went into thinking about the real problem. Interacting with build systems was not in our best interest! Even so, sometimes we were stuck with what I think were high-friction tools.

I can't properly express to the reader how great it's been to write a tool that reduces friction nontrivially. Let's take a quick look at how the language played a part in letting me create this framework, as I think it has the right kind of features all statically-typed languages should have going forward. Initially, the main struggles in making the testing system work were psychological, working to unlearn years of how I was told automated test frameworks should be structured.

I think Jon nearly had a heart attack. It was difficult to implement something that met the criteria listed at the beginning of this post, because my model was tied to old-school software testing models. The closest implementation I have ever found is CxxTest. What caught my interest was the second sentence on their homepage:. RTTI and it supports a very flexible form of test discovery.

It's trying to make itself invisible, and the term test discovery sounds like it's doing work on behalf of the user. Indeed, later the website goes on to say:. Additionally, CxxTest supports test discovery. Test discovery is the property I want frameworks to exhibit. In jai , however, we can specify our build system in the language. So I made a source file, tests. In said file, we can ask the compiler to compile any number of different files, and request it to open up its compilation stages to us as things get lexically scanned, parsed, and code generated.

That's more than we care to ask for, so our entire build system can amount to compiling a single file, in the same way we compile any normal code. As I just said, in tests. So I used that for the test discovery. The current mechanism, as we have seen, is to tag procedures that want to behave like an independent test program with the note TestProcedure.

Then, once the compiler has finished type-checking all source files, I ask it to give me the declarations tagged with the note. In the past, I used to check for procedure names starting with test , but Allen and Nico made me question whether that was too constraining.

Navigation menu

Turns out it was. The downside with the notes here, though, is there might be tests importing external modules that tagged some of their code with TestProcedure for unrelated reasons. I mitigate that by ignoring structs with that tag, and by verifying whether the procedure acts as an entry point i. There are other options available to us, but for now this is a good-enough solution.

The build file has access to that, as do people. If anyone wishes to avoid all the automatic magic and also believe editing the build file is not enough, they are free to ignore it altogether and figure out how the suite works. They can register tests manually, generate reports however they see fit, and perform the entire job of a framework themselves.

As we can see from the CxxTest homepage, it has different types of asserts for us to use. I had something similar going, but Ignacio reminded me that Jon added an assertion handler as part of the implicit context. That means we can override the behavior of regular asserts! This is what allows external modules to write officially-recognized tests inside their file and still compile anywhere--we're not requiring them to do anything special.

I now enjoy writing tests whenever I need to, and I hope this straightforward style sounds appealing to some people. In other news, for the purpose of testing a compiler, will an automated test suite suffice? You wake up, alone, on a strange island full of puzzles that will challenge and surprise you.

You don't remember who you are, and you don't remember how you got here, but there's one thing you can do: The Witness is a single-player game in an open world with dozens of locations to explore and over puzzles. This game respects you as an intelligent player and it treats your time as precious. There's no filler; each of those puzzles brings its own new idea into the mix. So, this is a game full of ideas. Overall the game is completely original, and that's something very difficult to achieve in today's gaming world.

Breathtaking visuals, peaceful feel, and endless challenge and exploration can keep a gamer hooked.

The Witness

Unfortunately the drawback comes from that same challenge. Many of the puzzles are near impossible to solve without looking up a solution.

The Witness

Once you dive deeper into the game, the connections some puzzles have with one another are hidden underneath hours of exploring and irritating, sometimes tedious, trial and error. Definitely a game to kick back with, but one you'll be taking alot of breaks from as well. The game definitely raises alot of questions that were unfortunately never answered.

Great puzzle game, but that's about it I will say though Its not like Myst. Yes there is more to it than just the puzzles you see. But its nothing special and Its just more of the same anyway. You mostly just walk around from puzzle stand to puzzle stand. The core concept of the game is fun. Solve little puzzles in various areas to hopefully uncover the mysteries of the game world. Further progression through these puzzles offers no real answers, just more questions which culminates to an ending i would rather not discuss. Get it while its free.

It was "free" through gold. Great looking yes, if looking from a artist standpoint. Looking at it from a gaming standpoint Like I said while nice to look at the "game" has no real purpose.

You will solve one puzzle only to solve another one and another with a rinse and repeat cycle. Do the puzzles get harder? But only because there is no direction on how solve them, also are just there to be there completely going nowhere. The other upside is that the "game" is pretty much open with no time limit and you solve most puzzles in any order, or quit at any time. Which is what I did because a "game" like this with no narrative holds no weight for me. The Witness tried to make an innovative attempt at a puzzle game. The game is choc full of 'oh yeah!

The game throws random symbols in most of its puzzles that you don't even know of yet. This doesn't help that places like the The Orchard right after the starting area is right next to The Town a moshpit of symbols. You even have to go across the map to get knowledge of a common symbol: The symbols themselves couldn't even be fully explained if you held a gun to my head.

I like a challenging puzzle game every now and then but at least they tell you what their symbols and methods are. There isn't even much of story in this game either besides concrete statues. The only thing this game has is the environment. Anyway, I'd only download this game for the gamerscore.

• Contabilidad Básica para Organizaciones sin Fines de Lucro (Spanish Edition)
• Bunny Buddhism: Hopping Along the Path to Enlightenment
• The Daze Of Our Lives. The most hated man in the history of music. Inventor of the bagpipes, Jock McBagpipe.
• Rache kann so sinnlich sein ... (Baccara) (German Edition)
• Promessas de Deus para abençoar seu casamento (Portuguese Edition)
• Sorcerers Code (The Elements of Sorcery Book 1)