Add Cuneiform programming language

[joergen7]

This PR adds an entry for Cuneiform in languages.yml. Also, it adds a few code samples to the sample directory.

• cuneiform-lang.org
• joergen7/cuneiform
• Cuneiform (programming language)

Description

Checklist:

• I am associating a language with a new file extension.

  • The new extension is used in hundreds of repositories on GitHub.com
    • Search results for each extension:
      • https://github.com/search?utf8=%E2%9C%93&type=Code&ref=searchresults&q=extension%3Acfl+def+Str+Bool+File+NOT+nothack
  • I have included a real-world usage sample for all extensions added in this PR:
    • Sample source(s):
      • ackermann.cfl
      • r_test.cfl
      • fizzbuzz_python.cfl
    • Sample license(s):
  • I have included a change to the heuristics to distinguish my language from others using the same extension.

• I am adding a new language.

  • The extension of the new language is used in hundreds of repositories on GitHub.com.
    • Search results for each extension:
      • https://github.com/search?utf8=%E2%9C%93&type=Code&ref=searchresults&q=extension%3AFOOBAR+KEYWORDS+NOT+nothack
  • I have included a real-world usage sample for all extensions added in this PR:
    • Sample source(s):
      • [URL to each sample source, if applicable]
    • Sample license(s):
  • I have included a syntax highlighting grammar.
  • I have included a change to the heuristics to distinguish my language from others using the same extension.

• I am fixing a misclassified language

  • I have included a new sample for the misclassified language:
    • Sample source(s):
      • [URL to each sample source, if applicable]
    • Sample license(s):
  • I have included a change to the heuristics to distinguish my language from others using the same extension.

• I am changing the source of a syntax highlighting grammar

  • Old: https://github-lightshow.herokuapp.com/
  • New: https://github-lightshow.herokuapp.com/

• I am adding new or changing current functionality

  • I have added or updated the tests for the new or changed functionality.

[Alhadis]

I'm thoroughly disappointed this isn't written in , or even in Ugaritic. 😢

[joergen7]

I admit I have difficulties understanding why some of the tests fail. Most of the failures seem to originate in test_language.rb which I did not touch at all.

[Alhadis]

Alllrighty, putting aside my language-obsessed nerdiness, let's have a look at these build results:

  1) Failure:
TestGrammars#test_submodules_have_approved_licenses [/home/travis/build/github/linguist/test/test_grammars.rb:115]:
The following submodules have unapproved licenses:
* vendor/grammars/FreeMarker.tmbundle: b81acf2ba52d312754bf5055845a723123bda388
* vendor/grammars/ant.tmbundle: 4da01d631a29c76456fd0bd16749c71e8d5f6dbf
* vendor/grammars/elixir-tmbundle: 220e011c8d686129e9c4163a7c655b9d64f61e59
* vendor/grammars/mako-tmbundle: 39f092c726491ca6a02354dbc6c3a0920bb44d4c
The license must be added to the LICENSE_WHITELIST in /test/test_grammars.rb once approved.
--- expected
+++ actual
@@ -1 +1 @@
-[]
+["vendor/grammars/FreeMarker.tmbundle: b81acf2ba52d312754bf5055845a723123bda388", "vendor/grammars/ant.tmbundle: 4da01d631a29c76456fd0bd16749c71e8d5f6dbf", "vendor/grammars/elixir-tmbundle: 220e011c8d686129e9c4163a7c655b9d64f61e59", "vendor/grammars/mako-tmbundle: 39f092c726491ca6a02354dbc6c3a0920bb44d4c"]
  2) Failure:
TestGrammars#test_whitelisted_hashes_dont_have_licenses [/home/travis/build/github/linguist/test/test_grammars.rb:130]:
The following whitelisted license hashes are unused:
* 7dfce11e2e3579ee43b83e69b1b64e77a2e378f0
* 62b97e52b78439c14550a44a3fe51332aeffb3a1
* 0acff2bb1536a3942a39ac74987ffd9c44905a6b
* 41cdc7e9f9d2e62eb8ac68a1a9359b9c39a7a9bf
Please remove them from the hash whitelist.
--- expected
+++ actual
@@ -1 +1 @@

These are unrelated to your real changes, and suggest that your fork is a few releases behind upstream. Specifically, your submodules need updating. Try running script/bootstrap and manually deleting any unlisted modules with rm -rf vendor/grammars/etc.

If that doesn't work, it might help to trash everything and start from a fresh fork. 👍

[Alhadis]

The language_id has an oddly low value. It's usually somewhere in the neighbourhood of 931814000: it gets generated when running script/set-language-ids --update. Did you use the script, or was it added by hand?

If you didn't use the script, it's also possible the submodule wasn't fully registered, which can lead to other sorts of test failures. =)

[joergen7]

No, I picked a number that did not appear in the yml file :D
will correct ...

[Alhadis]

Even though it's a different language, you can use Ruby's grammar for Cuneiform highlighting. The results look decent enough.

If that isn't good enough, just give me the formal language specification, and I'll write a grammar for you. 😉

[joergen7]

Even if my installation is broken, I don't understand how this affects Travis CI. I changed one file and added 4 samples, nothing more.

I did this:

jorgen@x240:~/git/linguist$ git merge upstream/master
Already up to date.

[Alhadis]

Hrm, that's strange. 😥 Your fork is up-to-date with master, so it can't be outdated modules that's causing this. It might have something to do with a recent update to the licensed gem, which is responsible for checking licenses (and their checksums).

@lildude, @jonabc Any idea if this is impacting CI?

[joergen7]

I have just run

bundle exec rake

on a clean checkout of the master branch and got the same errors.

[Alhadis]

I did this:

jorgen@x240:~/git/linguist$ git merge upstream/master
Already up to date.

@joergen7 What does script/bootstrap output when running it? If your submodules are in sync, you shouldn't see anything. OTOH, if you see output along the lines of "Submodule vendor/grammars/… registered for …etc", it indicates the locally checked-out submodules haven't been updated.

Submodules are managed independently of the "parent" module, which means it isn't enough to run git merge by itself. Sometimes git submodule init or git submodule update is needed to ensure each module is checked out to the commit registered in .gitmodules, and so forth. :)

on a clean checkout of the master branch and got the same errors.

Try running script/bootstrap: it handles all the heavy-lifting for you, including git submodule stuff.

[joergen7]

Running

script/bootstrap

outputs nothing the second time I run it. (The first time it is quite verbose, should I grep?)

[jonabc]

I don't believe that the latest version of licensed has been pulled into linguist yet, so #4152 (comment) shouldn't be an issue. You might be hitting similar issues to what was found in #4139 though?

[Alhadis]

The first time it is quite verbose, should I grep?

What do you see when you run git status? And are you still getting test failures?

[joergen7]

Running git status on my current pull request:

jorgen@x240:~/git/linguist$ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

Also when I build from a clean clone of linguist I get the following output:
Output of running scripts bootstrap on a clean checkout of linguist

I am getting two errors on both a clean checkout and my pull request. below is the output of bundle exec rake
Output of bundle exec rake when running on pull request

Here is how I sync the pull request with the master:

git fetch upstream
git merge upstream/master

[Alhadis]

I'm afraid I'm tapped. :( I'll leave this up to @lildude... @pchaigno might also know what's going on here..

[joergen7]

I guess I'll just wait for a fix.

In the mean time: Do you think I can use the grammar-spec I created for Sublime-text here? Is there a hint which grammar specs are ok, naming conventions for directories, etc?

[Alhadis]

Erm, I'm not sure. I recall that we had troubles a while ago with parsing .sublime-syntax grammars, and I don't believe the situation's changed. Sublime's grammar format is incompatible with the older TextMate-based format, so I'm leaning on an answer of "no". 😉

[joergen7]

That's fine. Now I know I have to build an ace-grammar. But that's good for ace users, I guess.

[Alhadis]

Although your grammar is pretty self-explanatory... I'm not seeing any of Sublime's "incompatible" grammar features (the pushing and popping thing). Give me a moment, I'll convert it to a CSON file which you can stick in a grammars/ subdirectory (which is the conventional location for Atom grammars).

[joergen7]

It pushes for string and pops for inside_string. Same for file/inside_file.

Hey, thanks for the contribution!

[Alhadis]

Done. See joergen7/cuneiform-atom#1 =)

Now, adding a grammar to Linguist is simply a case of running the script/add-grammar script. You'd run it like this:

$ script/add-grammar joergen7/cuneiform-syntax-highlight

# Or like this, both notations are supported.
$ script/add-grammar https://github.com/joergen7/cuneiform-syntax-highlight

[pchaigno]

The Travis CI build failure is unrelated to this pull request and we'll have to fix it on master.

In any case, using this search query, I count only 6 users of that file extension for Cuneiform. That won't be enough to meet our hundreds-of-repositories criteria.

[joergen7]

@pchaigno Thanks for having considered my pull request, even though it did not meet the user-threshold criterion.

I will just close this pull request and re-open it, should we manage to meet all necessary criteria.

[pchaigno]

I will just close this pull request and re-open it, should we manage to meet all necessary criteria.

👍