Nim 1.4

Words: narimiran - lobste.rs - 12:35 16-10-2020

We are very proud to announce Nim version 1.4 after six months of continuous development!

Other than version 1.0, this is probably the biggest Nim release yet and we’re very excited to release it!

It contains exactly 900 new commits which have not already been backported to our previous versions.

There are several new features and standard library additions compared to 1.2.

We tried to keep breaking changes to a minimum, but some bugfixes weren’t possible

without making those necessary changes, and we feel that our users will benefit

from them.

We would recommend to all of our users to upgrade and use version 1.4.

This release also includes the latest version of Nim’s package manager Nimble, v0.12.0,

the changelog for which is available here.

Check out if the package manager of your OS already ships version 1.4 or

install it as described here.

If you have installed a previous version of Nim using choosenim,

getting Nim 1.4 is as easy as:

$ choosenim update self

$ choosenim update stable

If you don’t have choosenim, you can follow the same

install link as above.

Our contributors are amazing, and there are

far too many

to list here.

Big thanks to all of you, we couldn’t have pulled off this release without you!

A new reference counting algorithm called ARC was first shipped with Nim 1.2.

If you haven’t watched it already, we recommend

this video from NimConf 2020

where Araq explains details behind ARC and shows some benchmarks that

illustrate the benefits.

As far as we know, ARC works with the complete standard library except for

the current implementation of async, because that introduces cycles that

ARC doesn’t deal with.

There are other async implementations in development that don’t introduce cycles.

ORC is the existing ARC algorithm plus a cycle collector, and --gc:orc is our

main new feature for this release!

We recommend reading

this introductory ARC/ORC post

to get an idea about ARC/ORC’s advantages.

If you haven’t used ARC already with Nim 1.2, all you have to do is to pass --gc:arc

on the command line:

nim c - d : release -- gc : arc myprogram . nim

For the majority of programs this is all it takes.

If your code uses cyclic data structures, or if you’re not sure if your code

produces cycles, you need to use --gc:orc and not --gc:arc.

If you have an object that looks cyclic, but you know it isn’t, you can help ORC by

marking it as { .acyclic. } in the following way:

type

Node {. acyclic .} = ref object

left , right : Node

data : string

To read more about { .acyclic. } pragma, please

check out the manual.

If you use system.deepCopy in your code, you need to enable it via

--deepCopy:on on the command line.

This is a band-aid and a better solution for this will arrive in the future.

The reason for this opt-in switch is that ARC and ORC do not use the old runtime

type information (RTTI) that system.deepCopy is built upon.

We’ve built ARC and ORC with embedded devices in mind where the overhead of the

old RTTI has been reported to be unacceptable.

The second gotcha people might encounter is the strict ordering requirement

for custom destructors; this affects old code as well, because finalizers are

mapped to destructors.

For example:

type

CustomObject = ref object

p : ptr int

proc tricky () =

# would use the default destructor

var x : CustomObject

proc finalizer ( x : CustomObject ) =

if x . p != nil :

dealloc ( x . p )

proc newCustomObject (): CustomObject =

new ( result , finalizer )

This produces the following error message:

temp.nim(15, 6) Error: cannot bind another '=destroy' to: CustomObject:ObjectType;

previous declaration was constructed here implicitly: temp.nim(8, 7)

The reason is that only after the new(result, finalizer) call the compiler knows

that it needs to generate a custom destructor for the CustomObject type,

but when it compiled tricky the default destructor was used.

The solution is to move the proc tricky after the newCustomObject declaration:

type

CustomObject = ref object

p : ptr int

proc finalizer ( x : CustomObject ) =

if x . p != nil :

dealloc ( x . p )

proc newCustomObject (): CustomObject =

new ( result , finalizer )

proc tricky () =

# uses the custom destructor

var x : CustomObject

This is a current limitation that will be improved in future versions of the compiler.

This is an experimental feature available via { .experimental: "strictFuncs" .} pragma in

your code or via the --experimental:strictFuncs switch.

It introduces a stricter definition of a “side effect”:

If an object is reachable via a parameter that is not declared as a var parameter,

any mutation to that object counts as a side effect.

Here is an example from

the experimental manual:

{. experimental : "strictFuncs" .}

type

Node = ref object

le , ri : Node

data : string

func len ( n : Node ): int =

# valid: len does not have side effects

var it = n

while it != nil :

inc result

it = it . ri

func mut ( n : Node ) =

let m = n # is the statement that connected the mutation to the parameter

m . data = "yeah" # the mutation is here

# Error: 'mut' can have side effects

# an object reachable from 'n' is potentially mutated

For more details, please see Araq’s blog post.

Added some enhancements to std/jsonutils module.

Added a possibility to deserialize JSON arrays directly to HashSet and

OrderedSet types and respectively to serialize those types to JSON arrays

via jsonutils.fromJson and jsonutils.toJson procedures.

Added a possibility to deserialize JSON null objects to Nim option objects

and respectively to serialize Nim option object to JSON object if isSome

or to JSON null object if isNone via jsonutils.fromJson and

jsonutils.toJson procedures.

Added a Joptions parameter to jsonutils.fromJson currently

containing two boolean options allowExtraKeys and allowMissingKeys.

If allowExtraKeys is true Nim’s object to which the JSON is parsed is

not required to have a field for every JSON key.

If allowMissingKeys is true Nim’s object to which JSON is parsed is

allowed to have fields without corresponding JSON keys.

Added bindParams, bindParam to db_sqlite for binding parameters into a SqlPrepared statement.

Added tryInsert,insert procs to db_* libs which accept primary key column name.

Added xmltree.newVerbatimText support create style’s,script’s text.

uri module now implements RFC-2397.

Added DOM Parser

to the dom module for the JavaScript target.

The default hash for Ordinal has changed to something more bit-scrambling.

import hashes; proc hash(x: myInt): Hash = hashIdentity(x) recovers the old

one in an instantiation context while -d:nimIntHash1 recovers it globally.

deques.peekFirst and deques.peekLast now have var Deque[T] -> var T overloads.

File handles created from high-level abstractions in the stdlib will no longer

be inherited by child processes. In particular, these modules are affected:

asyncdispatch, asyncnet, system, nativesockets, net and selectors.

For asyncdispatch, asyncnet, net and nativesockets, an inheritable

flag has been added to all procs that create sockets, allowing the user to

control whether the resulting socket is inheritable. This flag is provided to

ease the writing of multi-process servers, where sockets inheritance is

desired.

For a transition period, define nimInheritHandles to enable file handle

inheritance by default. This flag does not affect the selectors module

due to the differing semantics between operating systems.

asyncdispatch.setInheritable, system.setInheritable and

nativesockets.setInheritable are also introduced for setting file handle or

socket inheritance. Not all platforms have these procs defined.

The file descriptors created for internal bookkeeping by ioselector_kqueue

and ioselector_epoll will no longer be leaked to child processes.

strutils.formatFloat with precision = 0 has been restored to the version

1 behaviour that produces a trailing dot, e.g. formatFloat(3.14159, precision = 0)

is now 3., not 3.

Added commonPrefixLen to critbits.

relativePath(rel, abs) and relativePath(abs, rel) used to silently give wrong results

(see #13222); instead they now use getCurrentDir to resolve those cases,

and this can now throw in edge cases where getCurrentDir throws.

relativePath also now works for js with -d:nodejs.

JavaScript and NimScript standard library changes: streams.StringStream is

now supported in JavaScript, with the limitation that any buffer pointers

used must be castable to ptr string, any incompatible pointer type will not

work. The lexbase and streams modules used to fail to compile on

NimScript due to a bug, but this has been fixed.

The following modules now compile on both JS and NimScript: parsecsv,

parsecfg, parsesql, xmlparser, htmlparser and ropes. Additionally

supported for JS is cstrutils.startsWith and cstrutils.endsWith, for

NimScript: json, parsejson, strtabs and unidecode.

Added streams.readStr and streams.peekStr overloads to

accept an existing string to modify, which avoids memory

allocations, similar to streams.readLine (#13857).

Added high-level asyncnet.sendTo and asyncnet.recvFrom UDP functionality.

dollars.$ now works for unsigned ints with nim js.

Improvements to the bitops module, including bitslices, non-mutating versions

of the original masking functions, mask/masked, and varargs support for

bitand, bitor, and bitxor.

sugar.=> and sugar.-> changes: Previously (x, y: int) was transformed

into (x: auto, y: int), it now becomes (x: int, y: int) for consistency

with regular proc definitions (although you cannot use semicolons).

Pragmas and using a name are now allowed on the lefthand side of =>. Here

is an example of these changes:

( x , y : ) x + y

( x : , y : = x + y

The fields of times.DateTime are now private, and are accessed with getters and deprecated setters.

The times module now handles the default value for DateTime more consistently.

Most procs raise an assertion error when given

an uninitialized DateTime, the exceptions are == and $ (which returns "Uninitialized DateTime").

The proc times.isInitialized has been added which can be used to check if

a DateTime has been initialized.

Fix a bug where calling close on io streams in osproc.startProcess was a noop and led to

hangs if a process had both reads from stdin and writes (e.g. to stdout).

The callback that is passed to system.onThreadDestruction must now be .raises: [].

The callback that is assigned to system.onUnhandledException must now be .gcsafe.

osproc.execCmdEx now takes an optional input for stdin, workingDir and env

parameters.

Added a ssl_config module containing lists of secure ciphers as recommended by

Mozilla OpSec

net.newContext now defaults to the list of ciphers targeting

“Intermediate compatibility”

per Mozilla’s recommendation instead of ALL. This change should protect

users from the use of weak and insecure ciphers while still provides

adequate compatibility with the majority of the Internet.

A new module std/jsonutils with hookable jsonTo,toJson,fromJson operations for json

serialization/deserialization of custom types was added.

A new proc heapqueue.find[T](heap: HeapQueue[T], x: T): int to get index of element x

was added.

Added rstgen.rstToLatex a convenience proc for renderRstToOut and initRstGenerator.

Added os.normalizeExe.

macros.newLit now preserves named vs unnamed tuples.

Added random.gauss, that uses the ratio of uniforms method of sampling from a Gaussian distribution.

Added typetraits.elementType to get the element type of an iterable.

typetraits.$ changes: $(int,) is now "(int,)" instead of "(int)";

$tuple[] is now "tuple[]" instead of "tuple";

$((int, float), int) is now "((int, float), int)" instead of "(tuple of (int, float), int)"

Added macros.extractDocCommentsAndRunnables helper.

strformat.fmt and strformat.& support specifier =. fmt"{expr=}" now

expands to fmt"expr={expr}".

Deprecations: instead of os.existsDir use dirExists, instead of os.existsFile use fileExists.

Added the jsre module, Regular Expressions for the JavaScript target..

Made maxLines argument Positive in logging.newRollingFileLogger,

because negative values will result in a new file being created for each logged

line which doesn’t make sense.

Changed log in logging to use proper log level for JavaScript,

e.g. debug uses console.debug, info uses console.info, warn uses console.warn, etc.

Tables, HashSets, SharedTables and deques don’t require anymore that the passed

initial size must be a power of two - this is done internally.

Proc rightSize for Tables and HashSets is deprecated, as it is not needed anymore.

CountTable.inc takes val: int again not val: Positive; i.e. it can “count down” again.

Removed deprecated symbols from macros module, some of which were deprecated already in 0.15.

Removed sugar.distinctBase, deprecated since 0.19. Use typetraits.distinctBase.

asyncdispatch.PDispatcher.handles is exported so that an external low-level libraries can access it.

std/with, sugar.dup now support object field assignment expressions:

/

=

x , y :

=

:

x =

y =

Proc math.round is no longer deprecated. The advice to use strformat instead

cannot be applied to every use case. The limitations and the (lack of) reliability

of round are well documented.

Added getprotobyname to winlean. Added getProtoByname to nativesockets which returns a protocol code

from the database that matches the protocol name.

Added missing attributes and methods to dom.Navigator like deviceMemory, onLine, vibrate(), etc.

Added strutils.indentation and strutils.dedent which enable indented string literals:

""" This is cool! """

Added initUri(isIpv6: bool) to uri module, now uri supports parsing ipv6 hostname.

Added readLines(p: Process) to osproc.

Added the below toX procs for collections. The usage is similar to procs such as

sets.toHashSet and tables.toTable. Previously, it was necessary to create the

respective empty collection and add items manually.

critbits.toCritBitTree, which creates a CritBitTree from an openArray of

items or an openArray of pairs.

deques.toDeque, which creates a Deque from an openArray.

heapqueue.toHeapQueue, which creates a HeapQueue from an openArray.

intsets.toIntSet, which creates an IntSet from an openArray.

Added progressInterval argument to asyncftpclient.newAsyncFtpClient to control the interval

at which progress callbacks are called.

Added os.copyFileToDir.

The =destroy hook no longer has to reset its target, as the compiler now automatically inserts

wasMoved calls where needed.

The = hook is now called =copy for clarity. The old name = is still available so there

is no need to update your code. This change was backported to 1.2 too so you can use the

more readable =copy without loss of compatibility.

In the newruntime it is now allowed to assign to the discriminator field

without restrictions as long as the case object doesn’t have a custom destructor.

The discriminator value doesn’t have to be a constant either. If you have a

custom destructor for a case object and you do want to freely assign discriminator

fields, it is recommended to refactor the object into 2 objects like this:

=

:

: y : [ ]

: z : [ ]

( x : ) =

x . x . y :

( x . y )

=

: [ ]

=

:

: y :

: z : [ ]

( x : ) =

x . :

( x . )

getImpl on enum type symbols now returns field syms instead of idents. This helps

with writing typed macros. The old behavior for backwards compatibility can be restored

with --useVersion:1.0.

The typed AST for proc headers will now have the arguments be syms instead of idents.

This helps with writing typed macros. The old behaviour for backwards compatibility can

be restored with --useVersion:1.0.

let statements can now be used without a value if declared with

importc/importcpp/importjs/importobjc.

The keyword from is now usable as an operator.

Exceptions inheriting from system.Defect are no longer tracked with

the .raises: [] exception tracking mechanism. This is more consistent with the

built-in operations. The following always used to compile (and still does):

( a , b : =

a b # can raise an DivByZeroDefect

( a , b : =

b 0 : ( , )

: = a b

The reason for this is that DivByZeroDefect inherits from Defect and

with --panics:on Defects become unrecoverable errors.

Added the thiscall calling convention as specified by Microsoft, mostly for hooking purposes.

Deprecated the { } pragma, because it was always ignored by the compiler anyway.

Removed the deprecated strutils.isNilOrWhitespace.

Removed the deprecated sharedtables.initSharedTable.

Removed the deprecated asyncdispatch.newAsyncNativeSocket.

Removed the deprecated dom.releaseEvents and dom.captureEvents.

Removed sharedlists.initSharedList, was deprecated and produces undefined behaviour.

There is a new experimental feature called “strictFuncs” which makes the definition of

.noSideEffect stricter. See here

for more information.

“for-loop macros” (see the manual) are no longer

an experimental feature. In other words, you don’t have to write pragma

{ if you want to use them.

Added the .noalias pragma. It is mapped to C’s restrict keyword for the increased

performance this keyword can enable.

items no longer compiles with enums with holes as its behavior was error prone, see #14004.

system.deepcopy has to be enabled explicitly for --gc:arc and --gc:orc via

--deepcopy:on.

Added the std/effecttraits module for introspection of the inferred effects.

We hope this enables async macros that are precise about the possible exceptions that

can be raised.

The pragma blocks { } : and { } : can now also be

written as { } : and { } : . This is the new

preferred way of writing these, emphasizing their unsafe nature.

Specific warnings can now be turned into errors via --warningAsError[X]:on|off.

The define and undef pragmas have been de-deprecated.

New command: nim r main.nim [args...] which compiles and runs main.nim, and implies --usenimcache

so that the output is saved to $nimcache/main$exeExt, using the same logic as nim c -r to

avoid recompilations when sources don’t change.

Example:

nim r compiler/nim.nim --help # only compiled the first time

'import os; echo getCurrentCompilerExe()' | nim r -

nim r compiler/nim.nim --fullhelp

nim r --nimcache:/tmp main # binary saved to /tmp/main

--hint:processing is now supported and means --hint:processing:on

(likewise with other hints and warnings), which is consistent with all other bool flags.

(since 1.3.3).

nim doc -r main and nim rst2html -r main now call openDefaultBrowser.

Added the new hint --hint:msgOrigin will show where a compiler msg (hint|warning|error)

was generated; this helps in particular when it’s non obvious where it came from

either because multiple locations generate the same message, or because the

message involves runtime formatting.

Added the new flag --backend:js|c|cpp|objc (or -b:js etc), to change the backend; can be

used with any command (e.g. nim r, doc, check etc); safe to re-assign.

Added the new flag --doccmd:cmd to pass additional flags for runnableExamples,

e.g.: --doccmd:-d:foo --threads

use --doccmd:skip to skip runnableExamples and rst test snippets.

Added the new flag --usenimcache to output binary files to nimcache.

runnableExamples "-b:cpp -r:off": code is now supported, allowing to override

how an example is compiled and run, for example to change the backend.

nim doc now outputs under $projectPath/htmldocs when --outdir is unspecified

(with or without --project); passing --project now automatically generates

an index and enables search.

See docgen for details.

Removed the --oldNewlines switch.

Removed the --laxStrings switch for mutating the internal zero terminator on strings.

Removed the --oldast switch.

Removed the --oldgensym switch.

$getType(untyped) is now “untyped” instead of “expr”, $getType(typed) is

now “typed” instead of “stmt”.

Sink inference is now disabled per default and has to enabled explicitly via

--sinkInference:on. Note: For the standard library sink inference remains

enabled. This change is most relevant for the --gc:arc, --gc:orc memory

management modes.