Add JAX device selection when formatting

What's in this PR?
After exploring for a while the JAX integration in 🤗datasets, I found out that, even though JAX prioritizes the TPU and GPU as the default device when available, the JaxFormatter doesn't let you specify the device where you want to place the jax.Arrays in case you don't want to rely on JAX's default array placement.
So on, I've included the device param in JaxFormatter but there are some things to take into consideration:

A formatted Dataset is copied with copy.deepcopy which means that if one adds the param device in JaxFormatter as a jaxlib.xla_extension.Device, it "fails" because that object cannot be serialized (instead of serializing the param adds a random hash instead). That's the reason why I added a function _map_devices_to_str to basically create a mapping of strings to jaxlib.xla_extension.Devices so that self.device is a string and not a jaxlib.xla_extension.Device.
To create a jax.Array in a device you need to either create it in the default device and then move it to the desired device with jax.device_put or directly create it in the device you want with jax.default_device() context manager.
JAX will create an array by default in jax.devices()[0]

More information on JAX device management is available at https://jax.readthedocs.io/en/latest/faq.html#controlling-data-and-computation-placement-on-devices
What's missing in this PR?
I've tested it both locally in CPU (Mac M2 and Mac M1, as no GPU support for Mac yet), and in GPU and TPU in Google Colab, let me know if you want me to provide you the Notebook for the latter.
But I did not implement any integration test as I wanted to get your feedback first.

The code below was throwing a warning:
class JaxFormatter(Formatter[Mapping, "jax.Array", Mapping]):
    def __init__(self, features=None, device=None, **jnp_array_kwargs):
        super().__init__(features=features)
        import jax
        from jaxlib.xla_extension import Device

        self.device = (
            device if isinstance(device, Device) else jax.devices()[0]
        )
        self.jnp_array_kwargs = jnp_array_kwargs

    ...

    def _tensorize(self, value):
        ...

        with jax.default_device(self.device):
            # calling jnp.array on a np.ndarray does copy the data
            # see https://github.com/google/jax/issues/4486
            return jnp.array(value, **{**default_dtype, **self.jnp_array_kwargs})

When providing device via param:
from datasets import Dataset
import jax

ds = Dataset.from_dict({"a": [1, 2, 3], "b": [4, 5, 6]})
ds = ds.with_format("jax", device=jax.devices()[0])
print(ds[0])

Producing the following warning:
WARNING:datasets.fingerprint:Parameter 'device'=TFRT_CPU_0 of the transform datasets.arrow_dataset.Dataset.set_format couldn't be hashed properly, a random hash was used instead. Make sure your transforms and parameters are serializable with pickle or dill for the dataset fingerprinting and caching to work. If you reuse this transform, the caching mechanism will consider it to be different from the previous calls and recompute everything. This warning is only showed once. Subsequent hashing failures won't be showed.

That's why I decided to map all the available devices, and assign their string representation e.g. TFRT_CPU_0 to self.device instead of jaxlib.xla_extension.Device, so that the value of the param device is washable. So on, the code that remains at the end is:
class JaxFormatter(Formatter[Mapping, "jax.Array", Mapping]):
    def __init__(self, features=None, device=None, **jnp_array_kwargs):
        super().__init__(features=features)
        import jax
        from jaxlib.xla_client import Device

        self.device_mapping = self._map_devices_to_str()
        self.device = (
            device if isinstance(device, str) else str(device) if isinstance(device, Device) else str(jax.devices()[0])
        )
        self.jnp_array_kwargs = jnp_array_kwargs

    def _map_devices_to_str(self) -> Mapping[str, "jaxlib.xla_extension.Device"]:
        import jax

        return {str(device): device for device in jax.devices()}

    ...

    def _tensorize(self, value):
        ...

        with jax.default_device(self.device_mapping[self.device]):
            # calling jnp.array on a np.ndarray does copy the data
            # see https://github.com/google/jax/issues/4486
            return jnp.array(value, **{**default_dtype, **self.jnp_array_kwargs})

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

After some investigation, it seems that when using device=jaxlib.xla_extension.Device instead of device=string it shows the warning so that later formats fail as that cannot be unpickled.
So I think we can either add that specifically in use_with_jax.mdx documentation entry I'm creating at #5535 so that the users know that they need to surroung the jaxlib.xla_extension.Device with str(), or find a workaround to override default deepcopy behavior with def __deepcopy__(self) so that the device param is converted to string if provided as a jaxlib.xla_extension.Device, but not sure if the latter works 😕
Do you think there's any other possible solution to this issue? Thanks, @lhoestq

Cool ! Specifying the device is indeed super important.
I think we can just require device to always be a string for now, and add an example in the doc on how to get the string that corresponds to a jaxlib.xla_extension.Device ? This way we never deal with objects that are not picklable

Cool ! Specifying the device is indeed super important.
I think we can just require device to always be a string for now, and add an example in the doc on how to get the string that corresponds to a jaxlib.xla_extension.Device ? This way we never deal with objects that are not picklable

Sure, then I'll restrict it to string for now! Also regarding the documentation update, should we wait until #5535 is merged so that I add this on top of that?

CI is failing due to missing resampy in librosa already being fixed by @lhoestq in https://github.com/huggingface/datasets/pull/5554

@lhoestq already moved to a global variable, I can confirm that the following now works:
import copy
import pickle

import jax
import pyarrow as pa

from datasets.formatting import JaxFormatter

_COL_A = [0, 1, 2]
_COL_B = ["foo", "bar", "foobar"]
_COL_C = [[[1.0, 0.0, 0.0]] * 2, [[0.0, 1.0, 0.0]] * 2, [[0.0, 0.0, 1.0]] * 2]
pa_table = pa.Table.from_pydict({"a": _COL_A, "b": _COL_B, "c": _COL_C})

device = jax.devices()[0]
formatter = JaxFormatter(device=str(device))

pickle.dumps(formatter)
copy.deepcopy(formatter)

Looks all good now thank you !
Is there anything else you wanted to add ? Otherwise I think it's ready for merge

Nothing else to add, I've already applied your suggestions. Thanks for your input/feedback @lhoestq :hugs:


Make upper case letter K better with Ʞ instead of ⋊

Fixes issue #9

Also changed the single K in the MIT license to the new one for consistency reasons


Fix/remove curr

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 71.119%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
394

Relevant Lines:
554

💛  - Coveralls


May we please have a screenshot mechanism?

As far as I know this isn't really possible.  There's probably good security reasons, and stuff.  That's largely over my head, TBH.
What if you required a DOM flag "screenshottable," failed if it contained any iframes or cors-tainted stuff, implied a white background, et cetera?
Being able to take images of what's going on in the browser is super useful and powerful

Both the DOM, and images of any kind, are part of the web spec, and not part of the language itself. This seems like it might be better as a web proposal than a language proposal?

fair point


Issue (Gamebreaking): Forcepower break character sheet

If you add a new forcepower to a character, the sheet doesn't update. You don't see the new forcepower. After a f5 reload, you see the forcepower, but now the sheet is buggy. All Talents are gone, specializations won't open anymore.
I consider this a game breaking bug.

After deleting the newly added forcepowers, all (older) talents are back.

it seems, that this only happen, if you add a forcepower from the compendium, without importing it before,

Just pushed a fix into dev branch.  The issue was that it was trying to check the attributes on all items, but compendium object don't have attributes.  This first part fix will get past the error.  A second part fix will be to include attributes on imported items.

Hotfix release v0.21 is now on master.

Second part of the fix is in dev.  But the first part should resolve any issues, this fix is just making sure the data structure is correct during import.

Updated the v0.21 download with the second part.


Runner scale up failed and the error is not quite clear

Today, I am trying to scale up my runner fleet, the runner container just self-restarted like this
Events:
  Type     Reason     Age                   From                                   Message
  ----     ------     ----                  ----                                   -------
  Normal   Scheduled  7m29s                 default-scheduler                      Successfully assigned default/meetup-android-runner-f6zfn-hvbjm to ip-10-181-3-182.ec2.internal
  Normal   Pulled     7m28s                 kubelet, ip-10-181-3-182.ec2.internal  Container image "docker:19.03.6-dind" already present on machine
  Normal   Created    7m28s                 kubelet, ip-10-181-3-182.ec2.internal  Created container docker
  Normal   Started    7m27s                 kubelet, ip-10-181-3-182.ec2.internal  Started container docker
  Warning  BackOff    2m24s                 kubelet, ip-10-181-3-182.ec2.internal  Back-off restarting failed container
  Normal   Created    2m9s (x3 over 7m28s)  kubelet, ip-10-181-3-182.ec2.internal  Created container runner
  Normal   Started    2m9s (x3 over 7m28s)  kubelet, ip-10-181-3-182.ec2.internal  Started container runner
  Normal   Pulling    2m9s (x3 over 7m28s)  kubelet, ip-10-181-3-182.ec2.internal  Pulling image "chenrui/actions-runner:v2.168.0"
  Normal   Pulled     2m9s (x3 over 7m28s)  kubelet, ip-10-181-3-182.ec2.internal  Successfully pulled image "chenrui/actions-runner:v2.168.0"

When I tried to check with the logs, I could not find anything useful (with runner, it is just info logs, with docker, it has something like this):
time="2020-04-15T16:30:44.882673953Z" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..." type=io.containerd.metadata.v1
time="2020-04-15T16:30:44.882858479Z" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
time="2020-04-15T16:30:44.883115566Z" level=warning msg="could not use snapshotter aufs in metadata plugin" error="modprobe aufs failed: "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1"
time="2020-04-15T16:30:44.883295667Z" level=warning msg="could not use snapshotter zfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin"
...
time="2020-04-15T16:30:45.174440728Z" level=info msg="Loading containers: start."
time="2020-04-15T16:30:45.244227388Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: ip: can't find device 'bridge'\nbridge                172032  1 br_netfilter\nstp                    16384  1 bridge\nllc                    16384  2 bridge,stp\nipv6                  524288 408 nf_conntrack_ipv6,nf_nat_ipv6,bridge,[permanent]\nip: can't find device 'br_netfilter'\nbr_netfilter           24576  0 \nbridge                172032  1 br_netfilter\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n, error: exit status 1"
time="2020-04-15T16:30:45.540762014Z" level=info msg="Default bridge (docker0) is assigned with an IP address <IP_ADDRESS>/16. Daemon option --bip can be used to set a preferred IP address"

Tested with the other runner, it seems working fine:
√ Connected to GitHub
2020-04-15 15:45:44Z: Runner reconnected.
2020-04-15 15:45:44Z: Listening for Jobs
2020-04-15 16:54:28Z: Running job: terraform
2020-04-15 16:55:20Z: Job terraform completed with result: Succeeded

here is the full log:

full log
Signature ok
subject=CN = docker:dind client
Getting CA Private Key
/certs/client/cert.pem: OK
time="2020-04-15T17:57:01.905239818Z" level=info msg="Starting up"
time="2020-04-15T17:57:01.907392150Z" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"
time="2020-04-15T17:57:01.908958121Z" level=info msg="libcontainerd: started new containerd process" pid=52
time="2020-04-15T17:57:01.909028418Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2020-04-15T17:57:01.909047549Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2020-04-15T17:57:01.909082849Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] <nil>}" module=grpc
time="2020-04-15T17:57:01.909116959Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2020-04-15T17:57:01.937120511Z" level=info msg="starting containerd" revision=35bd7a5f69c13e1563af8a93431411cd9ecf5021 version=v1.2.12
time="2020-04-15T17:57:01.937694134Z" level=info msg="loading plugin "io.containerd.content.v1.content"..." type=io.containerd.content.v1
time="2020-04-15T17:57:01.937872799Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.btrfs"..." type=io.containerd.snapshotter.v1
time="2020-04-15T17:57:01.938178407Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.btrfs" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
time="2020-04-15T17:57:01.938207120Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.aufs"..." type=io.containerd.snapshotter.v1
time="2020-04-15T17:57:01.949508715Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.aufs" error="modprobe aufs failed: "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1"
time="2020-04-15T17:57:01.949631592Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.native"..." type=io.containerd.snapshotter.v1
time="2020-04-15T17:57:01.949872500Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.overlayfs"..." type=io.containerd.snapshotter.v1
time="2020-04-15T17:57:01.950424047Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1
time="2020-04-15T17:57:01.950951168Z" level=info msg="skip loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1
time="2020-04-15T17:57:01.951052585Z" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..." type=io.containerd.metadata.v1
time="2020-04-15T17:57:01.951210991Z" level=warning msg="could not use snapshotter zfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin"
time="2020-04-15T17:57:01.951315618Z" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
time="2020-04-15T17:57:01.951365582Z" level=warning msg="could not use snapshotter aufs in metadata plugin" error="modprobe aufs failed: "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1"
time="2020-04-15T17:57:01.956762009Z" level=info msg="loading plugin "io.containerd.differ.v1.walking"..." type=io.containerd.differ.v1
time="2020-04-15T17:57:01.956910038Z" level=info msg="loading plugin "io.containerd.gc.v1.scheduler"..." type=io.containerd.gc.v1
time="2020-04-15T17:57:01.957127429Z" level=info msg="loading plugin "io.containerd.service.v1.containers-service"..." type=io.containerd.service.v1
time="2020-04-15T17:57:01.957264829Z" level=info msg="loading plugin "io.containerd.service.v1.content-service"..." type=io.containerd.service.v1
time="2020-04-15T17:57:01.957342959Z" level=info msg="loading plugin "io.containerd.service.v1.diff-service"..." type=io.containerd.service.v1
time="2020-04-15T17:57:01.957444119Z" level=info msg="loading plugin "io.containerd.service.v1.images-service"..." type=io.containerd.service.v1
time="2020-04-15T17:57:01.957529606Z" level=info msg="loading plugin "io.containerd.service.v1.leases-service"..." type=io.containerd.service.v1
time="2020-04-15T17:57:01.957631974Z" level=info msg="loading plugin "io.containerd.service.v1.namespaces-service"..." type=io.containerd.service.v1
time="2020-04-15T17:57:01.957709544Z" level=info msg="loading plugin "io.containerd.service.v1.snapshots-service"..." type=io.containerd.service.v1
time="2020-04-15T17:57:01.957821858Z" level=info msg="loading plugin "io.containerd.runtime.v1.linux"..." type=io.containerd.runtime.v1
time="2020-04-15T17:57:01.958103768Z" level=info msg="loading plugin "io.containerd.runtime.v2.task"..." type=io.containerd.runtime.v2
time="2020-04-15T17:57:01.958409905Z" level=info msg="loading plugin "io.containerd.monitor.v1.cgroups"..." type=io.containerd.monitor.v1
time="2020-04-15T17:57:01.959324080Z" level=info msg="loading plugin "io.containerd.service.v1.tasks-service"..." type=io.containerd.service.v1
time="2020-04-15T17:57:01.959436139Z" level=info msg="loading plugin "io.containerd.internal.v1.restart"..." type=io.containerd.internal.v1
time="2020-04-15T17:57:01.959611248Z" level=info msg="loading plugin "io.containerd.grpc.v1.containers"..." type=io.containerd.grpc.v1
time="2020-04-15T17:57:01.959712233Z" level=info msg="loading plugin "io.containerd.grpc.v1.content"..." type=io.containerd.grpc.v1
time="2020-04-15T17:57:01.959786925Z" level=info msg="loading plugin "io.containerd.grpc.v1.diff"..." type=io.containerd.grpc.v1
time="2020-04-15T17:57:01.959881007Z" level=info msg="loading plugin "io.containerd.grpc.v1.events"..." type=io.containerd.grpc.v1
time="2020-04-15T17:57:01.959956010Z" level=info msg="loading plugin "io.containerd.grpc.v1.healthcheck"..." type=io.containerd.grpc.v1
time="2020-04-15T17:57:01.960067808Z" level=info msg="loading plugin "io.containerd.grpc.v1.images"..." type=io.containerd.grpc.v1
time="2020-04-15T17:57:01.960162509Z" level=info msg="loading plugin "io.containerd.grpc.v1.leases"..." type=io.containerd.grpc.v1
time="2020-04-15T17:57:01.960293679Z" level=info msg="loading plugin "io.containerd.grpc.v1.namespaces"..." type=io.containerd.grpc.v1
time="2020-04-15T17:57:01.960500114Z" level=info msg="loading plugin "io.containerd.internal.v1.opt"..." type=io.containerd.internal.v1
time="2020-04-15T17:57:01.960963748Z" level=info msg="loading plugin "io.containerd.grpc.v1.snapshots"..." type=io.containerd.grpc.v1
time="2020-04-15T17:57:01.961089388Z" level=info msg="loading plugin "io.containerd.grpc.v1.tasks"..." type=io.containerd.grpc.v1
time="2020-04-15T17:57:01.961196834Z" level=info msg="loading plugin "io.containerd.grpc.v1.version"..." type=io.containerd.grpc.v1
time="2020-04-15T17:57:01.961304831Z" level=info msg="loading plugin "io.containerd.grpc.v1.introspection"..." type=io.containerd.grpc.v1
time="2020-04-15T17:57:01.961654400Z" level=info msg=serving... address="/var/run/docker/containerd/containerd-debug.sock"
time="2020-04-15T17:57:01.961921360Z" level=info msg=serving... address="/var/run/docker/containerd/containerd.sock"
time="2020-04-15T17:57:01.962034717Z" level=info msg="containerd successfully booted in 0.025530s"
time="2020-04-15T17:57:01.974776267Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2020-04-15T17:57:01.974919935Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2020-04-15T17:57:01.975066418Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] <nil>}" module=grpc
time="2020-04-15T17:57:01.975168231Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2020-04-15T17:57:01.976664640Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2020-04-15T17:57:01.976844599Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2020-04-15T17:57:01.977042247Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] <nil>}" module=grpc
time="2020-04-15T17:57:01.977249905Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2020-04-15T17:57:02.002276709Z" level=info msg="Loading containers: start."
time="2020-04-15T17:57:02.010320736Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: ip: can't find device 'bridge'\nbridge                172032  1 br_netfilter\nstp                    16384  1 bridge\nllc                    16384  2 bridge,stp\nipv6                  524288 449 nf_conntrack_ipv6,nf_nat_ipv6,bridge,[permanent]\nip: can't find device 'br_netfilter'\nbr_netfilter           24576  0 \nbridge                172032  1 br_netfilter\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n, error: exit status 1"
time="2020-04-15T17:57:02.223181931Z" level=info msg="Default bridge (docker0) is assigned with an IP address <IP_ADDRESS>/16. Daemon option --bip can be used to set a preferred IP address"
time="2020-04-15T17:57:02.410953878Z" level=info msg="Loading containers: done."
time="2020-04-15T17:57:02.455350319Z" level=info msg="Docker daemon" commit=369ce74a3c graphdriver(s)=overlay2 version=19.03.6
time="2020-04-15T17:57:02.455540195Z" level=info msg="Daemon has completed initialization"
time="2020-04-15T17:57:02.481630177Z" level=info msg="API listen on [::]:2376"
time="2020-04-15T17:57:02.481781411Z" level=info msg="API listen on /var/run/docker.sock"

@chenrui333 The runner pod is taught to restart the container on failure https://github.com/summerwind/actions-runner-controller/blob/5714459c24e7cad727eeefe2d188aedcadb47c39/controllers/runner_controller.go#L275 and the runner is given --once to stop on successful run.
So I suppose this is due to the runner container failing, not the dind container.
Would you mind sharing full logs from failed runner containers?
I guess kubectl logs --previous -c runner would work.

Actually, after bumping the runner to latest version today (v2.169.1), I have the runner working for my favor now (even though the runner still recreating the containers from time to time).

Since I have already bumped the fleet to the latest version, restart is no longer issue for me.

The container recreation would be something like this (it is re-created really fast though):

@chenrui333 Thanks for reporting!
I believe the pod recreation you're seeing currently is an expected behavior.
We do configure the runner agent to stop after each workflow run, so that the controller can recreate it and bring up an another, clean runner pod, which is important for reproducible builds.

@chenrui333 Regarding the original issue, I got to think that it's due to update failure. Reading the runner code, I see it exits with a non-zero code(3) as I've explained in https://github.com/summerwind/actions-runner-controller/issues/33#issuecomment-614585687.
That would result in the runner container(not dind container or the pod) to be stopped and restarted.

yeah, I feel the same way too, we need to do something to stop it self running.
But the container does get re-created pretty often though. :)

But the container does get re-created pretty often though. :)

I'd say it's an expected behaviour as long as it's getting recreated only after each workflow run.
Or does it get recreated without any workflow run nor any error?

Runner updates seem to be fine these days and I believe we learned a lot about how we should deal with it since then.
Regarding the ability to not restart the runner on each job run at all, we have another issue #457.
Closing this as resolved! Thanks for reporting.


Error when pushing

After pushing, I'm getting this error
yarn run v1.9.2
$ jest
FAIL  src/utils/tests/age.spec.ts
● Test suite failed to run
Plugin 0 specified in "/Users/leonardoferrari/rn/hekla/node_modules/babel-preset-react-native/index.js" provided an invalid property of "default" (While processing preset: "/Users/leonardoferrari/rn/hekla/node_modules/babel-preset-react-native/index.js")

  at Plugin.init (node_modules/jest-config/node_modules/babel-core/lib/transformation/plugin.js:131:13)
      at Array.map (<anonymous>)
      at Array.map (<anonymous>)

FAIL  src/utils/tests/prettyNumber.spec.ts
● Test suite failed to run
Plugin 0 specified in "/Users/leonardoferrari/rn/hekla/node_modules/babel-preset-react-native/index.js" provided an invalid property of "default" (While processing preset: "/Users/leonardoferrari/rn/hekla/node_modules/babel-preset-react-native/index.js")

  at Plugin.init (node_modules/jest-config/node_modules/babel-core/lib/transformation/plugin.js:131:13)
      at Array.map (<anonymous>)
      at Array.map (<anonymous>)

Test Suites: 2 failed, 2 total
Tests:       0 total
Snapshots:   0 total
Time:        1.924s
Ran all test suites.

One quick question. Did you use yarn or npm?

Thanks for the response:
Here is the output when I run jest:
 FAIL  src/utils/__tests__/age.spec.ts
  ● Test suite failed to run

    Plugin 0 specified in "/Users/leonardoferrari/rn/hekla/node_modules/babel-preset-react-native/index.js" provided an invalid property of "default" (While processing preset: "/Users/leonardoferrari/rn/hekla/node_modules/babel-preset-react-native/index.js")

      at Plugin.init (node_modules/jest-config/node_modules/babel-core/lib/transformation/plugin.js:131:13)
          at Array.map (<anonymous>)
          at Array.map (<anonymous>)

 FAIL  src/utils/__tests__/prettyNumber.spec.ts
  ● Test suite failed to run

    Plugin 0 specified in "/Users/leonardoferrari/rn/hekla/node_modules/babel-preset-react-native/index.js" provided an invalid property of "default" (While processing preset: "/Users/leonardoferrari/rn/hekla/node_modules/babel-preset-react-native/index.js")

      at Plugin.init (node_modules/jest-config/node_modules/babel-core/lib/transformation/plugin.js:131:13)
          at Array.map (<anonymous>)
          at Array.map (<anonymous>)

Test Suites: 2 failed, 2 total
Tests:       0 total
Snapshots:   0 total
Time:        1.342s
Ran all test suites.

Thank you!

What node version are you using?
node -v

Hey,
I'm running version 10.8.0.
Thanks
Enviado do Outlookhttp://aka.ms/weboutlook

De: Birkir Rafn Guðjónsson<EMAIL_ADDRESS>Enviado: terça-feira, 7 de agosto de 2018 11:02
Para: birkir/hekla
Cc: Leonardo Ferrari; Author
Assunto: Re: [birkir/hekla] Error when pushing (#6)
What node version are you using?
node -v
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/birkir/hekla/issues/6#issuecomment-411067042, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABcILb5XsE-EaCbes8KWKCvCMNGeGgYhks5uOZ3ggaJpZM4VuGvv.

This seems to be a related error: https://github.com/facebook/react-native/issues/19859
I can't reproduce it, I have another machine I can try a fresh install to reproduce, but as this is a known issue, you can try some of the steps mention in the thread and report back?
Thanks Leonardo

Fixed it, thanks!
Gonna send you a PR :D


Remove explicit LangVersion

This is now set globally to 7.1: aspnet/BuildTools#408

@Tratcher,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot


AuthSource implemented in Production Flavor

Don't you think you need to switch to Build Flavor evertime you need to test your app .The problem with that is every time you need to add a feature to your presenter class you need to switch your variant and test that feature you need switch back to Production build. If you count the time you are waiting for a build finish its huge.
Also what I have gathered from internet is you only need to remove Logging from your app in your production build and keep logging in your Debug built with your current architecture it is not possible .Because there is not Implementation of AuthSource in Debug build.

I'm curious as to what purpose you would use Build Variants/Product Flavors, if not to keep your Test Classes separate from your Production Classes, or swap out RxJava Schedulers based on whether you are running on the JVM or Android OS? These seem to me as primary Use Cases for these features in Gradle/Android Studio. Many repositories I've looked at from Developers much better than me or you also follow this convention.
I must ask, if not Build Variants, how would you go about keeping your Test sources separate from Production sources in absence of Build Flavors?
On your second point, I believe you are correct. However, I don't tend to use Logs for much at this point in my debugging process, and I like to remove them the moment they become useless (when I remember to anyways).
Thank you for the feedback.

I'm curious as to what purpose you would use Build Variants/Product Flavors, if not to keep your Test Classes separate from your Production Classes
You are right about having an Abstract Layer for Testing and mocking purpose. But you can have an implementation in your Debug built rather than pushing it on your production built. so rather than using Implemented class of service use the abstract class and change it's implementation to MockClass in testRepo


Support markdown in chat messages

related to issue
Here I added the a component for the message body which sanitises and generates html from markdown.

TODO: Replicate the same logic in chat-widget when the PR is approved

hey @ZmagoD thanks for tackling this! can we remove the margin from the bottom of the messages?
I'm not a fan of all this extra space at the bottom:

also side note: if you ever want to chat with us directly about whatever you're working on, feel free to join our slack or discord channels https://github.com/papercups-io/papercups#get-in-touch 😄

@reichert621 I managed to get rid of the empty space now that I'm testing with this I see that the styles are a bit off. Do you have some style guides how it should look like?

Another question 😅
Should we permit to add images? The problem happens if someone inputs something like /asset/images/sample.png, we will get a server error.

@ZmagoD we should probably not worry about images for now, does that error only happen when someone specifies a relative path? i'm not quite sure i understand how to reproduce 😅
regarding general style, we're trying to stick as close to this as possible https://ant.design/components/typography/
(I'm tempted to experiment with https://github.com/rexxars/react-markdown because it might make it easier to customize how things are rendered by using react components)
sorry for all the thrash on this 😭

No problem @reichert621  😄
Yes it makes more sense to use an already made component 😄 I will change the library, but maybe we will have the same problem with styles 🤔

@ZmagoD what are the main problems with the styles? is it mainly about keeping them the right color?
sorry this turned out to be trickier than i expected 😢 😛

I think the markdown we want to prioritize is probably just things like bold, italics, links, maybe this, and whatever you think most people would need in messages like this -- headers tags are probably not important at the moment!

No problem @reichert621 😄
yes basically the styles can be inconsistent because of the theme we use. I will add just bold italic links and pre. 🤔 maybe it will be better to use something like https://github.com/yabwe/medium-editor or http://wangzuo.github.io/react-medium-editor/ like slack uses WDYT?

hmm do those libraries support rendering the markdown when the message is sent, or just showing formatted text in the editor when typing? (it looks like it's only meant for the editor?) looks cool though!

I had a case like this in the past.
This library outputs HTML and I converted to markdown before save in the DB. When I rendered on the FE then I used https://github.com/rexxars/react-markdown to render it. But maybe this adds too much complexity in the project 🤔
I'm not sure what would your users like more 🤔

@ZmagoD let's start off with just rendering the markdown correctly, and then handle improving the editor in another ticket 😛 but i really like the idea!

@reichert621 I changed the library and updated the styles

@ZmagoD this looks great!! thanks so much for doing this :) I'm gonna play around with it first and then will merge if it all works as expected 👍


error: failed to final exec copy: pq: invalid input syntax for type numeric

I'm trying to copy the contents of a table in one PostgreSQL database to another PostgreSQL database.
In the first database, I create the data:
create table t1 (id text, price numeric);
insert into t1 values ('a', 100.01),('b',99.40),('c',5);

In the second database I create the table:
create table t1 (id text, price numeric);

Then I execute the copy command:
./usql -c "\copy pg://postgres:password@url.to.database:5432/postgres?sslmode=disable pg://postgres@localhost/postgres?sslmode=disable 'select * from t1' 't1'"

The result is:
error: failed to final exec copy: pq: invalid input syntax for type numeric: "\x3130302e3031"

It doesn't matter if I try to copy from the remote to the local pg instance or vice versa, the error is always the same.

Thanks for catching this. The \copy command is very new, and in a very experimental state at the moment. There's a lot of logic involved with converting the underlying Go drivers' values. We'll try to fix this, but I would suggest in the interim trying to copy to a separate database or file first, or to avoid using the numeric type for now, until we fix the value conversion.

I just verified this occurs with the DECIMAL data type as well.  I'll do some more experiments with it.

Can you try casting it to another type, like a varchar, in the SELECT?

As an alternative, is there a way to open connections to 2 databases at once and query between them, such as:
\c db1
\c db2
select * from db1.table into db2.table

pg:postgres@<IP_ADDRESS>:49153/postgres=> create table t1 (id text, price numeric);
CREATE TABLE
pg:postgres@<IP_ADDRESS>:49153/postgres=> create table t2 (id text, price numeric);
CREATE TABLE
pg:postgres@<IP_ADDRESS>:49153/postgres=> insert into t1 values ('a', 100.01),('b',99.40),('c',5);
INSERT 3
pg:postgres@<IP_ADDRESS>:49153/postgres=> \copy 'pg://postgres:pw@<IP_ADDRESS>:49153/postgres?sslmode=disable' 'pg://postgres:pw@<IP_ADDRESS>:49153/postgres?sslmode=disable' 'select * from t1' t2
error: failed to final exec copy: pq: invalid input syntax for type numeric: "\x3130302e3031"
pg:postgres@<IP_ADDRESS>:49153/postgres=> \copy 'pg://postgres:pw@<IP_ADDRESS>:49153/postgres?sslmode=disable' 'pg://postgres:pw@<IP_ADDRESS>:49153/postgres?sslmode=disable' 'select id,price::varchar from t1' t2
COPY 3
pg:postgres@<IP_ADDRESS>:49153/postgres=> select * from t2;
┌────┬────────┐
│ id │ price  │
├────┼────────┤
│ a  │ 100.01 │
│ b  │ 99.40  │
│ c  │ 5      │
└────┴────────┘
(3 rows)

LOL, I just tried cast(price as text) and that worked as well.

And even though I'm casting them as TEXT they still go into the database as numeric.  That works, it's just not what I'd expect.

@burggraf the reason here is that due to the way the Go SQL drivers work, we send a *interface{} to the SQL driver, and have the SQL driver return the type. By forcing it into a text or varchar type, the postgres driver will return the stored value as a string (as a base 10 decimal format). The type that gets sent back for numeric/decimal isn't well supported by the \copy command yet. In turn, the receiving database (not the driver!) will likely know how to interpret converting a string (passed by the driver) into the destination type. If a problem like this occurs with \copy, then one will need to CAST fields in the SELECT statement to an appropriate type -- whenever this is necessary, in almost all likelihood, this will almost always be the origin database's TEXT equivalent.

I tried noting that in the README. If it's not clear enough and you'd have suggestions on how to improve that section please open up a PR.

As an alternative, is there a way to open connections to 2 databases at once and query between them, such as:
\c db1
\c db2
select * from db1.table into db2.table

This would require parsing the query, which I don't think we want to do. But we were discussing using variables for connection strings, so the copy command would be a bit shorter:
\set conn_src pg://postgres:pw@<IP_ADDRESS>:49153/postgres?sslmode=disable
\set conn_dst pg://postgres:pw@<IP_ADDRESS>:49153/postgres?sslmode=disable
\copy conn_src conn_dst table_src table_dst

They you could keep the \set commands for commonly used connetions in your ~/.usqlrc.

@burggraf could you also try the pgx driver? It might handle types differently.
Is there something we can do here to improve this with the pg (libpq) driver? Maybe update the description in the README to stand out better? If not, can we close this issue?


Provide configuration for enabled SSL protocols

Provide configuration for enabled SSL protocols.
Upgrade netty to 4.1.34

@barchetta This requires PLS approval work for the netty version update.

@tomas-langer I agree, it makes sense to add the new "enabledProtocols" attribute to SSLContextBuilder rather than SocketConfiguration so that the config remains consistent.
However, SSLContextBuilder builds javax.net.ssl.SSLContext and does not hold the new attribute that we need to pass to the netty JdkSslContext constructor.
I'm thinking of creating a subclass of javax.net.ssl.SSLContext called io.helidon.webserver.SSLContext and have SSLContextBuilder return that.
Do you have any thoughts or suggestions ?


Error message does not show the line information anymore?

Note: this is the technical bug tracker, please use other platforms for getting support and starting a (non technical) discussion. See the getting help page for details.
I'm submitting a ...
[ x ] bug report
[ ] question about the decisions made in the repository
Action taken (what you did)
Tried to run my own bot, and I got just this on error on the command prompt:
xxx POST /api/backtest 500 623ms -
Error: non-error thrown: [object Object]
at Object.onerror (C:\Gekko\node_modules\koa\lib\context.js:105:40)
at
at process._tickCallback (internal/process/next_tick.js:188:7)
Expected result (what you hoped would happen)
Shouldn't there be a more specific information about the error and the cause of it? If I remember right, previously error messages told exactly the filename and the faulty line in it, but not on 0.6.6 anymore??
Actual result (unexpected outcome)
Cannot locate the line which caused the error.
Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, etc)

Make sure you dont require talib or tulip node libraries for your strategy. That error is quite generic indeed but from past experience it seems to be related to not having require libraries installed for node and your strategy/indicators.

Thanks but no, I don't use talib or tulip. And my method has been working previously just fine. But now, after maybe a month or so, I upgraded to the newest Gekko and started to code again, and noticed that I do not get valid error messages at all if I write dome bug on my bot??

Anyone? I still cannot get any error messages shown, which makes the development quite difficult since I don't get any clue if I make some mistake on my code..

Apologies for the late reply. Please see this video guide: https://www.youtube.com/watch?v=6-74ZhrG0BE
About your specific issue:
The latest version (both stable release & develop branch) should log the error, can you verify you are running the latest version? But after that: please run the strategy over the commandline, which is directly running your code in the same process. Using the UI there are a few layers in between, it should bubble through these errors.
I do want to look at it, but please provide a way to reproduce your problem. Since I currently cannot reproduce it locally.

Just to be sure, I downloaded the latest 0.6.6 version and extracted all files to my Gekko- folder. I don't know why, but something really changed; now I can't get any results for any strategy, they just keep on running with a text "Running Backtest..." on the UI, without stopping at all. I mean, even if I take some built-in example strategy such as MACD or RSI, I don't get any results for them, but I don't get any error messages anywhere either??

We are not having these issues, can you run them over the commandline with debug on and post logs?  Here is a guide how to if you are not aware: https://forum.gekko.wizb.it/thread-57581.html

Ok, going forward... I tried to run both MACD- and my own strategy from the command line, and they worked just fine, without any errors! :=)  And if I add some bug on the code, I'm getting a valid error message just like before.
But still, running the same backtest from UI makes the tester just halt, without any error messages shown anywhere. How could I help you to solve this problem somehow..?

By changing some code you can force log messages in the UI, read here how: https://github.com/askmike/gekko/issues/2451#issuecomment-413782847

Ok, now I did that and got more info on the screen. I can see that the backtester is running just fine, and my bot is doing some simulated trades. No errors anywhere.
But, for some reason it seems that the testing is started all over again! After the testing is done first time, the line "Setting up Gekko in backtest mode" is written again, and exactly the same trades are listed on the CLI again. And the UI does not give any test report, it just comes back to a starting point with blue "Backtest"- button available.

Now I went back to 0.6.5 and backtester works just fine with it, even thru the UI. So there really might be something wrong with the backtester UI in 0.6.6?
And with 0.6.5 I don't get any valid error messages if there's a bug in my code, but maybe this was already fixed on 0.6.6.?

@jasurakk please provide a way for us to reproduce your problems. I am not experiencing your problems..

I would be more than happy to do that, if I just would know how?? But I'll try to dig some deeper in a few days, let's see what can I get.


Add About us section to the home page

Issue Fixes #4
Added About us section
Linked it to navbar and footer

@pandey22 looks good, Thank You!


Beacon icon: Music note

Can you please add a music note to the list of icons for beacons? 🎵 or 🎼 or something?

Sure, I will add that
Can you also clarify what the music note would mean in terms of a location? I want to make sure I choose a standard map icon if that exists.

Several uses, at different scales:

Bandstand(s) at a big music festival.
Dance hall at a sleepaway dance camp.
Group campsite where a spot is designated for musical campers to have a jam session (far enough from tents for others to sleep). 😉

Thanks!


Use camelize instead of capitalize to support CamelCase model name.

Hi!
It's just a quick patch make classify to accept CamelCase model name.

@ka2n Thank you for your pull request!! 🙏

This change has just released as v0.3.3.

Thank you!


Keep prefixing output with /usr/local/src/your-app/

Hi,
When using -o flag, the output location is being prefixed with /usr/local/src/your-app/.
Thanks
Command
docker run -it --rm -v "$(pwd)":"$(pwd)" ferrarimarco/github-changelog-generator github_changelog_generator -u myself -p my-project -t my-token -o /my/path/to/app/CHANGELOG.md

Output
Using these options:
:date_format=>"%Y-%m-%d"
:output=>"/my/path/to/app/CHANGELOG.md"
...
...
Done!
Generated log placed in /usr/local/src/your-app//my/path/to/app/CHANGELOG.md

Hi @BentCoder !
What's exactly the issue? :)

This is the output I define: -o /my/path/to/app/CHANGELOG.md
This is what command reports: :output=>"/my/path/to/app/CHANGELOG.md"
But this is where it says it will generate the output: Generated log placed in /usr/local/src/your-app//my/path/to/app/CHANGELOG.md
Instead I would expect to see: Generated log placed in /my/path/to/app/CHANGELOG.md

To anyone who finds themselves here - this confused me too. Changing the -o parameter just changes where the file is dropped inside the container (see the Dockerfile). The way to control the output path is to either move to a new pwd (easiest) or mount the container-side /usr/local/src/your-app to another local directory by replacing $(pwd) in the docker run command (also very doable).
I really appreciate having the docker option for this helpful tool! So much better than maintaining Ruby and installing gems. Thank you @ferrarimarco 😄

+1


adding mfbButtonClose to be able to close menu on click or losing focus

adding a directive that close the menu on click:
<button mfb-button mfb-button-close></button>

If the toggling method is CLICK then as soon as you lose the menu focus (opening a modal or moving mouse out) the menu will close. If the toggling method is HOVER then nothing change.
To be able to access close, open and toggle methods from another directive I change mfbMenu link to controller.
therefore I can access mfbMenu controller that way:
return {
      restrict: 'A',
      require: '^mfbMenu',
      link: function($scope, $element, $attrs, mfbMenuController) {
        $element.bind('click', function() {
          mfbMenuController.close();
        });
      },
    };

I reorganized methods a little bit to follow angular-styleguide and added a .jsbeautifyrc file for Atom users.
Tests are ok:
PhantomJS 1.9.8 (Linux): Executed 22 of 22 SUCCESS (0.052 secs / 0.21 secs)

If you have any question feel free to ask,
Julien

Hi Julian, looks like a very good contribution to me :+1:
Before I merge can you please add a little documentation/quick example about this in the README so users will be more likely to leverage this functionality?

Sure :)

Done

@nobitagit Dude did you see that: https://material.angularjs.org/0.10.0-rc2/#/demo/material.components.fabSpeedDial

@shprink ahah, nope, thanks for the link. Well, that looks good, what do you think?
It was just a matter of time i guess, and it's also the reason why I never really wanted to tie this directive to Angular Material too much (and also provide it without any framework for those who want). As it has no dependency on ng Material it will always be there for those who just need the menu and nothing else.
Cheers! :smiley:

I know, a matter of time ;)


NATS.io support

Is there any appetite for a native integration with NATS.io? Being a serious contender to Rabbitmq and nservicebus it would be a great addition to the platform.

I'm all in favor of more transport options, yes


Drop the node.loggly.token.from_databag attr

Hi.  It seems the upstream cookbook is largely unmaintained and you now have the most up-to-date fork.  So I'm submitting this PR your way.
It drops the loggly.token.from_databag attr in favor of just checking whether loggly.token.value is set.  It probably seems like a trivial thing, but it actually bypasses a separate issue we encountered where AWS CloudFormation can't pass boolean json attrs to AWS OpsWorks stacks.  Given that this particular boolean attr isn't really necessary, this patch both makes the overall attrs cleaner and fixes our specific issue passing boolean chef attrs to OpsWorks.

Hey @ziuchkovski, sorry for the delay, I've been away.
Yes, this is the right fork to send pull requests too, it's the only one being maintained anymore. I'll review and merge this tonight!

Just saying this merge didn't happen since 1 year + 1 month :)


symmetric-padding support

I believe I made the necessary changes in the savant codebase to include symmetric-padding support, but I get this error:

I believe omegaconf is not recognizing the changes? Any guidance on how to include symmetric-padding support?

You are right that there is no key 'symmetric_padding'. So far, you can use it only when you pass it with Nvinfer config directly, but not with YAML.
Samples of how to do that are here: https://github.com/insight-platform/Savant/tree/develop/samples/nvidia_car_classification/flavors
However, we will implement the parameter shortly. If you like you can track the issue and see when it is closed in a couple of days.

@placccebo you can take a look here on how to do that:
https://github.com/insight-platform/Savant/pull/816
Use your identity demo to check that it is implemented correctly.

@pranavsrihari123 hi. The feature is merged.
A sample of the use can be found here: https://github.com/insight-platform/Savant/tree/develop/samples/bypass_model
Please wait until the job to complete: https://github.com/insight-platform/Savant/actions/runs/10283152157
After that, pull updated containers.

@pranavsrihari123 this job: https://github.com/insight-platform/Savant/actions/runs/10284461471

Excellent, thanks


Documentation for Deploy web applications and SQL Azure database using an ARM template

Is your feature request related to a problem? Please describe.
To me it is unclear how to obtain the values for

AD Application ID
AD Application Secret
ADMT Application ID

My understanding is that these are created during the install.
They are also not required for the other deployment options.
Describe the solution you'd like
I would like a description on how to obtain these parameters.
Describe alternatives you've considered
Additional context

Hi Michael.
For the AD ApplicationID and ADMT Application ID, those values are present in the output of the deployment script (at the end of running).
The ClientSecret is not written in the console output for privacy / security reasons. It is added to the KeyVault and referenced in the app automatically. In theory, you should not need it, but if you do, then you can read it from the KeyVault (if you're using v6 of the accelerator) or from the admin portal web app configuration section (if you're using a version before v6)

Hi Alex,
Thank you for answering.
Yes, the script shows the values after the deployment. My question was about the ARM template. There the values had been requested upfront. However, the link to the ARM template on https://github.com/Azure/Commercial-Marketplace-SaaS-Accelerator/blob/main/docs/Advanced-Instructions.md#deploy-web-applications-and-sql-azure-database-using-an-arm-template to https://github.com/Azure/Commercial-Marketplace-SaaS-Accelerator/blob/main/deployment/Templates/deploy.json does not work anymore. So, I think the ARM template is obsolete now.

Hi,
Confirming @MichaelSchreiber's note that the ARM template which should be found at /deployment/Templates/deploy.json is not there.
Also, attempting to install using the basic instructions on using Azure Powershell ran into time out issues, wherein Azure would close the powershell session.  If there is a way to prevent Azure from closing the session, it would be helpful to add to instructions.
Will continue on with lower sections to see if I can figure out how to set this up.  Thanks for all the hard work the team has put into this.

Hi @MichaelSchreiber /@ryanperrymba,
Thank you for raising this issue.
For now, we are hiding the advance installation docs to update to the latest changes. We will keep this issue open untill we have the docs updated.

@MichaelSchreiber Apologies for missing your reply (please feel free to @ me so I receive a notification)
You are correct, we do not deploy using the ARM template anymore as it has been replaced by the PowerShell script. The PS script contains all functionality (and extra) from the ARM template and that includes creating the App Registrations needed for the Marketplace integration. You can also supply your own App Ids if you already have them

@alexdrenea Thank you for the explanation. From my point of view the issue can be closed.


Add a weighted grandpa as a "safety" validator that runs the default node

We want some guarantees in early days that a known node is being followed. Proposal is to have a validator with oversized weight that all follow early on, so finalization happens from a known, unaltered deployment.
Probably need to fork polkadot repo to add this option.

 Authored by blakebyrnes
Dec 24, 2023

@dependabot rebase

 Authored by dependabot[bot]
Dec 24, 2023

Looks like serde_json is up-to-date now, so this is no longer needed.


fix: mysql ON UPDATE not taking an expression

Attempt to resolve the #543, the PR resolves the issue however I was not sure on the test construction so there is currently a pending todo before this can be merged. As per my comments on #543 any assistance/guidance would be appreciated.
Also not sure the extra enum variant is the best way to do this, but wasn't able to pull the tokens and expr from DialectSpecific variant.
closes #543

I think the enum idea is a good one

FYI, I am planning to make a new sqlparser-rs release in the next day or two. This PR has some outstanding comments and appears to have have stalled

Marking as draft to signify this PR has been reviewed. Please mark this PR as ready for review when it is ready again.

Commit updated:
Added verification the expr is one of 'CURRENT_TIMESTAMP', 'LOCALTIME', 'LOCALTIMESTAMP', 'NOW'.
Fixed test
Added test for invalid expr parser error

@alamb Just had a quick review and found an improvement could be made, so added a re

Added NOW keyword, included that in the parse time functions match arm
Fix check name to be case insensitive
Added test for lowercase now() fn use.

Given https://github.com/sqlparser-rs/sqlparser-rs/pull/602  has something similar, perhaps I can try and push this one through

I believe this PR is no longer necessary after https://github.com/sqlparser-rs/sqlparser-rs/pull/685. Can you please confirm @Sibz ?


Performance issue loading page with large number of selectpickers with relatively large number of options

Summary:  I am building a page that contains a large number of select elements (~200) each with a large number of options (~500).  When using selectpicker for these elements the page has a very long load time and encounters the dreaded long running script warning on IE (performance is especially bad on IE).  I made a few tweaks to reduce the load times and have attached the test with the tweaked code.  Not sure if I have broken any functionality or overlooked any gotchas, but so far the changes seem to work correctly.  I wanted to provide these changes to see if some variation could be incorporated into the project to provide similar performance improvement.
Environment details:  I encountered the issue in chrome, firefox and IE.  Performance is by far the worst in IE.  You can see the versions of jquery, bootstrap, and bootstrap-select in the attached example.
I tried to add appropriate comments in the code to explain the changes, but here is a rough summary of the changes:

changed code in render to only loop all options when select is multiple.  Otherwise just jump to the selected option using native JS.  Not a huge savings on chrome, but makes a bigger impact on IE.
added a lazyLoadLiElements option.  This option basically tells the selectpicker not to actually render the content on the dropdown menu when doing the inital load.  A bunch of expensive operations are skipped and then we bind an event so that when the dropdown menu is activated for the first time, we actually render its contents.  (it makes the the first time a little slower.  it is hardly noticable on chrome.  on IE it has a slight pause).
As part of lazy load, I disabled the firing of the rendered event on initial loading.  This event is super expensive on IE. Looking at the code, this may cause an issue if an invalid option was already selected when the page loaded.  Perhaps this functionality can be implemented differently and you could provide an option to turn of this event for those that don't need it.

Instructions for running the test:

extract contents of the attached zip file

load slow.htm in your browser

press "Create Select Fields(native)" button.  This will create 200 native select elements each with 500 options on the page

press "Selectpicker (orig)" button.  This will change the native selects into selectpickers using the original implementation and report the run time.

press reset (or refresh the page)

press "Create Select Fields(native)" button.  This will create 200 native select elements  each with 500 options on the page

press "Selectpicker(+perf changes)" button.  This will change the native selects into selectpickers using the performance tweaks described above and report the run time.

In my testing, I was seeing around an 80% improvement on chrome and around a 95% improvement on IE.
selectpicker.zip

Because we're on github with all of these PR things :-) here's a PR with extracted changes from the archive.
We've tried it on some of our pages (worst scenarios) where we have 540 bootstrap-select components with 35 options each. With the changes above Chrome browser spends in init() function ~3s while without those changes it took ~10s according CPU profiling.

could try lazy-loading https://github.com/snapappointments/bootstrap-select/issues/2187

Most of your enhancements have been addressed in one way or another in the latest release - v1.13.6. Using your slow.htm page but with the latest version results in the whole page being loaded in under half a second in Chrome 72. Thanks for your contribution!