Refresh account credentials and Add an account doesn't work

Azure Data Studio Version: 1.20.1
OS: macOS Catalina 10.15.6

Steps to Reproduce:

Under CONNECTIONS, click New Connection icon
In Connection Details section:

When an account is selected, clicking on "Refresh account credentials" doesn't do anything
Selecting "Add an account..." also doesn't do anything

In either cases, I would expect browser to launch and user would need to enter their credentials - but browser is not launching. I'm getting the same issue by downgrading ADS to previous version.
Version: 1.20.1
Commit: fe6312306647441b30102a64bb43abb06ebcf2be
Date: 2020-07-17T02:32:14.131Z
VS Code: 1.47.0
Electron: 7.3.2
Chrome: 78.0.3904.130
Node.js: 12.8.1
V8: 7.8.279.23-electron.0
OS: Darwin x64 19.6.0

@aaomidi do you know what could be happening in this scenario?

@hardevs Could you please try with the latest insiders build: https://github.com/microsoft/azuredatastudio#try-out-the-latest-insiders-build-from-main

@aaomidi  Insiders build works just fine.

@hardevs Could you please try with the latest insiders build: https://github.com/microsoft/azuredatastudio#try-out-the-latest-insiders-build-from-main

This fix will be in the next version of ADS :)


Update ccxt and address moderate vulnerability with class-validator

Address moderate vulnerability with class-validator
Towards addressing issues with FTX and KuCoin

A user in the Jackrabbit support Discord server reported that this (docker image) build from this code fixed things for them


[BUG] Old folders are left in case of a name change.

Description
You installed version 1.0.0 and the mod was in a folder named "X-Y-Z", then 1.0.1 is released, you update, but the folder is in a folder named "XYZ", Papa doesn't check and delete the folder, but just installs the new update.
This results in having two versions of the same mod, but with different versions.

Perhaps this could be solved with #19?

fixed in 97351f40798fad2685aa9f78d2962853e37c9b55


accessing a disposed binding (through a closure) could be an error

Consider
{
  using x = foo();
  setTimeout(() => consume(x), 10);
}

If I understand correctly, this will access a disposed resource.
Could we make accessing a disposed binding a TDZ-style error instead? Any reason this should be legal?
(You can accomplish this sort of leak in other ways, of course, but leaking a closure which captures a binding is a particularly easy mistake to make.)

Calling @@dispose on an object doesn't meant that the object becomes unusable, it means that the resources that object was holding should be freed. This can often be a 1:1 correlation, but that's not always the case.
For example, you could still access the incoming headers or url associated with an http server response after the response has been disposed. The object identity itself could still be useful if a disposable is used as a key in a map, to be cleared out in a later turn. Disposables can also potentially be reusable.
In C# and Java its usually the responsibility of the author of the resource to throw if it is accessed in a way inconsistent with its internal state.
Let's say foo() produced a NodeJS FileHandle. Calling consume() on a closed handle will likely throw if you try to call read() or write() anyways, so making the binding itself unusable doesn't improve anything, and forces you to create a separate const binding if you do need to reference it in a closure that runs later.

It's true that there are some cases where you explicitly want the object to outlive its disposing. But it's trivial to accomplish this by introducing a new binding, in those unusual cases. On the other hand, we have years of experience telling us that people don't easily notice errors like the above, and I think guarding against them would be worthwhile. It's true that sometimes they'll get a different error from trying to use the resource after it has been disposed, but not all resources will provide a helpful error for those cases, and the language can help.
C# and Java are not particularly good analogies because it is much rarer to create closures in those languages.
The whole point of the RAII style is that the binding is live for the life of the block, and then the binding is discarded and the resource disposed. If the binding outlives the block it's not coherent.

I'm a bit wary about introducing a new TDZ behavior, but will consider it if it becomes necessary for advancement.

I'd also worry of having to explain that using is a const, except that it will be reset to undefined after disposal.

I am also wary of adding a new TDZ behavior. On the other hand, I am also wary of adding a whole new class of use-after-free bugs, and the latter concern weighs more heavily on me.
@mhofman My suggestion is that accessing it would be an error, not undefined.

Copy some discussion from the matrix:
@syg wrote an example of recursive mutex which will access the binding of outer scope in the inner scope:
// https://gist.github.com/syg/26e303748b5ebc3ed5206a2b875293fb
{
  using recLock = lockSomething(); // returns a locked recursive mutex

  executeEitherNowOrLaterAsTaskUnderSomeConditions(() => {
    using innerRecLock = recLock.lock(); // returns itself with depth+1 if already locked, otherwise blocks
  });
}

I guess @bakkot would like to refactor this code to
{
  const recLock = lockSomething(); // returns a locked recursive mutex
  using _ = recLock;

  executeEitherNowOrLaterAsTaskUnderSomeConditions(() => {
    using _ = recLock.lock(); // returns itself with depth+1 if already locked, otherwise blocks
  });
}

That would be a refactoring hazard if lines are added between the const and using declarations, and I would feel safer with the following style:
{
  using _recLock = lockSomething(); // returns a locked recursive mutex
  const recLock = _recLock;

  executeEitherNowOrLaterAsTaskUnderSomeConditions(() => {
    using _ = recLock.lock(); // returns itself with depth+1 if already locked, otherwise blocks
  });
}

I remain against adding a TDZ. I both disagree with the language design choice of being paternalistic here, and I dislike the implementation complexity (not necessarily difficulty) that additional TDZ behavior introduces.
More details and recapping of my thoughts from Matrix discussions below.

The badness of closing over stack-scoped things with destructors in other languages is primarily due to risk of actual use-after-free, as in freed memory, causing security bugs. This is not an issue in JavaScript, these are not use-after-frees. I expect non-reusable objects to throw anyway when operating on them after their @@dispose is called, so the main value add of re-entering TDZ is to throw a earlier in the evaluation. I find that paternalistic for not much gain in practice.
I don't think using an object who has had @@dispose called on it is even necessarily bad. The current design is more flexible and allows you to have reusable objects. Yes, a workaround exists, but it's kind of gross and since I disagree with the motivation that this is badness that ought to be prevented, I do not want to make this proposal less flexible to begin with.
In implementations, I suspect the free variable accesses of using bindings will end up with more TDZ check-and-throw cost than existing lexical bindings. This is undesirable for performance, both runtime and bytecode or JIT code size. For the same reasons that linters need to be very sophisticated to correctly lint misuses here, so will implementations need to be very sophisticated to optimize these away. I do not want to penalize context loads of using bindings. Specifically I want load costs of using bindings to be the same as other lexical bindings.
In implementations, it is nice to be able to reason about TDZs as one-way: once a binding is initialized it never uninitializes. This makes optimizations easier to do. Changes to this behavior can add complexity to every layer from bytecode compiler to baseline JIT compiler to optimizing JIT compiler. Combined with my other leanings, this complexity is not something I think is justified.

Thanks for the comment, @syg.
Our main disupute is the question of language design. I am perplexed by your view here. Why does it make sense to call @@dispose when the binding goes out of scope if you think it's coherent to continue to use the binding (as opposed to the object) after that point? The notion that the point in time at which the binding goes out of scope is the point in time at which you stop referring to it - i.e., that when it goes out of scope you are doing using it - is the only thing which seems to me to suggest that's the appropriate time to call @@dispose. If you disagree with that notion, I don't understand what going out of scope has to do with anything.

Why does it make sense to call @@dispose when the binding goes out of scope if you think it's coherent to continue to use the binding (as opposed to the object) after that point?

I suspect the disagreement comes down to because I see the value primarily as running "on scope exit" behavior. I am supportive of the resource management use case being a primary driver and getting extra syntax affordance via using, but I also don't want other use cases to need unnecessary workarounds like aliasing.
You've brought up lexically scoped defer before, which is salient here too. I have preference for the current syntax affordances on pure pragmatic grounds: I figure I'll have organized my code to have objects with on-scope-exit methods, some of those objects may be reusable. But if you were to force me to choose having using bindings + TDZ or defer, I'd choose defer.

I suspect the disagreement comes down to because I see the value primarily as running "on scope exit" behavior. I am supportive of the resource management use case being a primary driver and getting extra syntax affordance via using, but I also don't want other use cases to need unnecessary workarounds like aliasing.

I don't understand how the second sentence here relates to the first. If all you're doing is registering something to run at scope exit, you don't need a binding at all, so surely it's not problematic to be unable to use the binding after goes out of scope. When we're talking about reusing the binding, we're necessarily talking about something beyond just registering a function to run on scope exit.

I'm saying the fact that there are bindings to express this isn't that important to me -- I'm happy for it to exist, but not happy if it incurs complexity.

I understand not wanting to incur the implementation complexity, and if this ends up being one of the places we go with a less-than-ideal design because of implementation complexity, that's not the end of the world. But I don't at all understand the argument that it would be wrong choice from a language design perspective.

I generally agree with shu, post-free TDZ adds too much complexity here. Since the object being using is a resource, like a File, the File object can throw on its own to prevent invalid usage.
In other words (I'm not serious about the following suggestion 🤔), if you really want to prevent the use-after-dispose (with no workaround), we can change the semantic to this:
using x = expr
// equals to

const _1 = expr,
      _2 = %Proxy.revocable(_1, { __proto__: null })%;
%AddDisposableResource%(%env%, null, ~sync-dispose~, _2.revoke);
%AddDisposableResource%(%env%, _1, ~sync-dispose~);

const x = _3.proxy;

In other words (I'm not serious about the following suggestion 🤔), if you really want to prevent the use-after-dispose (with no workaround), we can change the semantic to this: [...]

All joking aside, I have considered adding [Symbol.dispose] to Proxy.revocable. In addition, you can create a proxy that intercepts [Symbol.dispose] to revoke itself:
Proxy.disposable = function(obj) {
    const savedDispose = obj[Symbol.dispose];
    if (typeof savedDispose !== "function") throw new TypeError();
    const { proxy, revoke } = Proxy.revocable(obj, {
        get(target, key, receiver) {
            return key === Symbol.dispose ? dispose : Reflect.get(target, key, receiver);
        },
        // ... (other handlers)
    });
    function dispose() {
        try {
            Reflect.apply(savedDispose, this, []);
        }
        finally {
            revoke();
        }
    }
    return proxy;
};

It's not quite the same behavior that @bakkot is suggesting, however, and has the added overhead of being a Proxy.

@syg: Is it my understanding that V8 objects to this?
@bakkot: Would that objection be sufficient cause to close this issue? Is the example in https://github.com/tc39/proposal-explicit-resource-management/issues/97#issuecomment-1247812768 sufficient to illustrate why preserving the binding is useful?

I talked to @syg some offline after this thread and we didn't come to a conclusion; I am not yet ready to drop this.

Is the example in https://github.com/tc39/proposal-explicit-resource-management/issues/97#issuecomment-1247812768 sufficient to illustrate why preserving the binding is useful?

No.
To be precise, I agree there are nonzero cases where someone might want to do this. I just don't think those are sufficient.

@msaboff, @codehag: Would anyone from your respective organizations be willing to investigate and provide feedback related to this concern? Would introducing a new TDZ (where using declarations become uninitialized when the block scope exits) impact runtime performance (i.e., optimizations, inlining, etc.) in your respective implementations, and would that impact be sufficient cause to object to this change? Conversely, would you support @bakkot's perspective that using bindings becoming uninitialized is a more correct and desirable outcome?
@phoddie, @patrick-soquet: I'm unsure as to the kinds of optimization decisions that Moddable makes regarding const bindings, but your feedback would be greatly appreciated as well.

I expect non-reusable objects to throw anyway when operating on them after their @@dispose is called

I think this is the key point. Whether or not we have a TDZ, and whether or not it is desirable to preserve and reuse the binding in some use cases, it is always trivially possible to preserve the object that is being disposed by storing a reference to it elsewhere. Operating on the disposed, non-reusable object should throw, but this needs to be implemented by the author of the disposable object. Having a TDZ might give them the false security of "oh, I don't need to remember that I'm disposed, since the binding gets uninitialised anyway", which would lead to bugs.

@bergus As far as I can tell that amounts to "we should remove this guardrail so that other people remember to implement a different guardrail", which I don't find to be a convincing argument.

My interpretation is "we shouldn't add an extra guardrail just in case someone forgets to implement their own guardrail". There is plenty of code that already exists that needs to defend itself just as thoroughly when using const or let, and that doesn't go away even when using is available because anyone can still just use const or let.
using doesn't make the kind of guarantee about that object's lifetime that introducing a TDZ seems to make. using doesn't free the object, it informs the object to free its resources. The fact that your workaround to alias a using binding via a second const binding could even work is evidence that a "free the object" guarantee isn't made, otherwise I'd expect even the const bindings would become invalid (i.e., the object reference itself becomes unreachable). Without that guarantee, this extra guardrail seems like overkill.
It's important to note that [Symbol.dispose]() isn't a finalizer, as its perfectly reasonable for a disposable object to be reusable. You can have [Symbol.dispose]() emulate a finalizer, but that is explicitly not a requirement. Introducing a new TDZ doesn't even prevent use-after-free, as evidenced by your workaround. It only potentially reduces it while making it harder to reason over whether a using binding in a callback might still be reachable. Type systems wouldn't even be able to help here because there's no way to know whether a callback might be invoked immediately or later, so all bets are off when it comes to control flow analysis. This is currently the case with let/const bindings used in a callback in TS since we can't guarantee that a callback would be invoked before the bindings are initialized. It seems like this would make it harder to use using in general, rather than make it easier to avoid use-after-free.

My interpretation is "we shouldn't add an extra guardrail just in case someone forgets to implement their own guardrail".

Well, ok. I disagree with that argument too.
When I look at the using syntax, I expect this guardrail to be here. I agree it's not an absolute guarantee (which is why I use the word "guardrail" in the first place; you can still jump over a guardrail if you put effort in). But it's a reasonable thing to expect given the semantics.
I would want there to be some strong reason to deliberately omit any guardrail which users would reasonably expect to exist. I don't think "the library author ought to have implemented a similar check themselves anyway" is a strong enough reason. "This would be a massive amount of work to implement in engines" would be enough, but it was not clear to me from the discussion above if that was the case.

It only potentially reduces it while making it harder to reason over whether a using binding in a callback might still be reachable.

I feel quite strongly that "the binding is only usable before scope exit triggers the dispose method" is easier to reason about than "after the dispose method gets triggered when the binding goes out of scope, a closure which refers to the binding will now see it in a post-disposal state".

It seems like this would make it harder to use using in general, rather than make it easier to avoid use-after-free.

I don't understand how; say more?
 That has not, in my experience, made let bindings are particularly hard to use with TypeScript.
(And this limitation of typescript is not restricted to temporal dead zones; I'm not really sure it's relevant.)

It seems like this would make it harder to use using in general, rather than make it easier to avoid use-after-free.

I don't understand how; say more?

I was referring to the lines preceding that statement:

It only potentially reduces it while making it harder to reason over whether a using binding in a callback might still be reachable. Type systems wouldn't even be able to help here because there's no way to know whether a callback might be invoked immediately or later, so all bets are off when it comes to control flow analysis. This is currently the case with let/const bindings used in a callback in TypeScript since we can't guarantee that a callback would be invoked before the bindings are initialized. It seems like this would make it harder to use using in general, rather than make it easier to avoid use-after-free.

With a new TDZ, rather than only needing to understand how using works, a user would also need completely understand how the closure in question will be used by the function it is supplied to. While this is easy to follow for something like an array, it becomes increasingly more difficult when working with a third-party library. You would potentially have to guard against TDZ for any callback that closes over a using binding for anything that is not a trivial .forEach or .map.
Rather than requiring users to increase the complexity of their code to guard against TDZ, you could instead have a lint rule that disallows referencing a using binding from any closure, and disable it on an as needed basis. That would put the decision firmly in the hands of the developer or development team, rather than forcing it on everyone.

You need to understand that either way, because closures which run after disposal will see the resource in a different state than closures which run before it. At least with TDZ you're guaranteed to get an error instead of just observing the thing in a state you weren't prepared for. This is a better experience in the typical case.
If you think people aren't always going to be paying attention to whether closures are running before or after disposal, I agree, and that's a very strong argument for having TDZ.

@rbuckton – above you asked:

I'm unsure as to the kinds of optimization decisions that Moddable makes regarding const bindings, but your feedback would be greatly appreciated as well.

XS doesn't have any provision for the value of  const to change after being initialized. Sorting out how to make that work and updating the closure doesn't feel trivial.
But, perhaps more importantly, both @patrick-soquet and I agree with overall point of @syg and you, that adding a new TDZ is significantly more complex, both in terms of engine implementation and required developer understanding, than is justified to help a little in one case. There are already so many other ways for a developer to use the disposed object outside the using block already, why address just this one? Host object implementations built on XS already take steps to defend on use-after-close/free/@@dispose and XS provides APIs that help hosts do this reliably.

You need to understand that either way, because closures which run after disposal will see the resource in a different state than closures which run before it. At least with TDZ you're guaranteed to get an error instead of just observing the thing in a state you weren't prepared for. This is a better experience in the typical case.
If you think people aren't always going to be paying attention to whether closures are running before or after disposal, I agree, and that's a very strong argument for having TDZ.

To be honest I think it's way too easy to escape the TDZ so it has too little value over the complexity of the feature and possible pitfalls it may induce.

I stand by my position, but I've talked this over with @syg at some length and I don't think we're likely to come to agreement. Since I think the feature is still worth having even if it retains (what I view as) this sharp edge, and I don't think more discussion is going to get us anywhere, I'll withdraw this suggestion.


Internal adjustments prior to Django merge.

Refs #25.

Regression in 88e60b125510a485fe5f4528e0fd3c6173bd2971.
Without adding get_nodes_by_type() to TemplateProxy, an error is introduced loading extends tags.

AttributeError: 'TemplateProxy' object has no attribute 'get_nodes_by_type()'

Possibly:

… caused by timing change in RenderPartialNode.
… adding get_nodes_by_type() is correct. But tests demonstrating this would be needed.

Reverting these changes, and disabling tox run on CI to investigate at leisure.

v24.3 added official 5.1 compat, with zero needed changes.


[WIP] Update SkiaSharp to v2

What does the pull request do?

What is the current behavior?

What is the updated/expected behavior with this PR?

How was the solution implemented (if it's not obvious)?

Checklist

[ ] Added unit tests (if possible)?
[ ] Added XML documentation to any related classes?
[ ] Consider submitting a PR to https://github.com/AvaloniaUI/Avaloniaui.net with user documentation

Breaking changes

Fixed issues

tested and it works great :D though let's keep this open until the current RC goes into official

I wonder why tests are failing on Linux

tested and it works great :D though let's keep this open until the current RC goes into official

if they release, then prob we should include in 0.10


Updating version page for 3.4 release

Coverage remained the same at 54.057% when pulling d02965d5334f948ba8f03a461b2c650f744e369f on version-page into 6b18e3edf1940e90cef025a43306665829d9e65f on develop.


Add support for manual deciders.

Some flows are more complex than others. For instance: some activities can
be scheduled based on a condition in the context. Being able to (easily)
write the decider has a lot of advantages.
For this: in your flow, you need to define a decider method that takes a
schedule param. To call an activity, all you have to do is to call
schedule, provide an immutable id, the activity and a list (if necessary)
of requirements. Example:
def decider(schedule):
    activity_1 = schedule(
        'activity_1', test_activity_1)
    activity_2 = schedule(
        'activity_2', test_activity_2, requires=[activity_1])

    for i in range(3):
        activity_3 = schedule(
            'activity-3.{}'.format(i), test_activity_3, requires=[activity_2])

        if activity_3.ready and activity_2.result.get('4') == 4:
            last_activity = schedule(
                'last_activity.{}'.format(i), test_activity_last,
                input=dict(the='end'))

Remember: the decider is called anytime you have an event has occurred. So this
method should only be focused on scheduling (and nothing else).
@rantonmattei

LGTM


no Elasticsearch node available

Hello,
I’m facing connection problem with 2.1.1 Elasticsearch. I went through your wiki and here is my output
$ curl -s -XGET 'http://<IP_ADDRESS>:9200/_nodes/http?pretty=1'
{
  "cluster_name" : "elasticsearch",
  "nodes" : {
    "PaUe2I24Q3m4m_p_mneBqg" : {
      "name" : "gen Harada",
      "transport_address" : "<IP_ADDRESS>:9300",
      "host" : "<IP_ADDRESS>",
      "ip" : "<IP_ADDRESS>",
      "version" : "2.1.1",
      "build" : "40e2c53",
      "http_address" : "<IP_ADDRESS>:9200",
      "http" : {
        "bound_address" : [ "<IP_ADDRESS>:9200", "[fe80::1]:9200", "[::1]:9200" ],
        "publish_address" : "<IP_ADDRESS>:9200",
        "max_content_length_in_bytes" : 104857600
      }
    }
  }
}

But when I run from go program
$ go run searchservice.go
panic: no Elasticsearch node available

Any idea where to debug?

Is that really 127001, not <IP_ADDRESS>? Can you show me the Go code where you're connecting to ES?

Am 27.12.2015 um 08:23 schrieb Sundar<EMAIL_ADDRESS>Hello,
I’m facing connectin problem with 211 Elasticsearch I went through your wiki and here is my output
$ curl -s -XGET 'http://127001:9200/_nodes/http?pretty=1'
{
"cluster_name" : "elasticsearch_sundar",
"nodes" : {
"PaUe2I24Q3m4m_p_mneBqg" : {
"name" : "gen Harada",
"transport_address" : "127001:9300",
"host" : "127001",
"ip" : "127001",
"version" : "211",
"build" : "40e2c53",
"http_address" : "127001:9200",
"http" : {
"bound_address" : [ "127001:9200", "[fe80::1]:9200", "[::1]:9200" ],
"publish_address" : "127001:9200",
"max_content_length_in_bytes" : 104857600
}
}
}
}
But when I run from go program
$ go run serachservicego
panic: no Elasticsearch node available
Any idea where to debug?
—
Reply to this email directly or view it on GitHub.

@sundarv85 Interesting. I was reading this on my iPhone over christmas, and it displayed <IP_ADDRESS> as 127001. So: Sorry for that :-)
When using elastic to connect to Elasticsearch 2.x, you should definitely use the following import path:  "gopkg.in/olivere/elastic.v3". Do you use that?

@sundarv85 I updated the wiki page on connection problems to include instructions on chosing the correct version of Elastic. You might want to read it again... maybe it solves your problem?

@olivere Ah yes, that was the issue. I had used incorrect path. My bad. Thanks for updating the wiki. Now it works.


Add support images 360°

Exemple: https://aframe.io/examples/showcase/sky/
Using aframevr.js

This issue will be closed because I'm not actively working on Lychee anymore. The repository of Lychee will stay available, but the main development continues under the Lychee Organisation. A fork maintained by the community. Important issues have been migrated. Feel free to re-add your issue(s) when necessary.


Treat ResourceNameConfig.entityName as Name

Methods that call ResourceNameConfig.getEntityName() pretty much immediately convert it to a Name anyways.
By making it a Name, we can take care of parsing the CamelCasing/snake_casing in one place, during construction, rather than trying to convert a weird String entityName into a Name in many other places.
Also make SingleResourceNameConfig an AutoValue.Builder built object so we can alter via toBuilder() later on.

PTAL


Remove top-border on history component when used in collection

Remove top-border on history component when used in the collection
remove folding arrow as well
remove "History" from the headline of the card

@yangwao will do this


improve installation [proposal]

Hello! I went through the installation process and I though on improve it using a makefile to automate it.
I am happy to create it :)

Hi @geroferrari !
Thanks so much for your interest in our project, a makefile would be absolutely amazing!
For contributing, please refer to our guidelines here for forking and making a PR. Once we review and test it, we will merge it with the main branch.
Please reach out if you need any assistance, we are happy to help : )


Support mysql as DB backend

This provides mysql as DB backend, and also reduces the size of the authorization key because mysql does not support keys so large (and they are not necessary)
Fixes #51
cc @indirect @smellsblue

:+1:


[Feature] Process consideration to keeping consistent data formatting

Is your feature request related to a problem? Please describe.
CSV file header changes can cause some issues with services that rely on the provided CSV formats provided by this awesome repo. This might break some downstream services that are expecting data in a specific format -i.e. data in a particular column index and maybe even having particular column header names.
To help ensure data guarantees to the community it would be great to keep these changes -if possible to a minimum and at least managed with some lean process. :)
Describe the solution you'd like

We could first consider only making column changes to a file append only. So if you want to add something new to the file -maybe consider appending it to the end of the file in that way it does not break any existing indexes others currently rely on.

We could also consider appending some sort of versioning to the end of a file name if we want to introduce a backwards incompatible change i.e reordering columns or renaming columns. Where the new file might get a _v2.csv added to it. That way people get time to upgrade to using the new file until sometime when we deprecate the old one.

Describe alternatives you've considered

We could add to the README that these file formats can break at anytime as it is not yet in a stable format.

We could also consider having a means of agreeing to the format before the format is used. So have the community vote (but this can be a bit too much I think).

Additional context
The reason I ask is that these types of changes should be consider as backwards incompatible as downstream services that rely on these files can break if they expect the values to be in certain columns with certain headings and thus not a great experience when things like this change.
I have experienced a few breakages related to the district data files and was hoping a simple, lean process could be considered when maintaining these data files so as to give the community some data structure guarantees. :)

Just for context, for the district and sub-district data, there was a question of creating a consolidated file that has all of the entries in a standard format by the demarcation board. This might be a good place to pick this up. If we can automate that conversion to have that single file that could be a good thing. We can then phase out the use of any names and use key files to reconcile. Thoughts.

Does anyone have an example of the demarcation file.
I agree with the proposals of @chadwpetersen though we really want to discourage it. Sometimes it is forced on us. For example Ekurhulieni was releasing Ekurhuleni East and North and then split them into East 1 and East 2

See the example of the Limpopo districts file. @JosephSefara has been using the demarcation names.

Also forgot @shaze we have the 2018 Demarcation key in https://github.com/dsfsi/covid19za/blob/master/data/district_data/LM_2018.csv

Great -- this is a coarser level than we are reporting. The main issues have been at sub-municipality issue.  I've had a quick look at the Demarcation Board web site -- can't see a convenient spreadsheet at lower level -- there are maps.

Great -- this is a coarser level than we are reporting. The main issues have been at sub-municipality issue. I've had a quick look at the Demarcation Board web site -- can't see a convenient spreadsheet at lower level -- there are maps.

@shaze You are referring to wards (is a sub-municipality) or subplaces (like suburbs) ?

There are three levels of sub-municipality data that I have seen
-- a suburb
-- a ward --
-- region -- e.g. Ekurhuleni North 1
Regions are a collection of wards for sure. I think wards are generally collection of suburbs but looking at the maps I have of my neighbourhood, I'm not 100% sure that this is 100% followed.

I think the demarcation key will help with standardising the district column names I think that is a good idea. Might be worth-while then to standardise the the cases details part -total cases, recoveries, deaths? Seems like not all files have them defined in the same columns with the same column names.

@heerden can you help with your inputs

My thoughts have settled on all the great suggestions in this thread.
I have a few practical steps we can start with, that will lead to a governable specification for data consistency for existing district data, future modifications and any districts we need to onboard.
The Readme in the data/district_data should define everything we agree on here.
A combine key file should then reflect the titles and the level of districts. I will submit the first draft soon.
While the demarcation keys are a great starting point, I see how they do not always align with the reported media releases for each province. If it is available, the combined key file will then also serve as a conversion for exiting titles to their demarcation equivalent.
We should also list the data collection leads to every province, to keep everyone in the loop. It might be a worthwhile task to list all our stakeholders as well, to contact them directly if there are "breaking changes". Internally, our API and the notebooks that commit calculated data can be seen as stakeholders.
Other automated checks (Github Actions) can be added to validate the combined key file with submitted data. The province lead can then be notified that there is a break-change that has not gone through the governance process.
The goal is still to prevent any stakeholders workflow from breaking. The only issue I see with versioning is that you will need to keep updating two sets of data files for a while until the old version is deprecated. If this is not an issue for the province leads, then we should keep this option open if there is no way to patch the data.
Appending data columns at the end of the data file might remain the best option, as we do not know how the stakeholder is reading the data. They might be using the column index number. We should thus encourage them to rather use keys.
I am not going to suggest drastic changes to any existing data but will need to consider each province, case by case.

Hey @chadwpetersen and @shaze please take a look at the pull request.

@shaze you mentioned you have a new "recovery" column for the GP districts. You can add them to the key file, which is a data column key file.

Yes, I will. First I want to add "Deaths" though which we have in the data but not in the keys


Image slides don't match with thumblenails

I have 5 images along with 5 thumbnails which set in the right position of sides. The problem is that one of thumbnail doesn't appear on the image in order and none of information appears correctly.
I inserted   between first and second thumbnails on html for now. It shows ok on PC but not on touch screen devices.... I used your example5 and changed the look as follows;

Is this because I changed height of thumbnails to the same height of images? Could you help me on this? Thank you, David.

Please post a link to your page, so I can have a better understanding of what's happening. Cheers.

Thanks David. This is a test page > http://misanodesign.com/HMT/metabolomics.html

It seems that you have an empty thumbnail in the code (), which might be causing the problem.
Cheers

Thank you for looking into the problem, David. Yes, I put an empty thumbnail cos' without this code, slides doesn't match with thumbnails on PC (only work for normal loop, not ). My problem is that because of this, updateHash, thumbnailArrows and clicking backwards go wrong... If you have any solution not using the fake thumbnail, please advise. Thank you!

Well, it's necessary not to add more or less thumbnails than needed; you need to have a thumbnail per slide. Not sure, though, about the problem you're referring to. Please provide more details.

Thank you for your response. I have kept the test site open without the extra tag . The followings are setting I have for the site.

   /*I have other sp-slide x 4 here*/

Thumbnail Title1
Thumbnail Description1
Next

   /*I have other sp-thumbnail x 4 here*/

-----------------------------------------------------------------------------------------

If you could help me to figure out the problem I have, that would be great. Thank you!

Sorry for the delay. Unfortunately, it's not clear to me what the problem is when you use only as many thumbnails as slides. I don't understand the matching problem you're referring to.

Hi David
Thank you again for your response.
I have 5 slides and 5 thumbnails in total, however, the only 2nd thumbnails skips and the 2nd slide shows 3rd thumbnail. To make the slides and thumbnails match with each other, I added a fake  between 2nd and 3rd thumbnails.
I updated the test page without  here.
http://misanodesign.com/HMT/metabolomics.html
Hope this explains better:).

I understand now what you're referring to and what causes this behavior. The thumbnail scroller will try to show as the last visible thumbnail the thumbnail that corresponds to the next slide, not the thumbnail that corresponds to the current slide. This makes perfect sense for the purpose for which the thumbnail scroller is intended.
My recommendation for you would be to create two sliders (without thumbnails) and have the synchronize. This is possible with the slider's API.

Hi David
Thank you for your great tip. I'll try to figure out synchronizing 2 sliders and keep you posted how it goes.
Thank you again!

Hi David
I have set up 2 sliders without thumbnails but couldn't figure out how to link them using buttons, arrows and the "NEXT" links (updateHash). If you have any tips please let me know. Thank you!

Hi. You can use the gotoSlide callback to detect when the slide changes and when that happens call gotoSlide( index ) on the other slider. You will also need to check if the selected slide of the two sliders differ and call gotoSlide only when that's the case.
Cheers.

Hi David
No success of adding callback function like below. Please advise any correct way to sync two slides, '#my-slider-one' and '#my-slider-two' ??
'$( '#my-slider-one' ).sliderPro({
gotoSlide: function( event ) {
console.log( event.index );
},
change: function() {
console.log( event );
},
});
$( '#my-slider-one' ).on( 'change.gotoSlide', function( event ) {
$('#my-slider-two' ).trigger('to.gotoSlide', [event.item.index,0,true]);
});
})'
Thank you again for your help!

Hi. I would do it like this:
var firstSliderIndex = 0;
var secondSliderIndex = 0;

$( '#my-slider-one' ).on( 'gotoSlide', function( event ) {
    if ( event.index !== secondSliderIndex) {
        $('#my-slider-two' ).sliderPro( 'gotoSlide', event.index );
    }
});

$( '#my-slider-two' ).on( 'gotoSlide', function( event ) {
    if ( event.index !== firstSliderIndex) {
        $('#my-slider-one' ).sliderPro( 'gotoSlide', event.index );
    }
});

I haven't actually tested the code, but you can get the idea. Cheers.

Hi David
Many thanks for your idea. It starts working very well, although the first screens are still jammed when going backwards by buttons and arrows. If I set up loop: false, it goes completely wrong. Any ideas?? Thank you again.

Hi. It seems that you've missed some parts from my code: firstSliderIndex = event.index and secondSliderIndex = event.index. Also, try to use console.log in your debugging, to see why some things are not working. Cheers.

Hi David
Apologize for my late action. I have added the codes you suggested, however when click through the navigation buttons, it still skips the 2nd slide and texts. This is the same problem I had with the original setting with one slide using thumbnail and couldn't figure out the problem using console.log too. Sorry again for the message after closing issue, but do you have any ideas?

Hi. I did some debugging and discovered the problem, so I made a small modification to the code:
var firstSliderIndex = 0;
var secondSliderIndex = 0;

$( '#metabo_slider').on( 'gotoSlide', function( event ) {
    if ( event.index !== secondSliderIndex) {
        secondSliderIndex = firstSliderIndex = event.index;
        $( '#metabo_thumbnail' ).sliderPro( 'gotoSlide', event.index );
    }
});

$( '#metabo_thumbnail' ).on( 'gotoSlide', function( event ) {
    if ( event.index !== firstSliderIndex) {
        firstSliderIndex = secondSliderIndex = event.index;
        $( '#metabo_slider' ).sliderPro( 'gotoSlide', event.index );
    }
});

Cheers.

Hi David,
It worked perfect! Thank you again for all your hard work and saving me with your spark of genius!