Closes #29 | Add dataset loader for IJELID

Closes #29
Notes:

IJELID is a token-level language identification task. I can't seem to find it in the supported tasks so I added a new one in this PR (https://github.com/SEACrowd/seacrowd-datahub/pull/45/commits/b472a1c53531f71f8091de41226c94e0ef3d002e)

Checkbox

[x] Confirm that this PR is linked to the dataset issue.
[x] Create the dataloader script seacrowd/sea_datasets/my_dataset/my_dataset.py (please use only lowercase and underscore for dataset naming).
[x] Provide values for the _CITATION, _DATASETNAME, _DESCRIPTION, _HOMEPAGE, _LICENSE, _URLs, _SUPPORTED_TASKS, _SOURCE_VERSION, and _SEACROWD_VERSION variables.
[x] Implement _info(), _split_generators() and _generate_examples() in dataloader script.
[x] Make sure that the BUILDER_CONFIGS class attribute is a list with at least one SEACrowdConfig for the source schema and one for a seacrowd schema.
[x] Confirm dataloader script works with datasets.load_dataset function.
[x] Confirm that your dataloader script passes the test suite run with python -m tests.test_seacrowd seacrowd/sea_datasets/<my_dataset>/<my_dataset>.py.
[ ] If my dataset is local, I have provided an output of the unit-tests in the PR (please copy paste). This is OPTIONAL for public datasets, as we can test these without access to the data files.

The dataloader runs (data = load_dataset("seacrowd/sea_datasets/ijelid/ijelid.py")) without specifying config name. Is this intended?

The dataloader runs (data = load_dataset("seacrowd/sea_datasets/ijelid/ijelid.py")) without needing to specify config name. Is this intended? @ljvmiranda921

reviewed and tested as well.

Thanks for the reviews @ryanignatius and @yongzx !
As for the load_dataset question, it automatically loads the DEFAULT_CONFIG_NAME (in this case, ijelid_souce) when nothing's passed to the name parameter.


Rules for ModSecurity to prevent Bruteforce Attacks coming from BurpSuite

Hi , I am testing DVWA using BurpSuite for Brute Force attacks .
My engine is On and I dont see ModSecurity Blocking any login requests .
My Stack
Kali Linux , DVWA , apache2, Modsecurity version 2.9.5

Hello @mohankk1297 ,
The ModSecurity engine (i.e. this github repository) provides the tools for detections.
You need to have rules in place to make use of those tools. You would need to review the rules that you are using to attempt to block such attacks (which you may have sourced from a third party).


Quitting an application with a window on another space breaks the other spaces window tree

Probably can't be fixed, as Kwm tracks individual windows and not applications windows.
In addition to this, OSX does not offer any way to get information about windows that are on a different space than the active one.

Makes sense, so I'm just gonna write a script that closes the application when the last window closes.


Create Posts “day-1”

Automatically generated by Netlify CMS

👷 Deploy Preview for eloquent-chandrasekhar-302983 processing.
🔨 Explore the source changes: b4476346be12d36438b5fc68467907f9759e4de5
🔍 Inspect the deploy log: https://app.netlify.com/sites/eloquent-chandrasekhar-302983/deploys/61accc3fb28adc00074c7f93


Render expressways with a different style than standard highways

Highway maps commonly show expressways–also known as limited-access, controlled-access, or divided highways–with a distinct line style from standard two lane highways.  Beyond simply using a different color, expressway lines are usually thicker and have higher contrast casing.  Some examples:

Other than highway=motorway OSM highway classifications are functional, not physical, so there are both trunk and primary expressways (I don't think secondary expressways exist but I suppose it's possible).  So to indicate the physical characteristics of an expressway we have a separate tag, expressway=yes.  It would be great for this style to be the first to render expressways distinctly based on this tag.  Below are some rendering ideas for how this could look.

Low zoom where individually mapped carriageways merge together as one line

Higher zoom where individually mapped carriageways are visible

All motorways are expressways, so highway=motorway implies expressway=yes and would always be rendered with the expressway line style.  trunkandprimarywould have a standard highway line style and an expressway line style that would be used whenexpressway=yes` is set.
Currently the expressway tag is primarily used in North America, but the concept is applicable elsewhere as well.  In other parts of the world the motorroad tag is far more common.  While expressway defines the physical construction of a highway, motorroad defines legal access restrictions (i.e. a road where only for motor vehicles are allowed).  Although the definitions are different, in practice motorroad=yes is often applied to the same or very similar kinds of limited access, divided highways as expressway=yes.  So for this map style we may want to interpret motorroad=yes as implying expressway=yes when the tag is absent and render these highways with the expressway line style.  In the case of an explicit expressway=no the standard highway line style would be used.
Update 2021-11-04:
My thinking on the above has changed and I'm not sure if it is actually a good idea to assume that motorroad=yes implies expressway=yes when expressway=no is not present.  I more clearly understand now that the reason for the motorroad=yes tag is that in countries where it is used, access restrictions are not explicitly signed on these roads.  Instead there is a blue sign with an icon of a car which means "motor vehicles only".  Since the sign doesn't say exactly what modes of transport are prohibited it's hard for mappers to set all the right access tags (horse=no, bicycle=no, etc).  In North America on the other hand, access restrictions generally are explicitly signed (No pedestrians, No bicycles, etc) so it is easy for mappers to tag them explicitly and there is no need for the motorroad tag.  All this is to say that while motorroad=yes is tagged on many roads that physically qualify as expressways, it also is tagged on some that do not so I'm not sure we should use it for this rendering distinction.

I just want to note that highway=secondary + expressway=yes is a valid combo that should be supported. I just downgraded an expressway in Rochester, NY to secondary based on its State ACC classification and use as an intra-urban feeder:  Way: Mount Read Boulevard (20116226). I downgraded to highway=secondary in Changeset 112773714.
Here's another example that is currently highway=secondary + expressway=yes, the access road to the Syracuse Airport: https://www.openstreetmap.org/way/38606400#map=15/43.1149/-76.1228
In even larger urban areas I could imagine a use for tertiary expressways, such as feeders for airports, theme parks, industrial parks, and other major traffic sources.

I just want to note that highway=secondary + expressway=yes is a valid combo that should be supported

I agree with this assessment.  In openmaptiles/openmaptiles#1148 I propose to add data support for expressway tagging on trunk through tertiary plus unclassified.  tertiary/unclassified + exressway=yes seem like...highly questionable combination, but if it's tagged I see no reason not to support it.  Perhaps we might find cases where a previously planned larger freeway was not completed, leaving behind expressway road sections that have limited connectivity value in the broader road network.

Expressway tagging was added to OpenMapTiles in openmaptiles/openmaptile#1313.  Thus, this issue can be worked on once OpenMapTiles 3.13 is released.

@ZeLonewolf is this issue ready for closure due to merging of #82 or are further enhancements needed?

Further enhancements are needed, but that is true of the whole road network, not just expressways.  Could close to the issue to show that the initial implementation is done, or leave it open for further work.  At the very least it's certainly not blocked anymore.

What enhancements do you see as still needed on expressways?
With #165 and #150 the general-purpose road network is now at least drafted (sample).

We still need to tweak the relative importance of primary and down as described in #84. Those changes could demand tweaks to expressways, but maybe not.

The purpose of this ticket is certainly satisfied.  Thanks @adamfranco for your hard work on getting expressways rendered.  Closed by #82, #150, #165


0 gets removed when encrypting a number string with leading 0

When encrypting and decrypting a string with a leading 0, it removes the 0.
import SimpleCrypto from "simple-crypto-js"

let simpleCrypto = new SimpleCrypto("hYLiR5U1g2ppct1Aw8hf");
let encrypted = simpleCrypto.encryptObject("0419383728");
console.log(encrypted);

let decrypted = simpleCrypto.decrypt(encrypted);
console.log(decrypted + typeof decrypted);

output
419383728number

Please provide SimpleCrypto version that you use. However, this may be a duplicate of #19 and has been fixed since version 2.4.2.

@bitsofalex I will closing this issue soon if there's no update on the issue.


Use latest container and reduce it's size

Changes Unknown when pulling af6487d38362763db3b567f1cb52c9be0a6e482d on latest-containers into ** on master**.


Differences between old sdk and new kotlin sdk

Expected Behavior
I expect the same event details for the old and new android sdk. (Session ID, Platform, Version)
Current Behavior
The new kotlin sdk does not seem to report the same event details.
Event captured with the old android sdk:

Same event captued with the new kotlin sdk:

The Kotlin sdk always reports a Session ID of -1. Device and Version are missing. Is this behavior expected?
Environment

Device OS and Version: Tested on an Pixel 2 Android Emulator running API 31

Hi @hoi4
Thanks for choosing our SDK.
I noticed you are still using the init release of our kotlin SDK. The current amplitude-kotlin already been GA, do you mind updating it to the latest version?
Let me know if you are still facing the same issue after the update.
Thanks.

Thanks for the reply, @yuhao900914
I am using com.amplitude:analytics-android:1.3.0 which is the latest version.

@hoi4  when importing the class, are you importing from the import com.amplitude.android.Amplitude? If not, can you try to update the import and have a try?

@yuhao900914 The correct import solved the issue for me. When changing the import, my app crashed instantly. It took me a while to figure out that the kotlin sdk requires the application context instead of the activity context (as the old sdk).
One last question: We used setSessionTimeoutMillis for the old sdk. There does not seem to be a similar setting in the kotlin sdk. Is there any replacement or is it not necessary anymore?

@hoi4
In kotlin sdk, it should be the minTimeBetweenSessionsMillis in the Configuration.

Thank you, @yuhao900914 !

@yuhao900914  Could you give a rough estimation on adding the getDeviceId method to the Kotlin SDK? 😊

https://amplitude.github.io/Amplitude-Kotlin/core/com.amplitude.core/-state/device-id.html?query=var deviceId: String? = null
Can I use this value?

@yuhao900914 Any news on this? :)

@yuhao900914 Would be great to get an update on this. It's the only thing that keeps my company from migrating to the Kotlin SDK 🙏🏻

Hi @hoi4, This is a feature request. We are mainly working on improving our process for now. So this is not on our road map yet. We won't deprecate the legacy android SDKs for a while, if you feel comfortable, you can still use the legacy android SDKs.

Interested in this answer too... getDeviceId() is important for tying any events logged via the HTTP API where you can/should pass a device ID to ensure matching of events to devices.

Hi, we encountered the same problem. It's causing the wrong number of unique users to display on the Amplitude Console page, and it should have a document or warning to mention the risk.

Old
New

com.amplitude:android-sdk:2.38.3
com.amplitude:analytics-android:1.7.1

Hi @dogtim, thanks for the suggestion, we will add this notification to our doc.
You are right. We are using SQLite in the old Android SDK. And for the new Android SDK, we removed the SQLite dependency and used a file system.

Unless I'm mistaken, it looks like getDeviceId() was added as part of this PR:
https://github.com/amplitude/Amplitude-Kotlin/pull/40#issuecomment-1553427391

Thank you all for supporting Amplitude! getDeviceId should be available in the Kotlin SDK now, check doc.


Vim(echoerr): could not invoke JSLint

Hi,
I am getting this error ( the one in the title ) when I try to open a .js file. In addition I get:
Error detected while processing function 87_JSLint:
Line 33:
could not invoke JSLint!
I am running Ubuntu 10.04. I am using jslint.vim_0.7. I have installed it with rake install. I have installed both node and spidermonkey.

Getting the same problem on MacOSX 10.7,  MacVim-snapshot 61, node 0.4.1
Fixed substituting the jslint-core.js with the one here https://raw.github.com/douglascrockford/JSLint/master/jslint.js

I too managed to make it work on 10.7 by pulling the latest jslint-core.js

Mine stop working with the latest (2012-01-25) version.

There seem to be some different issues here in different environments.  I'm seeing what I can do to address all of them.  To summarize:

The node executable is sometimes called nodejs.  I updated the plugin to check for this.

Updates to Mac OS X Lion and Snow Leopard caused the plugin to break.  Upgrading to a more recent version of JSLint fixes the problem.  I have merged @csexton's fix, and then upgraded to the current latest JSLint version, 2012-02-23.

If you are using node.js, upgrading JSLint will cause the plugin to break because the previous version had some stuff added to it to make it act like a node.js module.  I updated the plugin node.js code import JSLint without requiring any patching, which should fix that problem.

The plugin might fail to run if you are using shell other than bash.  I merged a pull request from @eventualbuddha that should fix that.

The plugin might fail to run if the path to your vim plugin directory includes directory names with spaces or other less common characters.  I pulled in the fix that @mrkschan described above to fix that.

So if you have been having problems with jslint.vim, the latest version may have a fix for your problem.  In case it doesn't, I included the change that @mafolz suggested which will output a more detailed error message if something goes wrong.  If you try the latest jslint.vim version and continue to have problems, I would appreciate it if you could post that more detailed error output here.
The errors that jslint.vim outputs might go by too fast for you to copy them down.  You can use the vim command :messages to bring them back.

@japrogramer, jslint.vim does bundle its own copy of JSLint.  I wrote the plugin before npm existed; and I am trying to support a few different environments where npm may not be available.  Going all node with a future version and tying into npm could be a good idea though.

I added  let $JS_CMD='node' in .vimrc file and  the error disappered.


get rid of inclusion of pf17 inside of unbounded_variant

So that one does not have to touch the ad-hoc in_place_type

Good idea, I will do this.


				PUNCT	.	_	18	punct	_	_
</Parse></Sentence>

The errors are caused by the prepositions that has an 'obl' relation with the verbs. I fixed the errors.

Can you add these sentences to the test cases? Once they're in and the test cases are passing, we can close this issue.


Add githash of code to JSON result files

We should stamp JSON results files with the githash (or version number)  of the code used to produced the results. This will help with reproducability.

Don't think this really helps things - it's still possible for people to run using local branches, or to run with modified files.


Does not support barcode?

I was following Apple's examples and the sign_pass utility successfully outputs the barcode object, but when using your passkit no barcode is on the pass
In my pass.json I have:
"barcode" : {
    "message" : "123456",
    "format" : "PKBarcodeFormatQR",
    "messageEncoding" : "iso-8859-1"
  },

But no barcode is present in the generated pass.

I found you can set the barcode programmatically. Wish it would pull from the json though
pass.barcodes([{
message: "1234567890",
format: 'PKBarcodeFormatQR',
messageEncoding: 'iso-8859-1'
}]);

Thanks for your help. This should be mentioned in the README. Can somebody update it or is it ok I make a PR?

@nickasd We would love to see documentations PR - it's way needed and appreciated!

Barcodes now seem to be accepted in the json, but I will make a PR for the readme to clarify other things.

pass.barcodes = [{
    message: "1234567890",
    format: 'PKBarcodeFormatQR',
    messageEncoding: 'iso-8859-1'
 }];

As the code changed to using setter.


[Task] Improve label of proxy options in connection

The labels for the proxy server and database server are a bit confusing.
I believe we should have a single database URL field that is always shown. Then the proxy toggle will show or hide the proxy server URL field.

Not going to do this because it causes too many downstream effects like documentation, screenshots, etc.


add newtype structs for additional primitives

Add newtype structs for distance, speed, and number of stalls.
Some of the derives in the original pystruct macro were incompatible with float types (Eq and Hash), so I had to come up with another approach. It's now a lot more verbose, but it's also much more flexible. This was also nice because it let me add math operations like Add and Sub on NumStalls.

my dudes, i'm in a meeting but i wanted to mention, if we're talking about making our numeric types strongly-typed, it might be good to make the types reference their unitary type or numeric range. for example instead of NumStalls, maybe NonNegativeNumber or such. this can be really helpful for creating conversion methods for things like speed/distance/time stuff, as well as reduce friction in re-casting stuff

ok, out of meeting, and, to elaborate, say we have these types, we can define these strongly-typed relationships (and no invalid relationships):
opaque type Miles = Double
opaque type Hours = Double
opaque type MilesPerHour = Double

object Miles:
	def apply(n: Double) = n

object Hours:
	def apply(n: Double) = n

extension (m: Miles) def /(h: Hours): MilesPerHour = m / h

println(Miles(60) / Hours(2))  // prints "30.0"

in fact, there's a lib for that. maybe there's a lib for that in rust? :thinking:
and, now that i'm done rushing to make this statement, lemme also say i like the idea to extend type safety to the numbers. that can especially get messy in scientific libs like this and deserves the work, thanks for getting this going!

maybe there's a lib for that in rust?

We've used the uom crate in another project but it did require a lot of macro wizardry to get it play well with pyo3 and so if we go that path it might be nice to confine it to just the rust scope?

used the uom crate in another project but it did require a lot of macro wizardry to get it play well with pyo3

good insight. for us, while it is important, numeric/arithmetic ops make up a small slice of the actual hive codebase. maybe we don't gain enough from that compromise.
would the roll-your-own approach i sketched above in scala be a good fit then? tradeoff: we would then need to implement all of the ops between our custom numeric types, and, future ops would need to touch those type definitions.

would the roll-your-own approach i sketched above in scala be a good fit then? tradeoff: we would then need to implement all of the ops between our custom numeric types, and, future ops would need to touch those type definitions.

yeah this is a tricky one; maybe we could do a short experiment with the uom library to get a feel for the level of complexity that it introduces for this and then make a more informed decision?

Closing this to cleanup my PR page. Feel free to adopt the branch if y'all want to use any of it!


💡 [REQUEST] - speechless-code-mistral-7b-v1.0

Model introduction
This model is fine-tune mistralai/Mistral-7B-v0.1 in order to improve the model's reasoning and planning abilities.
Model URL
https://huggingface.co/uukuguy/speechless-code-mistral-7b-v1.0
Data type
bfloat16
Additional instructions (Optional)
No response
Author
Yes
Security

[X] I confirm that the model is safe to run which does not contain any malicious code or content.

Integrity

[X] I confirm that the model comes from unique and original work and does not contain any plagiarism.

Evaluated and updated on https://evalplus.github.io/leaderboard.html!


[Thread-leakage]DruidTaskResolver does not stop poll thread  after tranquilizer is closed

Every time a new connection is established in IndexService, there is a DruidTaskResolver submitting a runnable to a new thread (poll thread) bind to the connection and periodically check the task running status, but when the beam is closed, the poll runnable cannot be notified and cancled properly. This caused a thread leakage in a long-last running jvm.

The single-threaded executor does not return a future, so the runnable cannot be stopped when the beam is closed.
pollExecutor.submit(
new Runnable
{
override def run(): Unit = {
while (!Thread.currentThread().isInterrupted) {
try {
val tasksSnapshot: Map[Long, (String, AtomicReference[Addr], Updatable[Addr])] = lock.synchronized {
while (timekeeper.now.millis < nextPoll) {
val waitMillis = nextPoll - timekeeper.now.millis
if (waitMillis > 0) {
lock.wait(waitMillis)
}
}
tasks
}
            val now = timekeeper.now
            val baseWait = now.plus(pollPeriod).millis - now.millis
            val fuzzyWait = (math.max(1 + 0.25 * Random.nextGaussian, 0) * baseWait).toLong
            nextPoll = now.millis + fuzzyWait

            val running = Await.result(indexService.runningTasks())

            log.debug(s"Polled[$indexServiceKey] and found ${running.size} tasks " +
              s"(${running.count(_._2.isBound)} bound).")

            for ((taskNumber, (taskId, addr, updatable)) <- tasksSnapshot) {
              val runningHostPort = running.get(taskId)
              val runningAddr: Addr = runningHostPort.map(_.toAddr).getOrElse(Addr.Neg)
              if (addr.get() != runningAddr) {
                log.info(s"Updating location for task[$taskId] (#$taskNumber) to[$runningAddr].")
                addr.set(runningAddr)
                updatable.update(runningAddr)
              }
            }
          }
          catch {
            case NonFatal(e) =>
              log.warn(e, s"Poll failed, trying again at[${new DateTime(nextPoll)}].")
          }
        }
      }
    }
  )


bugfix: we now only apply the lightuserdata mask on platforms that ar…

…e at least 64bits.

Relative issue: openresty/lua-nginx-module/issues/1423


Clone in plugins

I tried cloning this in .oh-my-zsh/plugins and adding phpcs in plugins var in .zshrc but it didn't work, $fpath variable is having phpcs path correctly.
Also, source path isn't working for me somehow.

@PatelUtkarsh, could you please provide your .zshrc here?

https://gist.github.com/PatelUtkarsh/77a39ad5c4abc731667c19fed2863e2a @voronkovich

@PatelUtkarsh, I can't reproduce this bug on my machine with your .zshrc. Autocompletion works as expected. Try this: https://github.com/robbyrussell/oh-my-zsh/wiki/Troubleshooting#completion-problems

@voronkovich Thank you for your help.

First, try to remove your completion cache with rm ~/.zcompdump*, close and reopen your shells.

This worked!

Currently, I am using it as a plugin.


SPIKE - multi tenancy approaches

Do a sample with separate schemas hosting different tables

Will be creating follow up stories related to the spike


Remove styled-components in favor of "native" solutions

styled-components was good for having fun (one of the project's purpose), but it goes against another project's purpose that is being pluggable. When using this lib with another lib on a project that that is using styled-components too, it could conflict, since the styled-components ask for being a peer dependency and the other lib could be asking for a different version of the styled-component.
So, I'm proposing to change the board to using "native" solutions like css or css-in-js (without the styled-components).

Is this issue still opened? I noticed that it was supposed to be closed at the PR #278 merge but the merge has happened and the issue is still opened.
Does this issue needs any help?

Hi, @tiagostutz.
The reason for this issue being opened is because the PR was only merged into the beta branch. This issue is going to be closed as soon as the beta gets merged into the master branch.
I'm planning to merge this as soon as I get enough feedback about the CSS API exposed there.
You can try it through the beta release (yarn add @lourenci/react-kanban@beta). Here the docs for the CSS API.


consume /generateSendOpReturn/ endpoint [WIP]

PR related to this trello task --> https://trello.com/c/tngCZznV
This is a work in progress

I'm closing this PR as we won't be using it. Over the weekend, I was able to take your code from this PR, adapt it, and merge it into minimal-slp-wallet. Rather than passing the OP_RETURN as a Buffer, it now gets passed as a hexidecimal string. It is then converted to  a Buffer by gatsby-ipfs-web-wallet.


Create An Automated Webdriver IO Test Creator Based on Test Examples

Feature Request
Description

Storybook has a concept called storyshots which allow you to write code like the following in order to Jest test all storybook examples:
import initStoryshots from '@storybook/addon-storyshots';

initStoryshots();

We are already encouraging developers to create hidden "test" routes that will be used for wdio tests.  We should in theory be able to use the dev site configuration to discover routes that need testing and execute them automatically.  At a base level, this could include simple non-interactive tests.  It could in theory allow for more complex tests with a little consumer-defined configuration.
Advantages
Advantages of this include:

Less testing code written by consumers
Greater control over screenshots and view port size logic
Testing code living alongside the example

Design
Modification to Site Config
Create a new option (wdioPageTypes) to flag certain pageTypes as ones that should have wdio tests generated for them automatically
Metadata for Test Examples
In order to support more complex tests, each test example can define its own metadata that supports:

viewports - The viewports associated to this example
selector - The selector to use when taking screenshots
themeableProperties - Any themeable properties that should also be tested
accessibilityRules - Any accessibility rules that should be considered when testing
steps - An array of objects containing the following

action - A function that should be run as part of this step
name - An optional name to use to distinguish this test
selector - Takes precedence over the test level one
accessibilityRules - Takes precedence over the test level one
themeableProperties - Takes precedence over the test level one

wdioTestDevSiteSnapshots
Create a new function that will be used to generate tests automatically.  It will take the following options:

package - For use in monorepos so that tests can be created for a single package within the monorepo

This function will parse the dev site config and for each page type in wdioPageTypes, determine the following:

name - The name of the page
groupingDirectory - The directory in which to place the screenshots, this will map to the directory structure of dev site.  It will be passed to a new parameter in Terra.should.matchScreenshot added in this issue.
selector - The selector to use when taking screenshots
url - The url to hit for this page
themeableProperties - Any themeable properties that should also be tested
accessibilityRules - Any accessibility rules that should be considered when testing
steps - An array of steps to include as part of testing.  Each step may have its own set of selectors, themeableProperties, and accessibilityRules that will take precedence over the test level counterpart

Then, a test will be created that navigates to the url, executes the appropriate steps, screenshots, tests accessibility.  It will then take corresponding screenshots with the appropriate themeable properties when necessary.
The above will be aggregated for each viewport corresponding to each example and will run all examples for a given viewport before proceeding to the next one.
Additional Context / Screenshots

I've prototyped this out.  Here are some concrete examples

New Configuration
Metadata

Example 1
Example 2

Usage of wdioTestDevSiteSnapshots

No Metadata Needed
Metadata Needed

I think this auto-generated setup is pretty neat! I have some reservations on the details - in a way I think we might be trying to do too much and I think this approach may be confusing to explain to consumers. t think it would be easier do less, at least initially and add more functionality as we see more use cases.
Here are some thoughts that I have:
Remove the metadata configuration files. A few reasons:

This moves test information way from tests/wdio/* pattern we've been pushing and it makes it 'hidden' to new developers.
This will add additional bulk to the released components that is not clearly distinguished as test specific.
We can easily pass this configuration at the test (*-spec) level.

Test-Specific Metadata Options:
Of the meta data options, I feel like these should be test (*-spec) specific:

viewports: +1 with fallback to wdio formFactor value
selector: +1 with fallback to wdio global selector

Example-Specific Metadata Options
Then, instead of metadata we could allow a json of example specific configurations that uses the example name as the key value:

selector: +1 with fallback to test specific selector
viewports: +1 with fallback to test specific viewports
accessibilityRules: +1 but as axeOptions to match Axe Service terminology
themeableProperties: +1 be passed
actions: I say lets drop this: IMO passing actions is too tricky and 'hidden' from the consumer. Do we execute these between both wdio and a11y tests? Do we take before and after action screenshots? That might be too much? I feel these should be left for consumers to write for themselves so we don't assume to much and they understand the test flow.

Dropping Metadata and Defining as Test Setup
Instead of metadata, we would have a testSetup which we pass to the wdioTestDevSiteSnapshots helper:
const testSetup: {
    viewports: ['medium', 'huge'],
    selector: '#root',
    examples: {
        'defaultComponent': {
            selector: '#no-overflow-column-data-grid'
            axeOptions: { ... }
            themeableProperties: { ... },
        }
    }
};

Example Usage
Then using data-grid metadata examples you provided, the component's test spec would look like:
// eslint-disable-next-line import/no-extraneous-dependencies
const wdioTestDevSiteSnapshots = require('terra-dev-site/lib/dev-site-snapshots/wdio/wdioTestDevSiteSnapshots').default;

const testSetup: {
    viewports: ['medium', 'huge'],
    examples: {
        'CustomHeightDataGrid': {
            selector: '#custom-height-data-grid',
            themeableProperties: { ... }
        }
        'NoOverflowColumnDataGrid': {
            selector: '#no-overflow-column-data-grid'
            themeableProperties: { ... },
        }
    }
};

// auto-generated test specs
wdioTestDevSiteSnapshots({
  package: 'terra-clinical-application',
  testSetup,
});

// custom / manual test specs
describe('NoOverflowColumnDataGrid', () => {
    describe('No Overflow Column Data Grid', ()=> {
        before(() => browser.url('...'));

        describe('with veritical overflow', () => {
            beforeEach(() => browser.click('[data-cell-label="section_0-29-Column-0"]'));

            Terra.should.matchScreenshot({ selector: '#no-overflow-column-data-grid' });
            Terra.should.beAccessible();
        });
    });
});

I do want to start simpler, I was just trying to play around and see if I could get all testing into this manner.  I agree with the config you suggested, but am curious what other opinions are as well.
I originally had the config passed into the wdioTestDevSiteSnapshots function, but I waffled and moved it to be located in the same place as the files they were associated with.  I see that as a convenience, but I agree with your points.  Curious about other feedback for this as well.

I also think we could extend this to a jest test version as well.  For open source I think we'll have a decent amount of benefits, but our pattern of Application -> Container -> View gets wonky because the dev site examples are typically at the Application level and we'd want to jest test at each View level.

Curious what people might think also rather than taking out actions completely, adding a "setup" to allow consumers to pass in things like "click" to set things up.


Create PreciseEditor.netkan

Adding a new mod called Precise Editor.

Here's the current error message:
#   Failed test 'PreciseEditor has a $kref/$vref and a download field, this is likely incorrect.'
#   at t/metadata.t line 63.

Usually you want a $kref so CKAN can determine the download property automatically. However, that will also raise an error because of the way the releases are currently set up; you want the download to be listed under Assets rather than linked in the description, see this example:
https://github.com/Kopernicus/Kopernicus/releases/tag/release-1.4.5-2
So I'd recommend:

Edit the release to list the download under Assets
Remove the download property from the .netkan

Looks good now! Just a few more sanity checks to go...
The license field says it's GPL 3, but the LICENSE.md file in the download looks like the MIT license. Should that be updated as well?

I'm not too familiar with licences but I think the one I'm looking for is GPL 3.0. I will update my mod and get back to you.

Okay I made a new release of the mod with the updated license to GPL 3.0. It's ready now. :-)

Cool, looks good, and thanks for the submission! Merging...


[RFC]: add C implementation for @stdlib/math/base/special/gamma-lanczos-sum

Description
This RFC proposes adding C implementation for @stdlib/math/base/special/gamma-lanczos-sum.
Related Issues
Related issues #649.
Questions
No.
Other
No.
Checklist

[X] I have read and understood the Code of Conduct.
[X] Searched for existing issues and pull requests.
[X] The issue name begins with RFC:.

I would like to work on this.

@gunjjoshi Assigned; thank you for working on this!


[IOTDB-3229] Field is 0 rather than null when query two series in one device

Coverage increased (+0.01%) to 44.837% when pulling c6c10262847e15c81ee06b489fbff40bf7ea85c9 on iotdb-3309 into e1dfec58e36a3f81cb06dbdf09e8cba1ef9b633c on master.


OPENTOK-50613, OPENTOK-49633: Simple Multy-party Video Chat with Metal sample app

Simple Multy-party Video Chat with Metal sample app

Few bugs regarding subscriber windows:

On running the app and subscribing, the subscriber window close (red button, top left) is disabled.
When you disconnect the main pub window, the subscriber window/s still exists and are frozen.
When I kill the app, the subscriber window still remains and only a Force quit can get rid of them

I get this warning when building, maybe can be updated

when running bunch of linker (duplicate) warnings
objc[98812]: Class RTCDefaultVideoDecoderFactory is implemented in both /Users/jshah2/Library/Developer/Xcode/DerivedData/Simple-Multiparty-glygygtplgduilenqceajvmfuhat/Build/Products/Debug/Simple-Multiparty.app/Contents/Frameworks/OpenTok.framework/Versions/A/OpenTok (0x112710980) and /Users/jshah2/Library/Developer/Xcode/DerivedData/Simple-Multiparty-glygygtplgduilenqceajvmfuhat/Build/Products/Debug/Simple-Multiparty.app/Contents/MacOS/Simple-Multiparty (0x10f35f6f0). One of the two will be used. Which one is undefined.

Few bugs regarding subscriber windows:

On running the app and subscribing, the subscriber window close (red button, top left) is disabled.
When you disconnect the main pub window, the subscriber window/s still exists and are frozen.
When I kill the app, the subscriber window still remains and only a quit can get rid of them

Thanks, @IGitGotIt.
Is this something that needs fixing now?  Or, can it be fixed in a followup PR?

I get this warning when building, maybe can be updated

Mmmm this is weird, the deployment target is set to 12.6:
https://github.com/opentok/opentok-macos-sdk-samples/pull/4/files#diff-fe9aeb48faf1e1085f8f4c6d6b6c87a39358c36dc61d295b9fddbf4255333ab1R307
What target is giving that warning?

when running bunch of linker (duplicate) warnings
objc[98812]: Class RTCDefaultVideoDecoderFactory is implemented in both /Users/jshah2/Library/Developer/Xcode/DerivedData/Simple-Multiparty-glygygtplgduilenqceajvmfuhat/Build/Products/Debug/Simple-Multiparty.app/Contents/Frameworks/OpenTok.framework/Versions/A/OpenTok (0x112710980) and /Users/jshah2/Library/Developer/Xcode/DerivedData/Simple-Multiparty-glygygtplgduilenqceajvmfuhat/Build/Products/Debug/Simple-Multiparty.app/Contents/MacOS/Simple-Multiparty (0x10f35f6f0). One of the two will be used. Which one is undefined.

I just run pod deintegrate and pushed those changes, maybe that was causing duplicate warnings

I prefer a fix , can be not that hard to do.
But if in a hurry a follow up PR also is ok (will be an irritant)

I get this warning when building, maybe can be updated

Mmmm this is weird, the deployment target is set to 12.6: https://github.com/opentok/opentok-macos-sdk-samples/pull/4/files#diff-fe9aeb48faf1e1085f8f4c6d6b6c87a39358c36dc61d295b9fddbf4255333ab1R307
What target is giving that warning?

Few bugs regarding subscriber windows:

On running the app and subscribing, the subscriber window close (red button, top left) is disabled.
When you disconnect the main pub window, the subscriber window/s still exists and are frozen.
When I kill the app, the subscriber window still remains and only a quit can get rid of them

The sub fixes are in https://github.com/opentok/opentok-macos-sdk-samples/commit/27bdb66abe8cd49e109c934df8cf56bad7dc9ec2 and the branch is https://github.com/opentok/opentok-macos-sdk-samples/tree/simple-multipart-subFixes


Add weighted histogram example using a table

Related to #976

I adapted @rochoa 's original gist into a 'visual table' example

@makella we can also merge this into your advanced-examples branch in case you wanna test this at DevCenter

ok! sounds good... do you want to merge it in there?

p.s. -- nice one!

OK, I'm gonna merge it there!


On installing package missing panel / not displayed. "AnamicoAlarmPanel"

Hi.
Upon installing the package, everything looks normal except the "AnamicoAlarmPanel" is missing and not displayed in the left hand node list. However reviewing the palette, AlarmPanel is displayed. I have installed the dependancies, async and lodash and restarted node-red but nothing changed.
Any ideas?
Thanks
Brent

AnamicoAlarmPanel is not supposed to show up as a node inte the left side node panel. You will find the settings for AnamicoAlarmPanel when editing any of the available nodes, look for the label "Panel". This is used to create one or more alarm panels for the other nodes to connect trough. I think the little grey square indicates the node type.


Remove deprecated callback hooks

What does this PR do?
Removes deprecated API.
Before submitting

[x] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[x] Did you make sure to update the documentation with your changes? (if necessary)
[x] Did you write any new necessary tests? (not for typos and docs)
[x] Did you verify new and existing tests pass locally with your changes?
[x] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review
Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

[x] Is this pull request ready for review? (if not, please submit in draft mode)
[x] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?
I made sure I had fun coding 🙃
cc @awaelchli @ananthsub @rohitgr7 @borda @justusschock

@carmocca, sorry, don't know what happened.

I think this is blocked by #14947

@lantiga I created this issue that summarizes the new plan we discussed offline #15049. PR #14835 would be the first to implement this pattern and then we follow with this one here. Further PRs will then bring back partially what was removed here: #14841


Add to docs context about CSI driver and K3s

In the CSI doc for Harvester we should let the user know that the CSI driver has not been certified yet for k3s.
https://docs.harvesterhci.io/v1.0/rancher/csi-driver/

Duplicate issue https://github.com/harvester/harvester/issues/1812


Don't let people create dupe conditional questions

From @ryooan: It would be nice to get a warning if a conditional you're submitting already exists. They're clearly defined by parent and child question id and there should be no reason for duplicates, so I think this should be easy to do?

Possible wrinkle I just realized, it should only do this for public questions. For example, if a conditional exists already in a private project, it shouldn't warn a user who tries to make one in a public project.


Logging - where?

I'm running with the systemd unit file as distributed.
In /etc/endlessh/config I have LogLevel 1.
Systemd file has (default) StandardOutput set to journal... yet journalctl records nothing upon start stop or connection. I tried setting to StandardOutput=file:/var/log/endlessh.log and StandardError the same, but nothing gets written.
If I start endlessh from a command line instead of the systemd unit with "endlessh -v >>/var/log/endlessh.log 2>>/var/log/endlessh.log" it does write to the log... but that doesn't help with running as a systemd unit.
I assume I'm missing something obvious, but how should I have this configured to monitor attempted connections/duration when running from a systemd file?

Try to use -s option.
ExecStart=/usr/local/bin/endlessh -s
...
SyslogIdentifier=endlessh

Pretty sure that endlessh outputs directly to syslog. A while back I had journalctl redirect all endlessh logs to its own file. Can't really help you since I switched to OpenBSD but you could try /var/log/syslog


Broken Link

Direct Link is pointing to Microsoft Internal Sharepoint
"Windows: Download and install USB driver from STMicroelectronics website or direct link"

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0cf9b53c-0438-f23c-fd2e-d57481d57e85
Version Independent ID: f01afd66-d093-ac44-2b20-df225530b39e
Content: IoT DevKit to cloud -- Connect IoT DevKit AZ3166 to Azure IoT Hub
Content Source: articles/iot-hub/iot-hub-arduino-iot-devkit-az3166-get-started.md
Service: iot-hub
GitHub Login: @wesmc7777
Microsoft Alias: wesmc

Looks like there is an existing similar issue already reported

@vmaruthi Thanks for taking the time for reporting this. I will close this as a duplicate of https://github.com/MicrosoftDocs/azure-docs/issues/31133 and I will investigate.
With that said if there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Feature request: reading little-endian pp files in Iris

Iris 1.6.1 can read UM fieldsfiles, but when these are converted to pp format using ff2pp on a little endian machine (ARCHER) Iris has the following error:
ValueError: PP file ['/scratch/nla27/xjgpya.pyt9c10.pp'] contains little-endian data, please convert to big-endian with command line utility "bigend".
Is it possible to allow Iris to read little-endian files?

closed (finally) by #1965


feat: created hook to load spec file

Description

As of now we are storing context and we access any context we are using the --context flag to load from the contexts.
This PR will complete the context store use case and intelligently figure out which specification file to load.
aims to autodetect specification file path asyncapi validate

Related issue(s)

Fixes #20

Not ready yet stuck with some testing issue.

Working as intended

I am on windows, did I miss something.

I deleted my dist folder then ran the tests again, all tests are passing.

I run tests in the mac, and sometimes they fail, so probably we have some bugs in tests.

I guess now the macos test is stuck

Hey, @magicmatatjahu is this PR good to go.

@Souvikns
The --flag doesn't work. If I run the cli as:
node ./dist/cli.js validate --file ./spec.yaml

then CLI validates the spec from current context. You have missed changes in https://github.com/asyncapi/cli/blob/master/src/CliModels.ts#L35 file.`
Please update the indentations in the help command, because at the moment it looks like:

and update the Readme.md with new help output :)
Btw, overall good, except that file flag, which doesn't work :)

@Souvikns Please also update the package-lock :)

Awesome work!


[ASv2 MVP] Meet with release engineering team re: infra design

Check-in with the release/devOps team on:

overview of current roadmap -- anything that is missing here?
best practices for service design
anything we should know about infra + service setup or things we should consider?
how should we continue working together during the infra design + implementation phase?

(is there any way we can help you develop the release team's best practices + advising process?)

Initial thoughts:

need for a clear transition plan from v1 to v2

excellent opportunity to start creating a production readiness checklist, which would cover at least the following (some references: https://sre.google/sre-book/service-best-practices/, https://www.opslevel.com/blog/production-readiness-in-depth/,

SLI / SLO / SLA
testing: unit, integration, e2e, performance/load/stress, chaos
desired CI/CD pipeline (build strategy, testing strategy, stages, deployment)
monitoring: metrics + logs + traces, dashboards, alerts
incident management: runbooks, risk assessment, mitigation strategies (eg: graceful degradation), escalation path
security: secrets, static/dynamic analysis, dependencies
capacity planning: resource needs, growth expectation, provisioning strategy (compute, storage, networking, misc)
documentation

would be good to have more eyes on the infra review depending on what components make it into the design

Meeting notes: https://www.notion.so/clabsco/ASv2-ID-DevOps-Release-dcb76801f82e400fb8a42b8157493c3f


[5.8] Serialziation fix for date fields which have "empty" non-numerical values

Calling toArray on a model serializes its dates to a string. Currently, if the date value being serialized is null it gets skipped. However, if the date value is an empty string, attempting to create a Carbon object throws an InvalidArgumentExceptionexception which cause the whole toArray call to fail.
In this PR, I simply check if the date value is not "empty" before attempting to serialize it. If so, it gets skipped, just like null. An exception is added to numeric values (i.e. 0) as it is can be a timestamp.

Date Value
Before
After

null
null
null

0
1970-01-01 00:00:00
1970-01-01 00:00:00

false
InvalidArgumentException
false

""
InvalidArgumentException
""

"abc"
InvalidArgumentException
InvalidArgumentException

Closes https://github.com/laravel/framework/issues/28047

No plans to change this on a point release.


Secure edit/view switch

Disable this switch just after user click, to prevent user click again when switch takes time

It's done!

Perfect Ghiles. It's validated.
Can you please merge with main and integration ? PR for main


Sometimes call remains in 'Connecting' state

After accepting call by pressing 'answer call' button on IOS sometimes the call remains in 'Connecting' state (in callkeep UI) and I cannot hear sound on either phone. Can't hear anything.

This is callkeep specific issue, please report to callkeep issue.
I'm closing this.

This is callkeep specific issue, please report to callkeep issue.
I'm closing this.


resize header height

My header loads with a specific height, then I fetch the user data on server and load all informations about the user.
I need to change the header size after all informations are ready.
I tried to use self.segmentedPager.parallaxHeader.height = CGRectGetHeight(headerBoxView.bounds); but didn't work
the CGRectGetHeight(headerBoxView.bounds returns the right height, but the header doesn't change
[headerBoxView setNeedsLayout];
self.segmentedPager.parallaxHeader.height = CGRectGetHeight(headerBoxView.bounds);
[self.segmentedPager reloadData];
[self.segmentedPager layoutIfNeeded];
[self.view layoutIfNeeded];

any updates about this issue?

@CavalcanteLeo Did you find a solution to this issue? I'm having the same problem.

Did anybody find a solution of this issue?

I'm having the same problem too

Same problem. Any solution?

Anyone knows how to fix it ? :')


Enable chaintest travis

The sqlite errors happening when the dynamodblocal runs on the travis CI slave appeared to be related to storing sqlite records used by dynamodbstreams.  Since the chain test doesn't test streams, I e-enabled the chain test, with dynamodb streams disabled and the test passes.
It's a bit of a head scratcher why only this particular test was failing when the other tests pass but at least we have a green light and we can use travis instead of SFCI.
Pass travis build here ... https://travis-ci.org/salesforce/mt-dynamo/builds/409959482

Pull Request Test Coverage Report for Build 29

7 of 7 (100.0%)  changed or added relevant lines in 1 file are covered.
2 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.2%) to 84.602%

Files with Coverage Reduction
New Missed Lines
%

src/main/java/com/salesforce/dynamodbv2/mt/mappers/sharedtable/impl/MtAmazonDynamoDbBySharedTable.java
2
82.5%

Totals

Change from base Build 23:
0.2%

Covered Lines:
1412

Relevant Lines:
1669

💛  - Coveralls


Dispatch index to subactions

Dispatches certain index actions to subactions using a dispatch table instead of switches.
Improves ABC metric of those actions, fixing some Rubocop ABC offenses.
See @mo-nathan's email and related discussion.
(This is v2 of now-closed #1304)

There is no manual test.

Coverage: 93.342% (+0.001%) from 93.341% when pulling 026074bd5b785edc839ca2bbeb351e82a9cd693b on jdc-v2-index-dispatch into 8f6c4b3627ead0ec7850afaff9bcafb58ddafd47 on master.

Yes, thank you.
On Wed, Jan 25, 2023 at 1:27 PM Joseph D. Cohen @.***>
wrote:

@.**** commented on this pull request.
In app/controllers/species_lists_controller.rb
https://github.com/MushroomObserver/mushroom-observer/pull/1307#discussion_r1087021209
:

@index_subaction_param_keys = [

:pattern,

@nimmolo https://github.com/nimmolo & @pellaea
https://github.com/pellaea: I deleted advanced_search because I don't
think there's a Species List advanced search.
Is that okay?
—
Reply to this email directly, view it on GitHub
https://github.com/MushroomObserver/mushroom-observer/pull/1307#pullrequestreview-1269896260,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAYTNNOFINMYOYIDGSKVDRTWUFWAVANCNFSM6AAAAAAUFTGW5A
.
You are receiving this because you were mentioned.Message ID:
@.***
com>

I think this is ready for review and merger.
The changes to the controllers are routine except:

for Observations, the dispatch table is in different file from the index action and subactions
one of the SpeciesLists subactions merely switches to other subactions.

I'd like to deal with name changes and a possible intermediate ApplicationIndexController in separate PRs.
@nimmolo: Would you be okay if I (in another PR) I changed subaction names like this:

If possible make the subaction name the same as the param key.
Ex: pattern instead of <model>_search.
This lets shorten the dispatch table / dictionary.
Else if possible, use the same name in different controllers.
Ex: something like results_of_query instead of index_<model>
Else if possible, something that makes redundant any accompanying comment.
This allows the comments to be deleted.

I approve.

Thanks @pellaea!
I wait to see if @mo-nathan weighs in again.
On Wed, Jan 25, 2023 at 11:48 AM Jason Hollinger @.***>
wrote:

I approve.
—
Reply to this email directly, view it on GitHub
https://github.com/MushroomObserver/mushroom-observer/pull/1307#issuecomment-1404144847,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAALDFHBLLNIGANKREES7X3WUF7QRANCNFSM6AAAAAAUFTGW5A
.
You are receiving this because you authored the thread.Message ID:
@.***>

I like the sub-method renaming proposal above.


When wasme can support istio 1.6.1?

now i use wasme 0.0.2, it only can select istio 1.5.x, but not support 1.6.1.

i believe @Sodman was looking into to update wasme to support newer version of gloo/istio. envoy-wasm does change frequently, so i'm not sure if it will be compatbile with 1.6.1 or 1.6.2

tks


Fix target-gen serialize

I broke this in #2428, sorry about that. The buffer needs to be cleared even if the line is skipped.

Good point, lemme add one


MultiIndex and data selection

[Edited for more clarity]
First of all, I find the MultiIndex very useful and I'm looking forward to see the TODOs in #719  implemented in the next releases, especially the three first ones in the list!
Apart from these issues, I think that some other aspects may be improved, notably regarding data selection. Or maybe I've not correctly understood how to deal with multi-index and data selection...
To illustrate this, I use some fake spectral data with two discontinuous bands of different length / resolution:
In [1]: import pandas as pd

In [2]: import xarray as xr

In [3]: band = np.array(['foo', 'foo', 'bar', 'bar', 'bar'])

In [4]: wavenumber = np.array([4050.2, 4050.3, 4100.1, 4100.3, 4100.5])

In [5]: spectrum = np.array([1.7e-4, 1.4e-4, 1.2e-4, 1.0e-4, 8.5e-5])

In [6]: s = pd.Series(spectrum, index=[band, wavenumber])

In [7]: s.index.names = ('band', 'wavenumber')

In [8]: da = xr.DataArray(s, dims='band_wavenumber')

In [9]: da
Out[9]:
<xarray.DataArray (band_wavenumber: 5)>
array([  1.70000000e-04,   1.40000000e-04,   1.20000000e-04,
         1.00000000e-04,   8.50000000e-05])
Coordinates:
  * band_wavenumber  (band_wavenumber) object ('foo', 4050.2) ...

I extract the band 'bar' using sel:
In [10]: da_bar = da.sel(band_wavenumber='bar')

In [11]: da_bar
Out[11]:
<xarray.DataArray (band_wavenumber: 3)>
array([  1.20000000e-04,   1.00000000e-04,   8.50000000e-05])
Coordinates:
  * band_wavenumber  (band_wavenumber) object ('bar', 4100.1) ...

It selects the data the way I want, although using the dimension name is confusing in this case. It would be nice if we can also use the MultiIndex names as arguments of the sel method, even though I don't know if it is easy to implement.
Futhermore, da_bar still has the 'band_wavenumber' dimension and the 'band' index-level, but it is not very useful anymore. Ideally, I'd rather like to obtain a DataArray object with a 'wavenumber' dimension / coordinate and the 'bar' band name dropped from the multi-index, i.e., something would require automatic index-level removal and/or automatic unstack when selecting data.
Extracting the band 'bar' from the pandas Series object gives something closer to what I need (see below), but using pandas is not an option as my spectral data involves other dimensions (e.g., time, scans, iterations...) not shown here for simplicity.
In [12]: s_bar = s.loc['bar']

In [13]: s_bar
Out[13]:
wavenumber
4100.1    0.000120
4100.3    0.000100
4100.5    0.000085
dtype: float64

The problem is also that the unstacked DataArray object resulting from the selection has the same dimensions and size than the original, unstacked DataArray object. The only difference is that unselected values are replaced by nan.
In [13]: da.unstack('band_wavenumber')
Out[13]:
<xarray.DataArray (band: 2, wavenumber: 5)>
array([[             nan,              nan,   1.20000000e-04,
          1.00000000e-04,   8.50000000e-05],
       [  1.70000000e-04,   1.40000000e-04,              nan,
                     nan,              nan]])
Coordinates:
  * band        (band) object 'bar' 'foo'
  * wavenumber  (wavenumber) float64 4.05e+03 4.05e+03 4.1e+03 4.1e+03 4.1e+03

In [14]: da_bar.unstack('band_wavenumber')
Out[14]:
<xarray.DataArray (band: 2, wavenumber: 5)>
array([[             nan,              nan,   1.20000000e-04,
          1.00000000e-04,   8.50000000e-05],
       [             nan,              nan,              nan,
                     nan,              nan]])
Coordinates:
  * band        (band) object 'bar' 'foo'
  * wavenumber  (wavenumber) float64 4.05e+03 4.05e+03 4.1e+03 4.1e+03 4.1e+03

Mmm now I'm wondering if the problem I explained above isn't just related to the 3rd TODO item in #719 (make levels accessible as coordinate variables).
Sorry for the post if it is the case.

This is a really good point that honestly I had not thought carefully about before. I agree that it would be very nice to have this behavior, though. This will require a bit of internal refactoring to pass on the level information to the MultiIndex during indexing.
To remove unused levels after unstacking, you need to add an explicit dropna, e.g., da.unstack('band_wavenumber').dropna('band_wavenumber'). This is definitely a break from pandas, but for a good reason (IMO). See here for discussion on this point.
I raised another issue for the bug related to copying MultiIndex that you had in the earlier version of this PR (#769).
More broadly, if you care about MultiIndex support, it would be great to get some help pushing it. I'm happy to answer questions, but I'm at a new job and don't have a lot of time to work on new development.

Thanks for the tip. So I finally obtain the desired result when selecting the band 'bar' by doing this:
In [21]: (da.sel(band_wavenumber='bar')
    ...:  .unstack('band_wavenumber')
    ...:  .dropna('band', how='all')
    ...:  .dropna('wavenumber', how='any')
    ...:  .sel(band='bar'))
Out[21]:
<xarray.DataArray (wavenumber: 3)>
array([  1.20000000e-04,   1.00000000e-04,   8.50000000e-05])
Coordinates:
    band        <U3 'bar'
  * wavenumber  (wavenumber) float64 4.1e+03 4.1e+03 4.1e+03

But it's still a lot of code to write for such a common operation.
I'd be happy to think more deeply about this and contribute to the development of this great package ! (within the limits of my skills)

Thinking about this issue, I'd like to know what you think of the suggestions below before considering any pull request.
The following line code gives the same result than in my previous comment, but it is more explicit and shorter:
da.unstack('band_wavenumber').sel(band='bar').dropna('wavenumber', how='any')

A nice shortcut to this would be adding a new xs method to DataArray and Dataset, which would be quite similar to the xs method of Pandas but here with an additional dim keyword argument:
da.xs('bar', dim='band_wavenumber', level='band', drop_level=True)

Like Pandas, the default value of drop_level would be True. But here drop_level rather sets whether or not to apply dropna to all (unstacked) index levels of dim except the specified level.
I think that this solution is better than, e.g., directly providing index level names as arguments of the sel method. This may be confusing and there may be conflict when different dimensions have the same index level names.
Another, though less elegant, solution would be to provide dictionnaries to the sel method:
da.sel(band_wavenumber={'band': 'bar'})

Besides this, It would be nice if the drop_level=True behavior could be applied by default to any selection (i.e., also when using loc, sel, etc.), like with Pandas. I don't know how Pandas does this (I'll look into that), but at first glance this would here imply checking for each dimension if it has a multi-index and then checking the labels for each index level.

OK, I've read more carefully the discussion you referred to, and now I understand why it is preferable to call dropna explicitely. My last suggestion above is not compatible with this.
The xs method (not sure about the name) may still provide a concise way to perform a selection with explicit unstack and dropna. Maybe it is more appropriate to use dropna instead of drop_level:
da.xs('bar', dim='band_wavenumber', level='band', dropna=True)

The good news about writing our own custom way to select levels is that because we can avoid the stack/unstack, we can simply omit unused levels without worrying about doing dropna with unstack. So as long as we are implementing this in own other method (e.g., sel or xs), we can default to drop_level=True.
I would be OK with xs, but da.xs('bar', dim='band_wavenumber', level='band') feels much more verbose to me than da.sel(band_wavenumber={'band': 'bar'}). The later solution involves inventing no new API, and because dictionaries are not hashable there's no potential conflict with existing functionality.
Last year at the SciPy conference sprints, @jonathanrocher was working on adding similar dictionary support into .loc in pandas (i.e., da.loc[{'band': 'band'}]). I don't think he ever finished up that PR, but he might have a branch worth looking at as a starting point.

I think that this solution is better than, e.g., directly providing index level names as arguments of the sel method. This may be confusing and there may be conflict when different dimensions have the same index level names.

This is a fair point, but such scenarios are unlikely to appear in practice. We might be able to, for example, update our handling of MultiIndexes to guarantee that level names cannot conflict with other variables. This might be done by inserting dummy-variables of some sort into the _coords dict whenever a MultiIndex is added. It would take some work to ensure this works smoothly, though.

Besides this, It would be nice if the drop_level=True behavior could be applied by default to any selection (i.e., also when using loc, sel, etc.), like with Pandas. I don't know how Pandas does this (I'll look into that), but at first glance this would here imply checking for each dimension if it has a multi-index and then checking the labels for each index level.

Yes, agreed. Unfortunately the pandas code that handles this is a complete mess of spaghetti code (see pandas/core/indexers.py). So are welcome to try decoding it, but in my opinion you might be better off starting from scratch. In xarray, the function convert_label_indexer would need an updated interface that allows it to possibly return a new pandas.Index object to replace the existing index.

From this point of view I agree that da.sel(band_wavenumber={'band': 'bar'}) is a nicer solution!
I'll follow your suggestion of returning a new pandas.Index object from convert_label_indexer.
Unless I miss a better solution, we can use pandas.MultiIndex.get_loc_level to get both the indexer and the new pandas.Index object. However, there may still be some advanced cases where it won't behave as expected. For example, selecting both the band 'bar' and a range of wavenumber values (that doesn't exactly match the range of that band)
da.sel(band_wavenumber={'band': 'bar', 'wavenumber': slice(4000, 4100.3)})`

will a-priori return a stacked DataArray with the full multi-index:
In [32]: idx = da.band_wavenumber.to_index()
In [33]: idx.get_loc_level(('bar', slice(4000, 4100.3)), level=('band', 'wavenumber'))
Out[33]:
(array([False, False,  True,  True, False], dtype=bool),
 MultiIndex(levels=[['bar', 'foo'], [4050.2, 4050.3, 4100.1, 4100.3, 4100.5]],
            labels=[[0, 0], [2, 3]],
            names=['band', 'wavenumber']))

If you try that doing that indexing with a pandas.Series, you actually get an error message:
In [71]: s.loc['bar', slice(4000, 4100.3)]
# ...
/Users/shoyer/conda/envs/xarray-dev/lib/python3.5/site-packages/pandas/core/indexing.py in _getitem_axis(self, key, axis)
   1363             # nested tuple slicing
   1364             if is_nested_tuple(key, labels):
-> 1365                 locs = labels.get_locs(key)
   1366                 indexer = [ slice(None) ] * self.ndim
   1367                 indexer[axis] = locs

/Users/shoyer/conda/envs/xarray-dev/lib/python3.5/site-packages/pandas/core/index.py in get_locs(self, tup)
   5692         if not self.is_lexsorted_for_tuple(tup):
   5693             raise KeyError('MultiIndex Slicing requires the index to be fully lexsorted'
-> 5694                            ' tuple len ({0}), lexsort depth ({1})'.format(len(tup), self.lexsort_depth))
   5695
   5696         # indexer
KeyError: 'MultiIndex Slicing requires the index to be fully lexsorted tuple len (2), lexsort depth (0,)

I guess it's also worth investigating get_locs as an alternative or companion to get_loc_level.

Fixed in #802 and #947.


Possible issue when downgrading Frontend

Description of the issue
I'm not entirely sure what happened here but recording as best I can.
It seemed that when I downgraded Frontend, the layout in .tmp represented the version I downgraded from, when it should have been the downgraded version.
Steps to reproduce the issue

Create a prototype
Update Frontend to a prerelease from GitHub using npm install
Run the kit. Stop the kit
Downgrade Frontend to the previous version
Run the kit

Actual vs expected behaviour
Actual: The kit was still importing components, a feature of the new version, not the old version.
Expected: The kit should not have been importing components, representing the currently installed version of Frontend
Environment (where applicable)

Operating system: MacOS
Browser: Chrome
GOV.UK Prototype Kit version: 0.0.1-beta.3

Ben and I investigated and could not reproduce


Release GPU process memory after leaving for SD 1.5 and SD Turbo

Fix #61

Add a "Release memory after leaving" switch in demo pages of SD 1.5 and SD Turbo, enabled by default
Run session.release() for browser beforeunload event

When go to new page in same tab with "Release memory after leaving" switch disabled

The memories of GPU process and blink (Tab) process are still high
When click browser "Back" button, users can run "generate image" directly without loading models and compiling models

When go to new page in same tab with "Release memory after leaving" switch enabled

The memory of GPU process will be released
The memory of blink (Tab) process is still high
When click browser "Back" button, users need to re-load and compile models then generate images

Note
It seems like the ONNX Runtime Web cannot release memory of blink (Tab) process.

https://github.com/microsoft/onnxruntime/issues/18165

@fs-eire WebAssembly memory cannot shrink. This means the wasm memory will keep the size of the peak memory usage, even if the code marks some part as "free".

https://github.com/WebAssembly/memory-control/blob/main/proposals/memory-control/Overview.md

@fdwr PTAL
CC @Honry @mingmingtasd @huningxin

In the process of improving the memory code, I found that getElementById() and querySelector() were mixed used, and I couldn't help but change these codes. Will keep in mind the guidance of separating code formatting, refactoring, etc. from the original PR. :) @fdwr


TEST batch fill outlines PLEASE IGNORE

Signed-off-by: Oleksii Kanishchev<EMAIL_ADDRESS>

Will be addressed in #514 .


win10 sdk 10.0.19041 install stucks a few hours

I'm sorry I don't know where is the correct way to post this issue. I had try the official way, but it show me exception. Please someone forward this issue to Visual Studio Team.

Solved, https://blog.csdn.net/cloud701/article/details/112087273


Front end CI tests are failing in master branch

Describe the issue
Front end tests are failing on master branch
Screenshots or log output
If applicable, add screenshots or log output to help explain your problem.
Screenshots are non-inclusive, example of one of the test failures.

Expected behavior
Tests to complete and pass
Deployment information
Describe what you've deployed and how:

Elyra version: master

Fixed by #1660


Show displays immediately after construction by default

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

14 of 14 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.01%) to 99.52%

Totals

Change from base Build<PHONE_NUMBER>:
0.01%

Covered Lines:
415

Relevant Lines:
417

💛  - Coveralls

Fixes #63 as well.


Intg test: remove and replicate old integration tests

old integration tests should have their test cases duplicated in the new framework, and then should be removed
https://github.com/MinaProtocol/mina/tree/develop/src/app/cli/src/tests
test cases and logic duplicated:

[x] snarkless.  this test runs src/app/cli/src/tests/full_test.ml which actually has a number of other things, like testing for replay attacks, and doing math after a transaction to make sure the balances are as expected.  once those two items are updated in the new payments_test.ml, then this should be good to remove.  it also tests the functionality of snarkless, which isn't tested in the new framework, but we don't use snarkless for anything at the moment, so it's safe to remove just remove this.   when snarkless becomes relevant again we can create a snarkless test.  it also tests some snarked ledger functionality but that's not pressing either, snapps sort of covers that

[x] split.  while it would be desirable to have a split-brain/network-partition test, it looks like the "split" integration test running in CI is not testing that at all. it's actually running coda-shared-prefix-multiproducer-test which looks like it's just sending a few transactions, which is well covered by the payments test. this test thereby can be deleted

[x] delegation.  https://github.com/MinaProtocol/mina/pull/10325

[x] bootstrap.  this test is safe to completely remove, as it's test cases are already covered in the new framework

[x] shared state.  this test seems to be just sending a few transactions, so it can be deleted since that functionality is already tested in the new framework.

[ ] removal of all the above tests from the codebase once they have all been entirely obsoleted

a revised more involved plan to make obsolete the old integration tests is here:
https://app.zenhub.com/workspaces/velocity-62264fddc441a100183f7f86/issues/minaprotocol/mina/11017
closing this ticket


Support .yaml.j2 and .json.j2 file suffixes for --config-path

Title: Support .yaml.j2 and .json.j2 file suffixes for --config-path
Description:
Envoy currently uses the file suffix to determine the file format being used, as described in the docs: https://www.envoyproxy.io/docs/envoy/latest/operations/cli#cmdoption-c
However, this is a bit inflexible. We call our config files envoy.yaml.j2, since they are Jinja templates that we deploy using Ansible. It makes sense to use the .yaml.j2 extension in this case, since it makes it more clear both to text editors and humans that this file supports Jinja template mechanisms.
However, this interacts poorly with Envoy:
$ envoy --mode validate --config-path overrides/foo/envoy/some_host_name/envoy.yaml.j2

[2020-10-23 10:55:07.743][4000][critical][main] [external/envoy/source/server/config_validation/server.cc:59] error initializing configuration 'overrides/foo/envoy/some_host_name/envoy.yaml.j2': Unable to parse JSON as proto (INVALID_ARGUMENT:Unexpected token.

If I rename the file to envoy.yaml, the validation succeeds without errors. As a workaround, I could add a symlink from envoy.yaml to envoy.yaml.j2 and make it "work" for now.
Suggestion
I think it would make some sense to add .yaml.j2 and .json.j2 as supported file extensions by Envoy. Of course, this does not mean that Envoy would magically understand Jinja syntax (this is completely out of scope for this issue), but it would make the process slightly smoother in cases where Jinja expressions are not being used (my use case currently).
I don't know if this is considered an edge case, but it's a bit annoying. If you agree that it would make sense to fix this, please point me to the appropriate part of the Envoy source and I'll happily submit a PR to support this. Thanks for a great project. :+1:

IMO a better way to handle this would be to allow a CLI flag to force a particular file format. I think this would avoid wackamole on file extensions?

Agreed on CLI flag for file format. We can still guess using extension as we do today but override with this flag.

Works for me, sounds good. :+1: Would you like me to give it a try?

Sure @perlun, I've assigned the issue to you.

Thanks @htuch. A pointer or two about where to start in this case wouldn't hurt. :smile: I haven't contributed any (code) changes to Envoy previously, but I'm probably able to figure the basics out if you point me in the right direction.

@perlun check out https://github.com/envoyproxy/envoy/blob/b9ed0a991d90363ad04da2191891f54790482dcf/source/common/protobuf/utility.cc#L392.

Sorry, I haven't gotten around to doing this. @mattklein123 - feel free to add the "help wanted" label and we might get someone to pick it up.


JSON Error in Offline Debug Mode

When running the dashboard in debug mode without internet connection, callbacks writing to the lap-data cache fail with a JSON serialization error.
This is present even in a virtual environment with the same package versions as the server environment.

Might have something to do with failed fastf1.get_event_schedule calls


Word wrap resets to default value when saving or renaming file

VSCode Version: Code 1.15.1 (41abd21afdf7424c89319ee7cb0445cc6f376959, 2017-08-16T17:16:27.618Z)
OS Version: Linux x64 4.10.0-27-generic

Without any extensions or custom user/workspace settings, word wrap always turns off when saving a file or renaming the current file from the explorer sidebar.

Steps to Reproduce:

Press Ctrl+N and type a long line.
Press Alt+Z to enable word wrap.
Save the file and note that word wrap has been disabled.
Re-enable word wrap and rename the current file. Word wrap should disable again.

Expected behaviour: I think the word wrap setting should stay the same when editing a file, no matter whether its name or location changes.

Reproduces without extensions: Yes

When saving an untitled file, the URI of the file changes. Currently, toggled word wrap is captured per URI.

@alexandrudima currently not, we just dispose the untitled editor model and create a new model here.
We do setValue on the target and also carry over the encoding, we could carry over more state if you have a way for me to retrieve it and then set it (similar to the editor view state maybe?).

@bpasero The state is kept in ICodeEditorService via setTransientModelProperty and getTransientModelProperty. IMHO this should move to ... the configuration service ...

@alexandrudima yes, we talked about this with @sandy081.

@sandy081 is there a way now to ask for transient configuration for a text model and apply it to a new model?

There is in memory configuration which is available until window reload.

Update: we are not using the in-memory config service for the editor yet.


Refactoring get_data function

breakdown get_data funcion into small execute_mdx methods in CellService
Tests for new execute_mdx methods
Prettify execute_process function. Now takes **kwargs arguments
(dynamic arguments with same name as the TI Parameters). Rename of name_process argument to process_name.
Adapt Tests for ProcessService
oursource cellset deletion into higher order function: tidy_cellset
Creation of execute_view_* functions
Rename build_dygraph_arrays_from_cellset to
build_ui_dygraph_arrays_from_cellset and build_arrays_from_cellset to build_ui_arrays_from_cellset

@vviau
Please check my changes and accept the Merge.


exportDataset function fails due to getData function not being found

Trying to use the exportDataset function from the library, I run into an error where the getData function mentioned below is not found.
https://github.com/cbroeckl/RAMClustR/blob/eb15f37a77bac25fb59c90e5b212169464de62f2/R/exportDataset.R#L27-L32
Which function is this or is the exportDataset function deprecated?
I'm very thankful for any advice.

@hechth - sorry for the delay.  i was doing a bit of cleanup and got a bit sloppy in removing this function.  it was still present in more an internal package i maintain, so i failed to  notice this error.
I have just reinstated this in the github version a few minutes ago.   thanks for catching it!

@cbroeckl Thank you for pointing that out and fixing it! :)


can't see PR content, just a title

Extension version: 0.76
VSCode Version: 1.84.0
OS: Macos
Repository Clone Configuration (single repository/fork of an upstream repository): wrokspace
Github Product (Github.com/Github Enterprise version x.x.x):

Steps to Reproduce:

Open PR section
Click on any PR

Are you using GitHub Enterprise? If this is a public repo, can you share a link to the repo?

I experience the same. Using my private github account, part of a github org.
» git remote -v
...
leo     git@github.com:leonid-s-usov/ceph-ceph.git (fetch)
leo     git@github.com:leonid-s-usov/ceph-ceph.git (push)
origin  git@github.com:ceph/ceph.git (fetch)
origin  git@github.com:ceph/ceph.git (push)
...

@NarHakobyan please check the output for GitHub Pull Request, does it show rate limiting errors? That is likely the cause for me

@leonid-s-usov can you share the output?


Fix " Open north" logo.

Overview
The "open north" logo should be displayed correctly( The open north logo is affiliated to" Code for Canada").
Action Items

[ ] Please verify and correct "Open north" logo.

Resources/Instructions

https://opennorth.ca/

Looks like it was like that in the original spreadsheet. It's still wrong in the data we use to initialize dev environments for the backend:
https://github.com/civictechindex/CTI-website-backend/blob/9578c02f5940a8aa54244c344a84cf263e665754/init/organizations.json#L11155


Riscv64 support currently dropped from base image

Recent ubuntu/debian image dropped support for riscv64 platform. Reasons are explained in this commit and situation should be temporary (no idea how long it will take).
Still, riscv builds can't be provided during that time.
@olof-nord you may be interested in.

oh no! I was not aware of this, thanks for keeping me in mind @ChatDeBlofeld!

Still, riscv builds can't be provided during that time.

Spoken too quickly, just have to pin the latest working release. Not a long term solution though, but good enough for a while.


New package: Evdial.MikuMod version 21

[ ] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] This PR only modifies one (1) manifest
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.4 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: codeflow:open?pullrequest=https://github.com/microsoft/winget-pkgs/pull/113164&drop=dogfoodAlpha

@wingetbot waivers Add Validation-Domain

@wingetbot run

This package looks like a mod for another game - should that game be a dependency?

This package looks like a mod for another game - should that game be a dependency?

https://gamebanana.com/questions/21214

Requires FileDaddy, dependancy

Requires FileDaddy, dependancy

This package isn't currently in the WinGet public manifests, so there is an opportunity to add it as well.

/AzurePipelines run

Dependencies are now supported in the 1.6 client. If you could please submit a new manifest for the latest version of the package, it would be greatly appreciated!

@wingetbot waivers Add Validation-Domain

@wingetbot waivers Add Policy-Test-1.2

Requires FileDaddy, dependancy

This package isn't currently in the WinGet public manifests, so there is an opportunity to add it as well.

This is unchanged.
PS C:\ManVal> Find-WinGetPackage FileDaddy
PS C:\ManVal>


[Feature request] Better support for rotation

Currently, when my device is rotated, the gif is either stretched and compressed if it is fullscreen, or if zoomed in it is moved off center. It would be great to have an option to disable rotation, or to specify different gifs/options for a rotated screen.

@Rubiginosa probably the easiest would be to block rotation.
What device are you on? I have to admit I don't know how to test that, as I pretty much only have a Pixel 3 at my disposal, and by default the background does not rotate.

My bad, I can actually activate wallpaper rotation, so I can reproduce.
Okay, I'll figure out something to add an option to disable rotation

Fantastic! (in case you still wanted to know, both Samsung devices that I have rotate and rescale the background)


Bug: a user in the spiral simulation cannot set a timestamp

When entering a timestamp and pressing enter, the timestamp will continue at the previous position, but not from the entered time.

Ah it was never supposed to have that feature, will see now how feasible it is with the widget library that I'm using (dat.GUI)

It is not a bug, it is a lacking feature ;)


Improve ST make targets

Stop etcd at end of test.
Insert iptables rule to allow access to etcd.  Fixes #1186.

I'm not super stoked by custom iptables rules - but if it helps I think it's worth merging.


Fixed #32943 -- Added support for covering indexes and exclusion constraints for SP-GiST on PostgreSQL 14+.

ticket-32943
Currently PostgreSQL 14 is still in beta, so marking as a draft.
Pinging @hannseman who may be interested in this having added the support for covering indexes and exclusion constraints.

Merged the first two commits in dd26362f6312e7ba7b04099b5285d8e1d2900e63 and 000d4302341f49f2ec5f219d94d5dddac1a2acf5.

@ngnpope Thanks for this patch :+1: I rebased and pushed small edits.


Create an OpenRefine endpoint & add to user documentation

Latest specs: https://reconciliation-api.github.io/specs/latest/
User request: https://twitter.com/AWhitTwit/status/1570163562638876672

thank you for adding your vote @amandawhitmire. This issue is also related to comment by @nickynicolson  at https://github.com/gnames/gnfinder/issues/107, so OpenRefine endpoints are definitely getting high priority now.
I will talk with @debpaul, who knows much more about OR than me, and I hope we will be able to make something useful.

Thank you very much, @dimus! Not sure if there is anything I can do to help, but if so, please ask.

Thank you very much, @dimus! Not sure if there is anything I can do to help, but if so, please ask.

@amandawhitmire you'll be able to help us test! Stay tuned. You'll also be able to spread the word :-)

@amandawhitmire can I ask you to give a small input/output examples that you would expect from OpenRefine+gnverifier with some explanation of a context. I am very new to OpenRefine, and I need to understand a workflow of how it is used in the wild.

Hi @dimus - very happy to do this for you. I'm traveling today, but will get to this ASAP. Would screenshots be helpful, or just a description of the workflow? I want to provide whatever will be most helpful.

Should it be of any use, here's a Sinatra-based controller: https://github.com/bionomia/bionomia-api/blob/master/controllers/reconcile_controller.rb, and helper methods: https://github.com/bionomia/bionomia-api/blob/master/helpers/reconcile.rb. These in concert merely acts as wrappers for elasticsearch queries. The trickiest part of this is identifying what properties to use to help a user "refine" their likely output + some cosmetics in constructing a fly-out that works within the somewhat limited possibilities within OpenRefine (I believe it uses an iframe).
In any case, I recommend an entry for this in Wikidata (like this: https://www.wikidata.org/wiki/Property:P6944#P6269) such that the API endpoint can be dynamically included (and can be played with!) on https://reconciliation-api.github.io/testbench/#/.

@amandawhitmire @dshorthouse thank you
@amandawhitmire , imagine you have the best OpenRefine "gnresolver plugin" that does everything you need.
What such "plugin" provides to you? What kind of problems it solves?

Why the community needs a taxonomic reconciliation endpoint
Recognizing that GlobalNames tools and taxonomic authorities can be accessed via APIs, there are at least a couple of reasons why OpenRefine is still needed:

not all people who do biodiversity work know how to write code and/or use APIs (esp. citizen scientists working in WikiData, many librarians)
taxonomic names are often part of a dataset that needs parsing and cleaning, and OpenRefine is a great tool for this. It is ideal to have name verification and data cleaning happen in the same place.
Verifying names line-by-line in a tabular dataset is a challenge.

As an example, imagine that I have a dataset of species occurrences from an oceanographic cruise in 1967. I want to publish the dataset in GBIF (and OBIS), so I need to:

verify the name from that dataset to get the current accepted name;
get the identifier for that species from whatever authority was used for the verification;
get the higher classification

In my dream scenario, I can:

choose which taxonomic authority to reconcile against (I currently pull data from GBIF and from WoRMS)
query pulls scientific name, but once the name is matched I can also query for:
-- canonical name
-- authority ID (eg., AphiaID)
-- the higher classification

@dimus, I put together a couple of slides to offer a visual on what I mean/do.

As an example, imagine that I have a dataset of species occurrences from an oceanographic cruise in 1967. I want to publish the dataset in GBIF (and OBIS), so I need to:
1. verify the name from that dataset to get the current accepted name;

2. get the identifier for that species from whatever authority was used for the verification;

3. get the higher classification

In my dream scenario, I can:
* choose which taxonomic authority to reconcile against (I currently pull data from GBIF and from WoRMS)

* query pulls scientific name, but once the name is matched I can also query for:
  -- canonical name
  -- authority ID (eg., AphiaID)
  -- the higher classification

Hi @amandawhitmire could you share a smallish dataset of names you would like to run through this workflow?

Smallish is like, 100?
Neritina oualaniensis Lesson
Nerita signata MacLeay
Nerita lineata Gmelin
Littorina scabra Linne
Cerithideopsilla fluviatile Potiez and Michaud
Cerithium morus Lamarck
Chicoreus capucinus Lamarck
Nassarius gracilis Pease
Nassarlus muricatus Quoy and Gaimard
Nassarius thersites Bruguiere
Anadara sp. 1
Placuna placenta Linne
Anomalodiscus squamosa Lamarck
Katelysia rimularis Lamarck
Phacosoma cf. gruneri Philippi
Placamen isabellina Philippi
Diadora cf. mus Reeve
Scutus unguis Linne
Euchelus atratus Gmelin
Monodonta labio Linne
Turbo argyrostomus Linne
Turbo cf. ticaoniscus Reeve
Neritina oualaniensis Lesson
Nerita undata Linne
Littorina scabra Linne
Nodilittorina pyramidalis Quoy and Gaimard
Nodilittorina granularis Gray
Cerithium morus Lamarck
Planaxis sulcatus Born
Strombus canarium Linne
Strombus urceus Linne
Polinices flemingiana Recluz
Cypraea erosa Linne
Cypraea errones Linne
Chicoreus brunneus Link
Morula fiscella Gmelin
Morula musiva Kiener
Thais aculeata Deshayes
Thais mancinella Linne, 1758
Columbella scripta Lamarck
Cantharus fumosus Dillwyn
Nassarius cf. stolatus Gmelin
Siphonaria laciniosa Linne
Siphonaria cf. sipho Sowerby
Acar divarlcata Sowerby
Barbatia cf. decussata Sowerby
Arcopsis symmetrica Reeve
Striarca cf. tenebrica Reeve
Lithophaga cf. lima Lamy
Perna viridis Linne
Isognomon isognomon Linne
Malleus malleus Linne
Pinna bicolor Gmelin
Plicatula cf. gibbosa Lamarck
Lima fragilis Gmelin
Monia cf. caelata Reeve
Placuna ephippium Retzius
Scintilla sp. 3
Chama iostoma Conrad
Trachycardium flavum Linne
Circe scripta Linne
Gafrarium divaricatum Gmelin
Semele camicolor Hanley
Solen cf. vaginoides Lamarck
Rocellaria cuneiformis Spengler
Martesia striata Linne
Monodonta labio Linne
Trochus maculatus Linne
Trochus pyramis Born
Angaria delphinus Linne
Turbo argyrostomus Linne
Turbo cf. ticaoniscus Reeve
Nerita albicilla Linne
Littorina scabra Linne
Planaxis sulcatus Born
Strombus urceus Linne
Polinices flemingiana Recluz
Polinices mammilla Linne
Trivia oryza Lamarck
Cypraea errones Linne
Cypraea arabica Linne
Chicoreus brunneus Link
Morula fiscella Gmelin
Columbella fulgurans Lamarck
Columbella scripta Lamarck
Cantharus fumosus Dillwyn
Nassarius sp. 3
Strigatella amphorella Lamarck
Conus magus Linne
Siphonaria laciniosa Linne
Anadara cf. erythraeonensis Philippi
Barbatia amygdalumtostum Roding
Septifer bilocularis Linne
Isognomon isognomon Linne
Malleus malleus Linne
Pinctada vulgaris Schumacher
Pinctada sp. 1
Atrina vexillum Born
Spondylus cf. ducalis Roding

@debpaul here is an actual dataset I'm working with.

@dimus maybe this is an opportunity to do

the OpenRefine Plug-in OR Extension AND
then, using this dataset, write up how to do this with

OpenRefine AND
the API/s

FWIW, the minimum viable result is just the first step to verify the name via Reconciliation. After that, there are manual steps that are easy enough and can be written up (like what exists in GBIF documentation sprinkled around the web). So, here's an OpenRefine workflow I wrote up for myself last year:

Click on the column menu for 'VerbatimName -> Edit column -> Add column based on this column -> enter value in box. Name column 'WoRMS Name'
Click on column menu for 'WoRMS Name' -> Reconcile -> Start reconciling -> WoRMS, Scientific name
Click on column menu for 'WoRMS Name' -> Reconcile -> Add entity identifiers column. Name column 'WoRMS ID'
Click on column menu for 'WoRMS Name' -> Reconcile -> Start reconciling -> GBIF, Scientific name
Click on column menu for 'GBIF Name' -> Reconcile -> Add entity identifiers column. Name column 'GBIF ID'
Click on column menu for 'GBIF Name' and then Edit column > Add column by fetching URLs..., call the new
column 'GBIF API data'
Change the Throttle Delay to 250 and paste the expression:
"http://api.gbif.org/v1/species/match?verbose=true&name="+escape(value,'url')

This will take some time.

Go to 'GBIF API data', click on column menu and then Edit column > Add column based on this column... Call the new
column 'GBIF-speciesKey' and paste the expression:  value.parseJson().get("usageKey")
Go to 'GBIF API data', click on column menu and then Edit column > Add column based on this column...Call the new
column higherClassification and paste the expression:

value.parseJson().get("kingdom")+
", "+value.parseJson().get("phylum")+
", "+value.parseJson().get("class")+
", "+value.parseJson().get("order")+
", "+value.parseJson().get("family")

You will see the Kingdom, Phylum, Class, Order and family of each taxon.

@amandawhitmire thank you very much, I  am dealing with name indexing for BHL, and then we have COL meeting next week, sorry for delay with this issue, hopefully I'll get back to it soon!

No worries, @dimus! I know you have a lot on your plate. I'm grateful that you have this on your list. TY!

@amandawhitmire

Many folks use APIs now, but many others still prefer or rely on OR for data cleaning and enhancement.

I see OR as a first step into using APIs - and its good to explain that an OR reconciliation endpoint is also an API endpoint. Obvious benefit is that we have a standard "matching" API which we can swap between different implementations. For those coding in Python there is a library which enables the reconciliation of data in a pandas dataframe against an OR endpoint: (https://github.com/jvfe/reconciler)- I guess there may be an R equivalent?


Error 500 on invalid token length

Otp api does not adhere to the proper response contract when passed a token with an invalid length.

OTP is now removed from this repository