Wrong wakeUpInterval value is submitted to device

Checklist

[X]  I am not using Home Assistant. Or: a developer has told me to come here.
[X] I have checked the troubleshooting section and my problem is not described there.
[X] I have read the changelog and my problem is not mentioned there.

Deploy method
Docker
Z-Wave JS UI version
8.4.1.ac29618
ZwaveJS version
10.3.0
Describe the bug
When setting the wakeUpInterval in the frontend, a different value is written to the device. As the log below shows, I set the wakeUpInterval to 3600 seconds, and then zwave-js-ui/zwave-js ignores 3600 and sends 14400 instead.
2022-11-17 08:56:01.569 INFO Z-WAVE: Calling api writeValue with args: [
  {
    nodeId: 56,
    commandClass: 132,
    endpoint: 0,
    property: 'wakeUpInterval'
  },
  3600,
  {},
  [length]: 3
]
2022-11-17 08:56:01.575 INFO Z-WAVE: Writing 3600 to 56-132-0-wakeUpInterval
2022-11-17 08:56:06.349 INFO Z-WAVE: Node 56 is now awake
2022-11-17 08:56:13.122 INFO Z-WAVE: Success zwave api call writeValue true
2022-11-17 08:56:13.195 INFO Z-WAVE: Success zwave api call writeValue true
2022-11-17 08:56:14.552 INFO Z-WAVE: Node 56: value updated: 132-0-wakeUpInterval 14400 => 14400
2022-11-17 08:56:14.555 INFO Z-WAVE: Node 56: value updated: 132-0-controllerNodeId 0 => 0
2022-11-17 08:56:14.921 INFO Z-WAVE: Success zwave api call healNode true
2022-11-17 08:56:15.950 INFO Z-WAVE: Node 56 is now asleep

The device is a Nexa SP816 motion sensor (a rebranded EverSpring SP816). I have no other battery powered device where this setting can be modified. Please let me know if I can provide any more details.
To Reproduce

Set the wakeUpInterval in the web UI
Wake up the device so it receives the new settings
Observe the wrong setting being submitted

Expected behavior
I would expect the correct value to be submitted.
Additional context
No response

PDF manual for the SP816 device, documenting the device will accept shorter wakeUpInterval values: https://www.everspring.com/wp-content/uploads/2019/04/SP816-A501112630R_bak_20170714-1.pdf

@bruberg I have a feel this is a device issue, as log shows there is no error while writing the value.
Please attach also zwave-js driver logs
cc @AlCalzone


parser confusion on directive followed by list in a list

Subject of the issue
const micromark = require("micromark/lib");
const directive = require("micromark-extension-directive");

micromark(`:::i
- +
a`, {
  extensions: [directive()],
});

throws
node:assert:385
    throw err;
    ^

AssertionError [ERR_ASSERTION]: expected a previous token
{
  generatedMessage: false,
  code: 'ERR_ASSERTION',
  actual: undefined,
  expected: true,
  operator: '=='
}

Your environment

OS: Ubuntu
Packages:├──<EMAIL_ADDRESS>└──<EMAIL_ADDRESS>

Env: node v15.2.1, npm 7.0.8

Steps to reproduce
run:
const micromark = require("micromark/lib");
const directive = require("micromark-extension-directive");

micromark(`:::i
- +
a`, {
  extensions: [directive()],
});

Expected behavior
No error, or a more specific markdown syntax related error
Actual behavior
node:assert:385
    throw err;
    ^

AssertionError [ERR_ASSERTION]: expected a previous token
{
  generatedMessage: false,
  code: 'ERR_ASSERTION',
  actual: undefined,
  expected: true,
  operator: '=='
}

Tracing this some more
this does not appear to be the missing previous
https://github.com/micromark/micromark-extension-directive/blob/67126633ba8a59c8539006e218f0552c14003c5d/lib/tokenize-directive-container.js#L113-L118
it's getting a value
nor does
https://github.com/micromark/micromark-extension-directive/blob/67126633ba8a59c8539006e218f0552c14003c5d/lib/tokenize-directive-text.js#L31
which has a code a expected
https://github.com/micromark/micromark/blob/main/lib/tokenize/list.mjs doesn't appear to have a previous token requirement(?)

Interesting. So, without the container we have a list item that start with a blank line in a list. That blank line starting the list item means it can’t start lazy continuation, so a is not in it, but thus a sibling to the parent list.

a

The “unraveling” of tokens is a bit complex. Because markdown is indent based, some spaces or markers are already parsed by a container (e.g., *  and on the next line   (two spaces) for a fictional list), and then the rest of the lines (say, a\n and b still have to be parsed). And because we have a flat list of tokens, a paragraph then has to start before a, and end after b, leading to the complex nesting.
If a developer (so, me here) makes a mistake, expected a previous token is thrown: https://github.com/micromark/micromark/blob/ac44b027357e36694efd2c59babba1b89515e73c/lib/util/subtokenize.mjs#L164

This might all be just because markdown (or no other micromark extension) has something that can contain containers.
Because this is sort of like fenced code, which wrap them 🤔

has something that can contain containers

I was thinking the same thing, the thing that confuses me.
:::a
*
a

parses fine
as does
:::a
* a

and
:::a
* -

even
:::a
* - a
a

it's oddly specific to empty nested containers.

Can also produce it with :::i\n> >\na but not :::i\n> \na

interesting
:::a
> >
>

parses
:::a
> > >
> >

parses as well
but,
:::a
> > >
>

fails
does not, the 2+ layers of nesting of empty seems to be the constant

wellll... this is weird 🙄

I'm seeing a similar issue:
:::card
Hello!

- This
  - Is
  - A
  - List

**Some post list content.**
:::

TypeError: token is undefined
    subcontent subtokenize.js:179
    subtokenize subtokenize.js:60
    postprocess postprocess.js:6
    fromMarkdown index.js:25
    parse index.js:13
    parse index.js:284
    pipelineParse index.js:23
    wrapped wrap.js:25
    next index.js:57
    run index.js:31
    executor index.js:375
    process index.js:370
    processSync index.js:399

I'm seeing a similar issue:
:::card
Hello!

- This
  - Is
  - A
  - List

**Some post list content.**
:::

TypeError: token is undefined
    subcontent subtokenize.js:179
    subtokenize subtokenize.js:60
    postprocess postprocess.js:6
    fromMarkdown index.js:25
    parse index.js:13
    parse index.js:284
    pipelineParse index.js:23
    wrapped wrap.js:25
    next index.js:57
    run index.js:31
    executor index.js:375
    process index.js:370
    processSync index.js:399

Looking into it again. Re-iterating https://github.com/micromark/micromark-extension-directive/issues/3#issuecomment-748538844: the document content type which contains containers such as lists and block quotes, which in normal markdown is the top-level content type, is allowed here as a child content type (of the directive).
I’m assuming something weird is going on when it’s not the top-level thing


Flake: TestPipelineTaskTimeout

logs: https://prow.tekton.dev/view/gs/tekton-prow/pr-logs/pull/tektoncd_pipeline/6942/pull-tekton-pipeline-beta-integration-tests/1681661652312788992
    timeout_test.go:399: Waiting for TaskRun from PipelineRun pipeline-task-timeout-kqxsnxcj in namespace arendelle-w7tsn to be timed out
    timeout_test.go:430: Error waiting for TaskRun pipeline-task-timeout-kqxsnxcj-pipelinetask1 to timeout: taskRun "pipeline-task-timeout-kqxsnxcj-pipelinetask1" failed, but should have been Succeeded

link to test case
It looks like one of the pipeline tasks is supposed to time out quickly, and did, and the other pipeline task is supposed to succeed quickly, but timed out.

also observed on https://github.com/tektoncd/pipeline/pull/6621


Float Equality

Add a utility function that can compare two floats when given an epsilon value.

I'm on it


Remove the angular dependency

I don't want it to also install angular alongside itself in bower as I use a mix of bower and npm.
Nobody is going to use this without Angular, the dependency doesnt need to be in the bower file.
I removed this...
"dependencies": {
"angular": ">=1.2"
},
I think it makes sense for ng-lodash to have that properly removed from its bower.json file.

No comment? This seems a valid suggestion.

To be honest this is quite common when it comes to bower for example if you look at something like ui-router they will do something the same it's more to ensure that the version of Angular supported by ng-lodash is downloaded. When wanting to use an Angular version which is not supported this is when you have to force your version of Angular to be a specific one.


avaloniaui support

https://github.com/StephenCleary/Hosting/issues/3

So I tried this out, and works nicely.... however... this initializes and builds the TApplication when adding it as a service..
Which makes it very difficult to for use  other services while for example App::OnFrameworkInitializationCompleted is executed.
So, to fix that you can delay said creation by chaning:
public AvaloniauiApplicationLifetime(IHostApplicationLifetime applicationLifetime, TApplication application)

to
public AvaloniaApplicationLifetime(IHostApplicationLifetime applicationLifetime, IServiceProvider provider)

and delay getting the TAppliction instance until the WaitForStartAsync, by doing:
var app = _provider.GetRequiredService<TApplication>();

So I tried this out, and works nicely.... however... this initializes and builds the TApplication when adding it as a service.. Which makes it very difficult to for use other services while for example App::OnFrameworkInitializationCompleted is executed.
So, to fix that you can delay said creation by chaning:
public AvaloniauiApplicationLifetime(IHostApplicationLifetime applicationLifetime, TApplication application)

to
public AvaloniaApplicationLifetime(IHostApplicationLifetime applicationLifetime, IServiceProvider provider)

and delay getting the TAppliction instance until the WaitForStartAsync, by doing:
var app = _provider.GetRequiredService<TApplication>();

I ended up with this: https://gist.github.com/tvandijck/6341cf0c92f3b67c89ab2ebb715fd77b

Thanks for your advice and nice job. I have updated this PR just now!


addresses - suite, unit, etc., and fix to not show physical if mailing is physical address

Also - need to point out to Quinten that the apt/suite/floor pdf attachment block fields seem inconsistent between physical and mailing...

The idea is to hide the mailing address fields if the two addresses are identical. It's not clear if this is needed but it does seem closer to the way a real person would fill in this field.
WOuld just need some mako if statements in the attachment block: % if users[i].mailing_address == users[i].address: e.g.

TRP said hiding mailing address if same as physical address is not needed.


adding profile to public/data

Fixes Issue

Changes proposed

Check List (Check all the applicable boxes)

[ ] My code follows the code style of this project.
[ ] My change requires changes to the documentation.
[ ] I have updated the documentation accordingly.
[ ] All new and existing tests passed.
[ ] This PR does not contain plagiarized content.
[ ] The title of my pull request is a short description of the requested changes.

Screenshots

Note to reviewers

@Panquesito7 Hey, I made the changes. Pls review it and tell me if it looks good. Thanks 😄


unable to retrieve the data from cassandra node though Stargate API (REST API)

Discussed in https://github.com/stargate/stargate/discussions/1595

Originally posted by Madhusudanarao123 January 27, 2022
Hi Team,
we are unable to retrieve the data from cassandra node though Stargate API
Thanks & Regards,
Madhu

GCP level create 3 node cluster and separate vm install stargate, We are facing the issue like Stargate API server not able to communicating with Cassandra node for Retrieve data.
PFB details..
Versions:
OS : Centos 7
Stargate Version : 1.0.41
Cassandra Version : 4.0.1
Using Command
curl -s -L -X GET '{emp2}{users_keyspace}/{rkeyspace}/{rtable}?where={"firstname":{"$in":["sai","Madhu"]}}' -H "X-Cassandra-Token: $AUTH_TOKEN" -H "Content-Type: application/json"
Issue: command excuted sucessfully but output not showing

Team,
please help on this issue..

We can not help until you show us the full URL you are trying to call?

STARGATE GATEWAY.docx
we are trying above document commands every command working fine expect GET DATA command. please find the attachment and do need full..
PFB details..
Versions:
OS : Centos 7
Stargate Version : 1.0.41
Cassandra Version : 4.0.1
Using Command
curl -s -L -X GET '{emp2}{users_keyspace}/{rkeyspace}/{rtable}?where={"firstname":{"$in":["sai","Madhu"]}}' -H "X-Cassandra-Token: $AUTH_TOKEN" -H "Content-Type: application/json"
Issue: command excuted sucessfully but output not showing

Your column is called name and you are searching for the firstname. :arrow_down:
curl -s -L -X GET '{emp2}{users_keyspace}/{rkeyspace}/{rtable}?where=\{"firstname":\{"$in":\["sai","Madhu"\]\}\}' -H "X-Cassandra-Token: $AUTH_TOKEN" -H "Content-Type: application/json"

Team i am excuted the below table only.
Keyspace name: users_keyspace
Table name : emp2

Can you hit the command like below, but without any variables, substitute {emp2}{users_keyspace}/{rkeyspace}/{rtable} with real path. Note the -v after the curl please..
curl -v -s -L -X GET '{emp2}{users_keyspace}/{rkeyspace}/{rtable}?where=\{"firstname":\{"$in":\["sai","Madhu"\]\}\}' -H "X-Cassandra-Token: $AUTH_TOKEN" -H "Content-Type: application/json"

Just tried the same request on my machine and it works fine:
$ curl -v -s -L -X GET 'http://<IP_ADDRESS>:8082/v2/keyspaces/docs_search/products?where=\{"key":\{"$in":\["41613e59-5726-4240-84fb-31367ff35fc2","Madhu"\]\}\}' -H "X-Cassandra-Token: ff0b0f1b-8084-492b-9076-ac2d26a2f0d2" -H "Content-Type: application/json"
*   Trying <IP_ADDRESS>:8082...
* TCP_NODELAY set
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 8082 (#0)
> GET /v2/keyspaces/docs_search/products?where={"key":{"$in":["41613e59-5726-4240-84fb-31367ff35fc2","Madhu"]}} HTTP/1.1
> Host: <IP_ADDRESS>:8082
> User-Agent: curl/7.68.0
> Accept: */*
> X-Cassandra-Token: ff0b0f1b-8084-492b-9076-ac2d26a2f0d2
> Content-Type: application/json
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Tue, 15 Feb 2022 09:24:48 GMT
< Content-Type: application/json
< Vary: Accept-Encoding
< Content-Length: 4182
<
{"count":6,"data":[....]}

Please recheck the path you are trying to execute. Or give the output with -v.

@Madhusudanarao123 Any updates, can I close the issue or not yet?

still issue not resolved, alternatively working in java code and  spyspark side checking for resolution.

No updates for over a year, no new info, closing.


SyntaxError: Cannot use import statement outside a module - Next js

Hi, I'm having this issue with this library and I don't know how to fix it. I'm using Next js
Sometimes it renders, but when you reload the error happens.
Here is an example of the error: https://codesandbox.io/s/affectionate-easley-8by211?file=/pages/index.tsx
The terminal error:
import { createStyled, shouldForwardProp } from '@mui/system';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at wrapSafe (internal/modules/cjs/loader.js:1001:16)
    at Module._compile (internal/modules/cjs/loader.js:1049:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
    at Module.load (internal/modules/cjs/loader.js:950:32)
    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
    at Module.require (internal/modules/cjs/loader.js:974:19)
    at require (internal/modules/cjs/helpers.js:101:18)
    at Object.<anonymous> (/sandbox/node_modules/mui-nested-menu/components/IconMenuItem.js:56:32)
    at Module._compile (internal/modules/cjs/loader.js:1085:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)

I've fixed it locally just changing the way that the component IconMenuItem imports styled
import { styled } from '@mui/material/styles';
I hope this help more people

Hey @Donivanes,
Check out this recent change to the import statements...
Can you just double check in your local build that importing like this works too:
import {styled} from '@mui/system';

cc @jraoult - thought you might be interested in this. Any thoughts from your recent research?

Hey @Donivanes,
Check out this recent change to the import statements...
Can you just double check in your local build that importing like this works too:
import {styled} from '@mui/system';

cc @jraoult - thought you might be interested in this. Any thoughts from your recent research?

Yes, with that import it also works.
I've also tried with the previous version (2.1.1) and the same error happens (check the codesandbox). That could be because of a MUI package recent updates?

@steviebaa @Donivanes, it is not related to the changes we made in the aforementioned PR, but it is defo related to the way ESM modules and CommonJS modules interact.

mui-nested-menu is only published as a CommonJS package: exclusively uses require to load dependencies)
MUI is published in both CommonJS format and ESM format: it can be imported with require or import if the config is right
it is not possible to use require to import an ESM module

Somehow, in the provided code sandbox, the Node CommonJS loader is loading the ESM module of styled instead of the CommonJS version  and then throws an error since it is not supported. I know almost nothing at NextJS, so I can not really help further, I'm afraid, but I'd try to understand why the Node loader is trying to load the wrong file

Nice one, thanks Jon.
Ivan, I'll try getting around to creating a branch which has the esm and cjs versions in the build. Ill try downloading that sandbox and testing it with that. also ill let you know so you can give it a go and we can publish if all goes well.
Im a bit flat out this week/weekend so might take me a week to get around to it.

Too interesting to not a quick go, who needs sleep...
@Donivanes If you want to pull down this branch then run the command npm run pack. Then try using the generated /dist folder in your project.
I haven't tested it myself yet.

I get this issue in jest tests - using the public import fixed it (in this case I mocked it):
jest.mock('@mui/material/styles/styled', () => {
  const { styled } = require('@mui/material');

  return styled;
});

I'm still getting the originally posted issue in NextJS:
import { createStyled, shouldForwardProp } from '@mui/system';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at Object.compileFunction (node:vm:360:18)
    at wrapSafe (node:internal/modules/cjs/loader:1055:15)
    at Module._compile (node:internal/modules/cjs/loader:1090:27)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1180:10)
    at Module.load (node:internal/modules/cjs/loader:1004:32)
    at Function.Module._load (node:internal/modules/cjs/loader:839:12)
    at Module.require (node:internal/modules/cjs/loader:1028:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/Users/jamesdaniels/next-amplified/node_modules/mui-nested-menu/index.js:7:38)
    at Module._compile (node:internal/modules/cjs/loader:1126:14) {
  page: '/'
}

"dependencies": {
...
    "@emotion/react": "^11.10.0",
    "@emotion/styled": "^11.10.0",
    "@mui/icons-material": "^5.8.4",
    "@mui/lab": "^5.0.0-alpha.96",
    "@mui/material": "^5.9.2",
    "@mui/x-data-grid": "^5.17.10",
    "@mui/x-date-pickers": "^5.0.0-beta.3",
    "mui-nested-menu": "^3.1.0",
    "next": "12.2.3",
    "react": "^17.0.2",
    "react-dom": "^17.0.2"
  },

Is there anything I can do to fix this?

i'm experiancing the same error
import { createStyled, shouldForwardProp } from '@mui/system';
^^^^^^

SyntaxError: Cannot use import statement outside a module

tried copying the code of the package directly to my project (that works fine) but i would rather use the package

Facing this issue as well.

In this thread MUI dev says 4 level import like @mui/material/styles/styled is not supported, so definitely something needs to be updated in mui-nested-menu codebase

P/R added: https://github.com/steviebaa/mui-nested-menu/pull/32

Ran into the same issue, wanted to say PR #32 fixed it for me

In this thread MUI dev says 4 level import like @mui/material/styles/styled is not supported, so definitely something needs to be updated in mui-nested-menu codebase

this work for me.
just have a file with the import "@mui/material/styles/styled" remove styled to import {styled} @mui/material/styles and work it. thank's


require() Caching - causes issues with reports

The fact that require() will cache modules, and will not reload them even with explicit require() calls.
A work-around or means to disable the caching is needed.

This is the work-around (tested OK) -
    // NOTE: require() does caching of modules, info:
    //      https://bambielli.com/til/2017-04-30-node-require-cache/
    //
    // Also using every() instead of forEach() to allow the ability
    // to break the loop:
    //      https://masteringjs.io/tutorials/fundamentals/foreach-break
    Object.keys(require.cache).every( (key) => {
        if(key.includes('reportdefs.js')) {
            // remove the module from the cache
            delete require.cache[key];
            // done!
            return false;
        }
        return true;
    });

    const reportdefs = require('./reportdefs.js');

resolved in the 20210531-logreporter branch


Error 500 occured 20.1. ~11:30-->

Hey,
On 20.1. around 11:30...12:00 the HA stopped receiving any nysse data (2 stops) with error:
Error
Config flow could not be loaded: 500 Internal Server Error Server got itself in trouble
Idk what happened nor why. Restarting HA and/or entities/integration had no effect.

Out of nowhere the integration works again...partially but the problem now seems to be on Nysses side; on their website there are only a small amount of departures visible per stop (idk why?) but due to that i have 1-3 real time schedules shown correctly and rest are after 00:00 even if i check in the morning at 8:00.
in the other words only realtime ones are shown and no scheduled/planned ones are visible.


Added the video in readme file and in docs

Current coverage is 34.08% (diff: 100%)

Merging #931 into development will not change coverage

@@           development       #931   diff @@
=============================================
  Files                7          7
  Lines             1021       1021
  Methods            131        131
  Messages             0          0
  Branches           157        157
=============================================
  Hits               348        348
  Misses             673        673
  Partials             0          0

Powered by Codecov. Last update afefee3...bf68abd

Nice video. Thanks for the request. I would prefer you to write the email address initially instead of doing it in middle. Also, could you make a gif like the existing one. It is super easy. Also record in full screen mode.
http://ricardoalcocer.com/2014/04/adding-videos-readme-file-github-repository/

@SrijanJha28 please follow best practices like testing first and then making a PR and making a summary of the PR in the PR body please do that.

@Princu7 Students are not supposed to write their emails in the Readme.
Add your email id, name of app (name of event),
This statement is a part of documentation which guides the users how to use the WebApp generator.

@the-dagger Thanks for the review. When I said write email first, I meant that in the context of gif. That is, in the initial gif, he was first uploading the zip file and then writing his email address in the form. I wanted him to follow the chronological order and enter the email address first and then upload the zip. :smile: Writing email in Readme is certainly wrong.
Also, the existing gif is a little outdated. We just wanted to update it. Would love to hear some of your suggestions. Thanks.

Got it!
@SrijanJha28 took it in the wrong context then I suppose.
Also for the gif, the gif that he has uploaded is very low both in terms of resolution and quality, so it would be nice if it could be increased by a bit.

@the-dagger No problem at all. Yeah, a slight miscommunication. I totally agree with you. I quality of the gif is really poor and needs to be improved.


Remove firestone references from code

Firestone reference and code removal https://gerrit.openbmc-project.xyz/#/c/10957/

Had a discussion with @gtmills and we are removing it from the test repo as well

https://gerrit.openbmc-project.xyz/#/c/10964/


Adds documentation for helper functions

Signed-off-by: Harsha Vamsi Kalluri<EMAIL_ADDRESS>Description
Adds documentation for various methods in the helper library
Issues Resolved
Closes #187
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@harshavamsi Want to finish this?

I'm going to close this for now, and if needed we can try again without the requirements changes.


Possible implementation of Casefolding String

@parijatm Here might be a way to implement casefolding strings.
It seems to assert appropriately in both comparisons with a standard string.
import pytest
from collections.abc import Sequence

class SpiceStr(Sequence):
    def __init__(self, s):
        self.s = s

    def __eq__(self, other):
        assert type(other) == SpiceStr
        return self.s.casefold() == other.s.casefold()

    def __hash__(self):
        return hash(self.s.casefold())

    def __str__(self):
        return self.s

    def __contains__(self, t):
        return self.s.casefold().__contains__( t.casefold())

    def __getitem__(self, i):
        return self.s[i]

    def __len__(self):
        return len(self.s)

def test_A():
    assert SpiceStr("Steve") == SpiceStr("steve")
    assert SpiceStr("Steve") != SpiceStr("burns")
    with pytest.raises(AssertionError):
        SpiceStr("Steve") == "steve"

    with pytest.raises(AssertionError):
        "steve" != SpiceStr("Steve")

    assert "Ste" in SpiceStr("steve")
    assert not "Burns" in SpiceStr("steve")

    # Get these because we are inheriting from collections.abc.Sequence

    assert SpiceStr("Steve")[0] == 'S'
    assert len(SpiceStr("Steve")) == 5

    assert list(reversed(SpiceStr("Steve"))) == list("evetS")

    assert list("Steve") == [c for c in SpiceStr("Steve")]

@parijatm Looks like you can make comparisons with regular strings work as well (you don't need to assert---maybe we should anyway):
import pytest
from collections.abc import Sequence

class SpiceStr(Sequence):
    def __init__(self, s):
        self.s = s

    def __eq__(self, other):
        t = other if type(other) == SpiceStr else SpiceStr(other)
        assert type(t) == SpiceStr
        return self.s.casefold() == t.s.casefold()

    def __hash__(self):
        return hash(self.s.casefold())

    def __str__(self):
        return self.s

    def __contains__(self, t):
        tt = str(t) if type(t) == SpiceStr else t
        return self.s.casefold().__contains__( tt.casefold())

    def __getitem__(self, i):
        return self.s[i]

    def __len__(self):
        return len(self.s)

def test_A():
    assert SpiceStr("Steve") == SpiceStr("steve")
    assert SpiceStr("Steve") != SpiceStr("burns")

    assert SpiceStr("Steve") == "steve"

    assert "steve" == SpiceStr("Steve")

    assert "burns" != SpiceStr("Steve")

    assert "Ste" in SpiceStr("steve")
    assert not "Burns" in SpiceStr("steve")

    assert SpiceStr("Ste") in SpiceStr("steve")

    # Get these because we are inheriting from collections.abc.Sequence

    assert SpiceStr("Steve")[0] == 'S'
    assert len(SpiceStr("Steve")) == 5

    assert list(reversed(SpiceStr("Steve"))) == list("evetS")

    assert list("Steve") == [c for c in SpiceStr("Steve")]

def test_B():
    s = { SpiceStr("Steve"), SpiceStr("Burns")}
    assert SpiceStr("Steve") in s
    assert SpiceStr("steve") in s

    assert "steve" in s
    assert "foo" not in s

Looks really good Steve ! Having said that, the following implementation appears to satisfy all tests as well.
class SpiceStr(str):

    def __eq__(self, other):
        return super().casefold() == other.casefold()

    def __hash__(self) -> int:
        return super().casefold().__hash__()

    def __contains__(self, other):
        return super().casefold().__contains__(other.casefold())

What was unexpected is that the following works for both implementations:
assert SpiceStr("Steve") == "StEve"
assert "StEve" == SpiceStr("Steve")

Which basically means that SpiceStr.__eq__() is called irrespective of whether the SpiceStr object is on the left or right of the == expression. So my previous concerns were a non-issue.

Even better. I'd leave a comment in there (or these tests) so that future maintainers know that this works since it wasn't obvious to us. And should you use str or collections.UserString as the base class?

I am adding the tests directly to #794 as well. Can also leave a pointer to this issue somewhere in the code.
As far as str vs. collections.UserString goes, I stuck with str upon reading this:

The class, UserString acts as a wrapper around string objects. The need for this class has been partially supplanted by the ability to subclass directly from str; however, this class can be easier to work with because the underlying string is accessible as an attribute.

I haven't seen the need to directly access the unmodified underlying string (yet). We can switch to UserString if we encounter said need.

Relevant blob I found in SO. Rich compare is tried both ways for existence of overloading.
https://github.com/python/cpython/blob/v3.5.10/Objects/object.c#L649-L699


linuxKernel.kernels.linux_zen: 6.10.9-zen1 -> 6.11-zen1; linuxKernel.…

…kernels.linux_lqx: 6.10.9-lqx1 -> 6.10.11-lqx2
Description of changes
Updated the zen and lxq kernels to their latest upstream version
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Superseded by https://github.com/NixOS/nixpkgs/pull/347734


Can't find the file of 'data/cifar10/train.txt'

Hi there, I just cant find the dir 'data/cifar10/train.txt', can you provide it?

May help. https://github.com/caoyue10/cvpr17-dvsq/tree/aaai16-dqn/data/nuswide_21

Hi, Sorry for the late reply. I've updated the readme for more detail about the dataset. Feel free to contact us if you still have any questions.


gui-apps/oguri: also dobin swaybg-oguri

The wrapper script (noted on the repo readme) is missing from the package

Seems like oguri development has stopped, I might possibly remove this package now as it is dead upstream.


Bug: prompt for confirmation of cluster id on delete

Fix #90

Are we okay with uncolored text for the prompt?

would it make more sense for it to be a y/n question instead of re-typing the cluster id? not really sure, but just what i would expect intuitively.

Typing y/n is a bit more intuitive. I decided to require typing the full cluster id as that is what the Azure portal requires when you delete a cluster.
Either is fine imo


filters

In this PR, I have tried to port the work on ffmpeg filters from this fork.
Many thanks to @schauveau for doing the initial work on this! The forked repo also contains a lot of useful tips, take a look if you don't know what filters are useful.
There were many changes in that fork. So far, I have picked only those that enable filters to work in wf-recorder. To my surprise, this resulted in a big simplification where the frame handling is done, because now we can just use a filter to upload data in the case of VAAPI codec. Since we can also convert data on the GPU with scale_vaapi, I think that OpenCL support is no longer necessary.
All in all, this is a pretty big change, so it would be good if more people can test before we merge this. A 0.3 release will probably make sense afterwards.
Fixes #3 (you can rotate with a filter)
Fixes #20 (you can limit framerate with -F fps=30 or -D -F framestep=2)
Fixes #38
Fixes #77 (scale/scale_vaapi filter)
Fixes #87 (because opencl is not a dependency anymore)
Happy new year everyone!

Can't wait for this to drop! Tell me if you need testing done or help with anything other than c++ coding ;)

Can't wait for this to drop! Tell me if you need testing done or help with anything other than c++ coding ;)

Can't wait for this to drop! Tell me if you need testing done or help with anything other than c++ coding ;)

This PR is waiting for testing ;) Otherwise, there's nothing else to do.

Can't wait for this to drop! Tell me if you need testing done or help with anything other than c++ coding ;)

This PR is waiting for testing ;) Otherwise, there's nothing else to do.

Nice work @ammen99, I am happy to test!
With v4l2loopback I sadly don't notice any performance improvements, in fact with downscale filter applied to record in FullHD instead of 4k the idle CPU usage is increased from 6% to 9%, and when I play the stream in mpv in both cases CPU usage goes to 100%. If on the other hand I select 1/4th of the screen with -g and don't apply any scale filters, CPU usage will be 2% in idle and 25% when is playing in mpv.
$ ./wf-recorder-filters --muxer=v4l2 --codec=rawvideo --pixel-format=yuv420p -F 'scale=w=1920:h=1200' --file=/dev/video9

With vaapi, sadly I can't even get it to work in this branch 😞
When running from master, this works:
$ ./wf-recorder-master -f test-vaapi.mkv -c h264_vaapi -d /dev/dri/renderD128
selected region 0 0 0 0
[h264_vaapi @ 0x7fcaac001d00] No quality level set; using default (20).
Output #0, matroska, to 'test-vaapi.mkv':
    Stream #0:0: Unknown: none (h264_vaapi)
[matroska @ 0x7fcaac000d40] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
^C[matroska @ 0x7fcaac000d40] pts (0) < dts (1570) in stream 0

When running from filters branch, it crashes:
$ ./wf-recorder-filters -f test-vaapi.mkv -c h264_vaapi -d /dev/dri/renderD128
selected region 0 0 0 0
[Source @ 0x7f4ec404e100] sws_param option is deprecated and ignored
Using video filter: hwupload,scale_vaapi=format=nv12
[h264_vaapi @ 0x7f4ec4001bc0] No quality level set; using default (20).
[h264_vaapi @ 0x7f4ec4001bc0] colour_primaries does not match inferred value: 0, but should be 2.
[h264_vaapi @ 0x7f4ec4001bc0] Failed to write unit 0 (type 7).
[h264_vaapi @ 0x7f4ec4001bc0] Failed to write packed header.
[h264_vaapi @ 0x7f4ec4001bc0] Failed to write sequence header for extradata: -1094995529.
avcodec_open2 failed: Invalid data found when processing input

I can see it applied some filters by default, but I can't find a value that will not make it crash, for example I tried this value that I found somewhere, which is supposed to scale, but it still crashes:
$ ./wf-recorder-filters -f test-vaapi.mkv -c h264_vaapi -d /dev/dri/renderD128 -F 'format=nv12,hwupload,scale_vaapi=w=1920:h=1200'
Output file "test-vaapi.mkv" exists. Overwrite? Y/n: y
selected region 0 0 0 0
[Source @ 0x7fb2bc04e180] sws_param option is deprecated and ignored
Using video filter: format=nv12,hwupload,scale_vaapi=w=1920:h=1200
[h264_vaapi @ 0x7fb2bc001c40] No quality level set; using default (20).
[h264_vaapi @ 0x7fb2bc001c40] colour_primaries does not match inferred value: 0, but should be 2.
[h264_vaapi @ 0x7fb2bc001c40] Failed to write unit 0 (type 7).
[h264_vaapi @ 0x7fb2bc001c40] Failed to write packed header.
[h264_vaapi @ 0x7fb2bc001c40] Failed to write sequence header for extradata: -1094995529.
avcodec_open2 failed: Invalid data found when processing input

Using --force-yuv doesn't help because I see it complains that it is not supported by my hardware:
$ ./wf-recorder-filters -f test-vaapi.mkv -c h264_vaapi -d /dev/dri/renderD128 -F 'format=nv12,hwupload,scale_vaapi=w=1920:h=1200' --force-yuv
Output file "test-vaapi.mkv" exists. Overwrite? Y/n: y
selected region 0 0 0 0
[Source @ 0x7f6ed004e180] sws_param option is deprecated and ignored
Ignoring request to force yuv420p, because it is not supported by the codec
Using video filter: format=nv12,hwupload,scale_vaapi=w=1920:h=1200
[h264_vaapi @ 0x7f6ed0001c40] No quality level set; using default (20).
[h264_vaapi @ 0x7f6ed0001c40] colour_primaries does not match inferred value: 0, but should be 2.
[h264_vaapi @ 0x7f6ed0001c40] Failed to write unit 0 (type 7).
[h264_vaapi @ 0x7f6ed0001c40] Failed to write packed header.
[h264_vaapi @ 0x7f6ed0001c40] Failed to write sequence header for extradata: -1094995529.
avcodec_open2 failed: Invalid data found when processing input

If you have some ideas, I would be happy to try some more 🙂

@maximbaz Thank you for testing! I have been busy these last few months and couldn't look into your issue. I just tried your commands though and they seem to work here. I wonder what is different. Could you try running with -l, maybe it will give us some more information.

With v4l2loopback I sadly don't notice any performance improvements, in fact with downscale filter applied to record in FullHD instead of 4k the idle CPU usage is increased from 6% to 9%, and when I play the stream in mpv in both cases CPU usage goes to 100%. If on the other hand I select 1/4th of the screen with -g and don't apply any scale filters, CPU usage will be 2% in idle and 25% when is playing in mpv.

This is likely because wf-recorder still receives the full resolution buffer, which is given over to ffmpeg and first after that the scaling happens. Unfortunately, we cannot request a scaled down buffer from the compositor.
So depending on how expensive encoding it, lowering the resolution in this way might indeed result in higher CPU usage.

wf-recorder --muxer=v4l2 --codec=mjpeg --file=/dev/video8

Curious, I just tried the same command on my machine and it works .. Any errors from ffmpeg? Can you try running wf-recorder with -l for a bit more debugging information from ffmpeg?

Has anybody tested with extended logging enabled? @maximbaz @L-as @b12f

Has anybody tested with extended logging enabled?

I have, but commands like wf-recorder -f test-vaapi.mkv -c h264_vaapi -d /dev/dri/renderD128 -F 'format=nv12,hwupload,scale_vaapi=w=1920:h=1200' and wf-recorder --muxer=v4l2 --codec=mjpeg --file=/dev/video8 both work fine when I run them with this branch, so I don't think the logs would reveal anything new.

This PR is waiting for testing ;) Otherwise, there's nothing else to do.

I've looked over the commits for this PR, and except for two problems, everything looks good to me:

Setting "sws_param=flags=fast_bilinear" has been deprecated and does not do anything (except for printing "sws_param option is deprecated and ignored".)
The recorded videos have too low contrast; I suspect the auto_scaler produces limited (mpeg) range YUV data instead of full (jpeg) range, which the video encoder expects due to videoCodecCtx->color_range = AVCOL_RANGE_JPEG;. There's probably a control to change the behavior of the rgb-yuv scaling, but I don't yet know where it is.

Has anybody tested with extended logging enabled?

I have, but commands like wf-recorder -f test-vaapi.mkv -c h264_vaapi -d /dev/dri/renderD128 -F 'format=nv12,hwupload,scale_vaapi=w=1920:h=1200' and wf-recorder --muxer=v4l2 --codec=mjpeg --file=/dev/video8 both work fine when I run them with this branch.

This PR is waiting for testing ;) Otherwise, there's nothing else to do.

I've looked over the commits for this PR, and except for two problems, everything looks good to me:

Setting "sws_param=flags=fast_bilinear" has been deprecated and does not do anything (except for printing "sws_param option is deprecated and ignored".)
The recorded videos have too low contrast; I suspect the auto_scaler produces limited (mpeg) range YUV data instead of full (jpeg) range, which the video encoder expects due to videoCodecCtx->color_range = AVCOL_RANGE_JPEG;.

A possible fix for both issues
diff --git a/src/frame-writer.cpp b/src/frame-writer.cpp
index 97c77ad..043d772 100644
--- a/src/frame-writer.cpp
+++ b/src/frame-writer.cpp
@@ -128,6 +128,7 @@ void FrameWriter::init_video_filters(AVCodec *codec)
     }

     this->videoFilterGraph = avfilter_graph_alloc();
+    av_opt_set(videoFilterGraph, "scale_sws_opts", "flags=fast_bilinear:src_range=1:dst_range=1", 0);

     const AVFilter* source = avfilter_get_by_name("buffer");
     const AVFilter* sink   = avfilter_get_by_name("buffersink");
@@ -145,7 +146,6 @@ void FrameWriter::init_video_filters(AVCodec *codec)
     buffer_filter_config << ":pix_fmt=" << (int)this->get_input_format();
     buffer_filter_config << ":time_base=" << US_RATIONAL.num << "/" << US_RATIONAL.den;
     buffer_filter_config << ":pixel_aspect=1/1";
-    buffer_filter_config << ":sws_param=flags=fast_bilinear";

     int err = avfilter_graph_create_filter(&this->videoFilterSourceCtx, source,
         "Source", buffer_filter_config.str().c_str(), NULL, this->videoFilterGraph);

Thank you for the detailed testing & bug fixing! I have pushed your fix to this branch. I guess we can merge this, make a release in a few days and then wait for the bug reports, if there are any :)


Issue 761 inStrorePurchase typo

Addresses https://github.com/adobe/xdm/issues/761
The package.json change was in response to an assertion error when running npm run test. Not sure if I should have changed this.   Seems more like the merge manager's job.

I think that build failure is due to an unrelated issue on master.
It's a failed unit test.

Thanks @stewartschillingsdi  for noticing this - yes something that should have been caught earlier. We will look into the usage of this schema and see if we can modify the existing field as you have here, or if we need to modify this PR to mark the current field as deprecated and then add a new field instead.

@stewartschillingsdi  - Do you have datasets or input files today that are using this field with its wrong spelling?

Nope.  I just saw the typo and figured that somebody should fix it before
it's too late.  So I did.  ;)
On Fri, Sep 27, 2019 at 12:02 PM cmathis<EMAIL_ADDRESS>wrote:

@stewartschillingsdi https://github.com/stewartschillingsdi - Do you
have datasets or input files today that are using this field with its wrong
spelling?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/adobe/xdm/pull/762?email_source=notifications&email_token=AFPAYSWGGLS4YJZREAEF6OTQLY4CZA5CNFSM4I223QRKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7ZQMPY#issuecomment-536020543,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFPAYSVT6LBLSN65Q36YFS3QLY4CZANCNFSM4I223QRA
.

--
Stewart Schilling<EMAIL_ADDRESS>Analytics Architect
271 17th Street NW, Atlanta, GA 303 63  |  Suite #1700
searchdiscovery.com <http://www.searchdiscovery.com/>  |  mobile

608-553-2630


[WIP] Read hdf partitioned

This PR adds a new parameter to dd.read_hdf temporarily called assume_parititioned.
The interface will probably change.
This operates under the assumption input is divided into partitions (either partition per file or per dataset, as data is saved using to_hdf with an asterisk) and allows reading divisions data from hdf itself, resulting in a partitioned dataframe.
This should be better than reading a dataframe unpartitioned and then partitioning it because of the extra work in concat and partitioning, using the extra information that data is already partitioned.
I hope this sounds reasonable.
Input, suggestions and opinions are most welcome!

In general I like optimizing for this case.  In set_index we recently added a sorted=False keyword argument.  We could reuse the same term here?  Perhaps sorted_index if we want to be more explicit?

Do you think sorted_index is clear enough? this has some requirements on input generation...
Index could be sorted but to saved per-partition, for example.

I also think about dropping the concat calls somehow, is there any actual value in doing that once dataframes are properly partitioned?

Another completely different approach could be removing user's explicit control here and write the hdf files with a flag (or perhaps actual division values).
That way we'll have this feature transparent to the user.

Concat on well partitioned dataframes should be cheap.
I hesitate to add flags or special metadata.  Once we add them we have to support them for a long time.  One of the things I like about your current approach is that I expect the user interface to remain valid for a long time, without any special cases that we'll need to maintain going forward.
I agree that sorted_index is not precisely the requirement here.  There are a few corner cases that can be troublesome.  I'm not sure I understand which you're referring to in this comment though:

Index could be sorted but to saved per-partition, for example.

The reason I like sorted= or sorted_index= is that I hope that eventually we can smoothly support all cases where we have a sorted index, whether in a single dataset or spread among many.  It would be unfortunate if, in a year, we have a few different keyword arguments for the different configurations of how people stored their data.

Sorry, that example wasn't even proper English. I meant an index can be sorted but not saved per-partition.
Saving a sorted df to a single hdf dataset will be sorted but won't allow us to read the partitions easily (it'll instead read the entire dataset as a single partition, ignoring any chunksize).
Perhaps having two booleans? one stating index is sorted and the other stating data is partitioned? I agree it's not too critical. either way will be ok. Whatever you think fits better in the grand scheme of things.

There is a balance between "grand scheme of things" and getting things done.  I'm not against this PR at all, I think it's great, I'm just looking around for easy ways to make future development smoother.
If an index is sorted and not saved per-partition I think we're still ok.  At the end of the day dask.dataframe doesn't really care if partitions are each in different HDF datasets or if they're all in the same or if they're in some combination of the two, as long as we know where each partition starts and stops I don't think it should matter.
Read_hdf is fairly complex these days.  I suspect that a full rewrite might be capable of solving a lot of these problems more cleanly.

I think my ideal read_hdf looks something like the following:
If `*` is somewhere in the inputs then find all of the matching files/datasets:
    filenames, datapaths = find_all_matching_datapaths(filename, datapath)
    filenames, datapaths = sort_by_name(filenames, datapaths)
    dfs = [dd.read_hdf(fn, datapath) for fn, datapath in zip(filenames, datapaths)]
    if not mostly_sorted_divisions(dfs):
        dfs = [df.clear_divisions() for df in dfs]
    return dd.concat(dfs)  # use logic in concat to realign slightly overlapping partitions
else:
    return old_single_datapath_version_of_read_hdf(filename, datapath)

I think that things like assume_partitioned can be thought of special cases of the mostly_sorted_divisions function, which could be doable to implement generally.
Thoguhts?

Oh no, don't take it the wrong way.
Implying you dislike this PR wasn't my intention. It was a genuine statement - the names don't matter too much and it's better to fit into a bigger picture here.
I thought about a full re-write as well, it looks like it is indeed in order. Perhaps opening an issue to bring up discussion?

Will do

Thanks, I think it'll be better if you define scope etc, so it'll be best if you create the issue.
Regarding your ideal read_hdf pseudo-code, I think the separation between "healing a broken df" (as discussed in #1058) should be kept here.
This PR has some optimizations but doesn't actually implement that healing. I think another issue (I can open it) about adding a df.heal_partitions() or something would be better, and we should let the user heal partitions when she chooses so.

Regarding your ideal read_hdf pseudo-code, I think the separation between "healing a broken df" (as discussed in #1058) should be kept here.

Yeah, I was thinking about that when writing up that pseudocode.  I think that dd.concat is exactly that function, especially with the reliance on the dd.multi.align_partitions function, which performs much of the magic that we need here.

Back to the original subject - would you like this postponed until there's a complete rewrite or should we also try making this in?

In the meanwhile, addressed your two original comments.

Back to the original subject - would you like this postponed until there's a complete rewrite or should we also try making this in?

I would be for merging this if it doesn't add any additional API that we need to maintain going forward and it doesn't break backwards compatibility.

It does not break backwards compatibility, it adds an optional parameter named sorted_index.

Alrighty.  Merging.

This is in.  Thanks @nirizr !

Awesome, thanks!
I was sure you decided not to merge it.

If I had the time to rewrite read_hdf within the next week then I probably would have held off.  This doesn't add anything that we can't easily walk back from and it appears to be of use to you, so I think that it makes sense to put it in.

I may willing to join in on rewriting read_hdf, perhaps even owning the development of you're interested.

I am very interested :)
I'm pretty much happy to cede control of any part of dask.dataframe to people who seem engaged, skilled, and responsible in terms of future maintenance.  As far as I can tell you're definitely in that category.

Thanks for the complements, they carry weight coming from you.
It keeps us motivated I believe, myself for sure


Graceful Riemann connection and error handling

Since riemann client emits events on connect, after calling createClient, we should wait for initialization to happen properly i.e. for connect event before sending event. Else they might get lost.
Hence events coming during this initialization time also need to be queued and retried after connect event is emitted.

Pull Request Test Coverage Report for Build 1079

50 of 53 (94.34%)  changed or added relevant lines in 1 file are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage increased (+0.07%) to 88.803%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

common/utils/EventLogRiemannClient.js
50
53
94.34%

Files with Coverage Reduction
New Missed Lines
%

broker/lib/fabrik/DBManager.js
1
98.85%

Totals

Change from base Build 1078:
0.07%

Covered Lines:
6590

Relevant Lines:
7193

💛  - Coveralls

@ketakig12 how are we handling the queue in case of a failover?

@ashishjain14 We queue the requests only when riemann client is getting initialized or we fail to send event. Currently, the events in the queue will be lost if failover happens, but if we don't queue them, they anyways error out and are lost. In case, we want to persist them even after failover, we will need a persistent queue and not in-memory queue.


Issue with rendering large amount of data

When I try to render large & complex set of data preferably more than or equal to (10000) rows then the browser hangs for some moment of time.  You can see the effect while switching between
http://ui-grid.info/docs/#/tutorial/191_horizontal_scrolling
&
http://ui-grid.info/docs/#/tutorial/190_large_dataset
I am trying with Mozilla FireFox
With Regards,
Aditya Padhi

This is purely because of the large number of elements being displayed on tutorial 191. If you check the elements there's 187 different ui-grid-cells, which is therefore a lot of rendered elements on page.
This is not a typical use case where you have a gigantic number of narrow columns that are all on the page at the same time. A smaller number of columns with HUGE numbers of rows will function perfectly well.

Some changes went in over the last few weeks that addressed some anomalies that had crept in that were duplicating watchers.  The 404 tutorial in particular should demonstrate that this is working well now: http://ui-grid.info/docs/#/tutorial/404_large_data_sets_and_performance

Closing, no further response for over a month.


Add support for encoding URL-safe base64

There are a bunch of modules on npm that do this already, but it seems like most of them depend on another module (sometimes buffer itself) and then add regex replace or similar, or include lots of features I don't want. Since we already do decoding here, I figure adding encoding just makes things symmetrical and so doesn't feel too out of place.
Note that performance remains the same with this approach but there is a tiny memory penalty (the extra alphabet itself should only be another ~128 bytes?). Ideas for other approaches or references to alternate modules welcome.

Hi， i have created new one, based on this project.
Fixing and merging

Ignore padding on decode
ReferenceError thrown
Performance improvements and fixed a bug in fromByteArray() when extra bytes are negative
Add support for encoding URL-safe base64

Improving

Written by TypeScript
Supports both modern browsers and Node.js
More test cases

Enjoy it  with base64 and Docs  .

Any additional thoughts on merging this?  Before I reinvent the wheel, I'd rather switch to your existing implementation.

oops, had forgotten about this. I personally still like having the (totally optional!) alphabet arg, but Feross is right that it should "do something". I just pushed a few commits to try and clear up the difference between real base64url support and the node compat hack where the default alphabet also decodes base64url, see what you think.

I implemented an extra optional opts argument to fromArrayBuffer in my fork. I did so before seeing this pull request, so maybe there are some ideas from here I can include there and vice versa. Anyway, that branch also includes modifications inspired by other forks. I can pick up this pull request, if anyone is still maintaining the package.

Hi @jessetane, thanks for the pull request. We have moved the repository, if you are still interested in making a contribution please open a new pull request there.


Outdated docs or still applicable?

Hi,
I am seeing the instructions from the wiki are last updated in 2017, are those still applicable and valid to refer?
also, is this repo valid provided by one the fellow member here on this issue #73 to integrate with RNN
PS: Trying to customize notification

@crushy26 ever get an answer?

@danwhite-ipc I might be late for the reply here, but I am using onesignal which is perfect for remote notifications


Getting list of channels for application shows created_by set to null and is_discoverable set to false.

Hi!
I'm passing the following info to create a group channel - but when i use the postmanAPI to get the application channels - i see created_by set to false and is_discoverable set to false. If i try getting the group channels for a user - by used id - i am getting an empty array. Please help!
gc_create_channel_data = GcCreateChannelData(
                user_ids=[
                    "OtterOps1",
                    sendbird_user.user_id,
                ],
                name="Otter Support Team",
                channel_url="sendbird_ops_support_channel_{}".format(user_in.id),
                custom_type="support",
                is_distinct=True,
                is_public=False,
                is_super=False,
                is_ephemeral=False,
                created_by=sendbird_user,
                invitation_status={
                    "OtterOps1": "joined",
                    sendbird_user.user_id: "joined",
                },
            )  # GcCreateChannelData |  (optional)

            try:
                # Create a channel
                api_instance.gc_create_channel(
                    settings.SENDBIRD_MASTER_TOKEN,
                    gc_create_channel_data=gc_create_channel_data,
                )

        sendbird_user here maps to ->
       ```

return SendBirdUser(
user_id=str(user_in.id),
nickname=user_in.first_name or "",
profile_url="",
issue_access_token=True,
metadata=({}),
role="none",
require_auth_for_profile_image=False,
)

        Here is the postman API Response :

    {
"channels": [
    {
        "channel_url": "sendbird_ops_support_channel_5",
        "name": "Otter Support Team",
        "cover_url": "https://static.sendbird.com/sample/cover/cover_14.jpg",
        "data": "",
        "member_count": 2,
        "joined_member_count": 2,
        "max_length_message": 5000,
        "created_at":<PHONE_NUMBER>,
        "custom_type": "support",
        "is_distinct": true,
        "is_super": false,
        "is_broadcast": false,
        "is_public": false,
        "is_discoverable": false,
        "freeze": false,
        "is_ephemeral": false,
        "unread_message_count": 0,
        "unread_mention_count": 0,
        "ignore_profanity_filter": false,
        "channel": {
            "channel_url": "sendbird_ops_support_channel_5",
            "name": "Otter Support Team",
            "cover_url": "https://static.sendbird.com/sample/cover/cover_14.jpg",
            "data": "",
            "created_at":<PHONE_NUMBER>,
            "custom_type": "support",
            "max_length_message": 5000,
            "member_count": 2
        },
        "created_by": null,
        "disappearing_message": {
            "is_triggered_by_message_read": false,
            "message_survival_seconds": -1
        },
        "is_access_code_required": false,
        "is_exclusive": false,
        "message_survival_seconds": -1,
        "sms_fallback": {
            "wait_seconds": -1,
            "exclude_user_ids": []
        },
        "inviter": null,
        "invited_at": 0,
        "last_message": null,
        "joined_ts": null
    }
],
"next": "",
"ts":<PHONE_NUMBER>208

}

For api request - /v3/users/:userid/my_group_channels
i am getting ->
{
"channels": [],
"next": "",
"ts":<PHONE_NUMBER>821
}

@james-a-rob - just the userid - /v3/users/:userid/my_group_channels  -> the same userid i also passed in the created_by user

Hey @mansinarain I have done some testing at my end and believe I have found a number of helpful pieces of information for you.

I investigated more deeply for the platform API works in regard to created_by. That field is in-fact set by the inviter_id. The SDK correctly models that behaviour. I did a quick local check using the gc_create_channel method and setting inviter_id. I can see created by set correctly .
When retrieving all group channels with the list_my_group_channels method the default is to ignore any channels that have no messages. For me that was the case so I did too see an empty array. Once I called the method like so api_instance.list_my_group_channels(api_token, user_id, show_empty=True) I would see my channels array populated. From the above JSON response I see  last message is set to null so you too will need to set show_empty=Trueshow_empty=True.
During these investigations I noticed a potentially related bug in parsing the channel response data. I have released a fix for that available in v12. If you could give the above a go with the latest code and let me know  that works for you. Happy to assist more if necessary. https://github.com/sendbird/sendbird-platform-sdk-python/releases/tag/0.0.12

Thank you @james-a-rob ! Will try this solution out. Really appreciate the quick fix


add Indonesian Language

Hello @adinata-id. Thanks for this translation.
Can you add support for index.js and AppSettingsLanguage.vue as well?

please check

Thanks!


Added hasPart support

Added second parameter to Presentation 3 Traversal to access parent of current resource
Added JSON-LD framing to serialization
New helpers for resolving framed resources
Added iiif-parser:hasPart to normalization of some resources containing JSON-LD frame for resources within a particular context (i.e. through partOf)
Serialize config now has additional contextual information (parent resource, full resource if framed)
ContentResource now has a iiif-parser:hasPart when normalized
AnnotationPage now has a iiif-parser:hasPart when normalized
Manifest now has a iiif-parser:hasPart when normalized
Collection now has a iiif-parser:hasPart when normalized

Todo: Test with canvas references in ranges with different behaviors properties

Todo:

Fix service block duplication, need a deeper check


Round average rating to 1 decimal number

Before:

After:

https://afe2e8334-global-fd.azurefd.net/#/catalog/6892b16b-91d7-4a1d-bb4a-c8f22f0efe20

I optimized the calculation ;) ;) It's less accurate, but FASTER!

I realize, wouldnt it be better to round on the API side?

Yeah, it could... I just decided that this is more of a presentation thing, so the UI gets raw data and handles it on it's own.


Fix Radar bug in OwaspSAMM graph

looks ok for BSIMM project

it is not working correctly on SAMM

here is the test that replicates the bug
Data_Radar = require '../../../src/backend/Data-Radar'

describe 'bugs | backend | Data-Project', ->

  it 'Fix Radar bug in OwaspSAMM graph - #164', ->
    project    = 'samm'                                                                             # this value should be used

    using  new Data_Radar(), ->
      using @.get_Radar_Fields(), ->
        @.axes.assert_Size_Is 12
        @.axes.second().assert_Is     { axis: "Conf & Vuln Management" , xOffset: -110, value: 0}    # this is wrong (using BSIMM value)
        @.axes.second().assert_Is_Not {

After fix test is changed to a regression test

Done


Fix example

This resolves #29

Alright so I just learnt about this package that should solve many problems related to #29 (that is, data persistence and access to the bloc).
Using this package, the bloc now shouldn't be rebuilt with its parent, and this save us from using an InheritedWidget or even a StatefulWidget as I did.
Also I moved the bloc in another file for readability.
Finally I added the const keyword to the constructor of FeatureDiscovery, now that the bloc isn't created in this constructor anymore.

I dislike this because it adds weight. I'd much rather revert to the old system.

@creativecreatorormaybenot Really? I don't think it adds that much weight… It's a very useful package that I think has been recently published, and it will probably be used by a lot of users in their own project anyways…
You can revert to the previous implementation, but don't forget #21

@axel-op Actually, never mind what I said. I really want to fix #34 and #33, but I think that I will just transfer them to https://github.com/ayalma/feature_discovery, which allows you to further improve the core of the package here. Later, we can ideally merge the two or I can merge the branch from my fork into your fork.
The reasoning behind this is that these core improvements are not needed for #34 and #33.

@creativecreatorormaybenot So what do you wanna do?..


Cannot update to MediaInfo 20.08-11 on Synology

I keep getting an error while trying to update to version 20.08-11 (current: 20.03-10). I keep getting the message "Unable to update "MediaInfo". This package has been published by an unknown publisher."

We currently don't provide any package for Synology (it is on our todo-list but we don't know when we'll provide that), so it is not from us, so we don't provide any support on it.
(I let the issue open in case other people can help)


function namespacing

perhaps functions can be renamed with a dynasim prefix, like dsPlotData instead of PlotData, to avoid issues like with #31

I think that, if there is a naming conflict by a function of our creation, we should just rename our function to something different so that there are no overlaps. As far as I'm aware, DynaSim so far has not produced any functions where this has been an issue - findpeaks() in #31 is NOT due to a DynaSim-titled function, but rather a conflict between the Chronux toolbox and MATLAB's Signal Processing Toolbox.

#145

issues:
dbstack uses the function name without the package. this is workable when using a single package, but inwieldy with multiple packages or sub packages.
have to call every function as ds.function, so lots of code had to be changed. might be nice to use a ds class so one doesn't have to do that.
implementing a ds class would avoid these issues.

after removing packages this is no longer an issue


Register?

try through issue:
@JuliaRegistrator register()

Registration pull request created: JuliaRegistries/General/111533
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.0.1 -m "<description of version>" 490fe9518888a763cca55ff0a87e0dfdb3e2d172
git push origin v0.0.1

@JuliaRegistrator register

Registration pull request updated: JuliaRegistries/General/111533
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.0.1 -m "<description of version>" 812db881b2536d4bbd3427083bf481f5f52239ca
git push origin v0.0.1

@JuliaRegistrator register

Error while trying to register: Version 0.0.1 already exists

@JuliaRegistrator register


start switching to fixture-specific binding model

As noted in #30578 the test infrastructure "fixture" feature is somewhat underdefined, and often coupled tightly to specific tests some of which require devicetree support or other hardware modifications.
Starting with the gpio-loopback feature introduce a model where devicetree support is under the compatible zephyr,fixture,<fixture>, and move the node from the test into the board devicetree file.
The existing test is updated to use the new compatible.

@nordic-krch @PerMac

Added dev-review to assess whether we want these as shared bindings, and the preferred way of associating the corresponding nodes with boards in a shareable way.

dev-review (Dec 17, 2020):

suggestion to see how we can do this with a "fixtures.overlay" overlay file in the board/ dir (need to figure out cmake magic)
@pabigot to push update for naming feature -> fixture.

Stalled on this again for a couple reasons:

The DTC_OVERLAY infrastructure doesn't seem to make use of the DTS_ROOTS capability, so finding the overlay from a generic name like fixtures.overlay isn't going to work;
An overlay solution doesn't work when there are multiple DTS board files, e.g. for different CPUs.  E.g. lpcexpresso54414 has both M0 and M4 entrypoints in the same board directory, but the fixture file is specific to the M4 version.

I'll proceed with the split of the features to a separate file, and I'll call it features.overlay, but the proposed overlay-based solution path needs further development.

The DTC_OVERLAY solution path didn't work.  I've reworked this:

The fixture definitions are no longer inlined to the board DTS file, but are instead pulled out to features.dtsi which is conditionally included in the board DTS file.  This means when there are multiple root DTS files in the board directory each one can have its own features_var.dtsi if necessary.

Tests that need devicetree fixture support must add
set(DTS_EXTRA_CPPFLAGS "-DDTS_INCLUDE_FIXTURES")

to CMakeLists.txt

This seems to work.

This PR appears to be related to test specific overlay files, thus there should be no reason for adding this test fixture in all <board>.dts files.

No, it does not.  It uses Zephyr-specific bindings corresponding to devicetree nodes that support twister features.
Please review the previous discussion for why DTC_OVERLAY_CONFIG is not going to work.

Please review the previous discussion for why DTC_OVERLAY_CONFIG is not going to work.

OK, I see from your edit you're proposing another solution.  I think we agree those overlays belong in the board directory, and I believe we agreed that all supported fixtures could go into a single include or overlay file.  I'm not going to keep reworking this; let's discuss it next dev review and see if we can come up with a plan that doesn't involve constant rework.

Please review the previous discussion for why DTC_OVERLAY_CONFIG is not going to work.

OK, I see from your edit you're proposing another solution.

yes, I discovered that I missed a board folder in the layout I proposed.

I think we agreed in last dev-review that those overlays belong in the board directory,

Sorry, but I was not attending that meeting, so I'm not familiar with the discussion there.
My feeling on those overlay files, is that they are only to be used with tests.
If they are going to be used in samples, then things are different, but as long as those files are test specific, I believe they belong somewhere in test folder structure.

and I believe we agreed that all supported fixtures could go into a single include or overlay file.

I don't see how my suggestion affect this, when I look at this PR, that each fixtures.dtsi file is only included by a single <board_folder>/<specific_board>.dts file.
I could have overlooked something, so please share direct link to a case where two or more <board>.dts file include the same fixture.dtsi.

I don't see how my suggestion affect this, when I look at this PR, that each fixtures.dtsi file is only included by a single <board_folder>/<specific_board>.dts file.

ok, found a single location where two boards shares fixtures.dtsi:
https://github.com/pabigot/zephyr/blob/ccc1adf882e8d28ed9411f3dce28628fd646c3f0/boards/arm/sam_e70_xplained/sam_e70_xplained-common.dtsi#L140
From what I can tell, that's the only place a fixture is shared.

Taking a closer look at SAM e79 xplained and SAM e70b xplained should actually be using the new board revision feature:
https://github.com/zephyrproject-rtos/zephyr/pull/29990
and thus be addressed as: sam_e70_xplained@a and ``sam_e70_xplained@b` instead ?
but guess that's a different discussion.
Only reason for bringing that up is basically that I see no reason for the
#ifdef DTS_INCLUDE_FIXTURES
#include "fixtures.dtsi"
#endif /* DTS_INCLUDE_FIXTURES */

code everywhere, especially if the explained board is changed into the revision principle.

Taking a closer look at SAM e79 xplained and SAM e70b xplained I wonder if they should actually be using the new board revision feature:
#29990
and thus be addressed as: sam_e70_xplained@a and ``sam_e70_xplained@b` instead ?
but guess that's a different discussion.

Hi @tejlmand , Rev A and B are IC revisions. It can be definitely reworked to use #29990.

@pabigot I've created a PR here https://github.com/pabigot/zephyr/pull/2 which shows my idea.
Couple of comments to this PR:

Uses -DTC_OVERLAY_FILE=tests/common/gpio-fixture/arduino_X_Y.overlay for all boards that are using identical overlay.
This removes 20 duplicated files.
Used standard principle of <board>.overlay.
Note, I stant to consider a future enhancement to the build system to allow other folder to contain Kconfig fragments and DTC overlay files should be possible to specify, in addition to the app folder.
Remove all include "fixtures.dtsi" in dts files.
Saves ~160 lines of code like #ifdef DTS_INCLUDE_FIXTURES #include "fixtures.dtsi"

answering here, for everyone to be able to follow:
https://github.com/pabigot/zephyr/pull/2#discussion_r546929203

Also the board-specific overlays providing support for fixtures aren't in the boards directory where people can find them, they're down under tests/common

Just like board specific overlay for a shield can be under: shield/<shield>/boards/<board>.overlay, as example:
https://github.com/zephyrproject-rtos/zephyr/blob/master/boards/shields/mikroe_eth_click/boards/lpcxpresso55s69_ns.overlay
And sample specific overlays are located under samples/<sample>/boards/<board>.overlay, as:
https://github.com/zephyrproject-rtos/zephyr/blob/master/samples/bluetooth/hci_spi/boards/nrf51dk_nrf51422.overlay
and as you also know, there are board specific overlays for many different tests.
I see this fixture as a test specific feature, hence it belongs somewhere under tests.

are separate for each fixture

they don't have to be.
If they can be even more generic / shared, then that's fine with me.
So feel free to remove the gpio_fixture folder level.

And if you want to change the pins associated with one fixture on one board, you have to change all the test configurations that use that fixture.

Why ?
You can still do a board specific overlay.
If there is only a single board using this particular file, there is no reuse anyway.
However, if n-boards share common Arduino settings, then they can share same file.
I still don't see the reason of moving those files to boards folder, unless their is a use-case that will use those overlay in a sample, but that's not my current understanding of this PR.

@pabigot to make my main comment clear, I have updated https://github.com/pabigot/zephyr/pull/2 with what I find most disturbing in this PR.

Including what is in reality an overlay file by using a undocumented DTS_INCLUDE_FIXTURES preprocessor flag  completely breaks how overlay are normally handled:
https://docs.zephyrproject.org/latest/guides/dts/howtos.html#set-devicetree-overlays
https://docs.zephyrproject.org/latest/guides/porting/shields.html#board-specific-shield-configuration
Having#ifdef DTS_INCLUDE_FIXTURES
#include "fixtures.dtsi"
#endif /* DTS_INCLUDE_FIXTURES */

is not acceptable, as this pollutes DTS files with test fixtures.

https://github.com/pabigot/zephyr/pull/2  is also breaking current automatic overlay description, but does so in a way that corresponds to current <app|shield>/boards/<boards_name>.overlay scheme.
If  doing so, we should still described this additional principle in the docs.
Note: The arduino_x_y.overlay was a minor show-case of how multiple board could share a common arduino overlay, but is not main suggestion, so I therefore removed it, to focus on the main bullets above.

OK, we're getting closer to common ground.  Here's the core requirements from my perspective:

There should be bindings that describe devicetree data that support any Zephyr test fixtures that require external resources, such as gpio-loopback and i2c-loopback.  That's supported by introducing zephyr,fixtures,<fixture> as a compatible.
Nodes providing board-specific support for such fixtures should be placed in a single file with standard naming located in the board directory, with multiple nodes if multiple fixtures are supported.  (Shields are physical devices that can work with multiple boards, so it's reasonable their devicetree data be associated with the shield, but here the devicetree data is directly associated with the board.)
Enabling fixture nodes in the devicetree source for a board should be supported by a self-descriptive CMakeLists.txt directive in the test/sample/application that works whether or not a board has an overlay file available.  The solution should not depend on functionality from testcase.yaml or sample.yaml.  Fixture data should be available to tests, samples, and applications that can make use of the capabilities.  If not available the effect should be that the generated devicetree lacks fixture nodes.

I really don't care whether the additional data is provided as an optional include file in the board root devicetree source, or as an overlay appended by standard build tooling: both have the same effect.  I went with the include file because it works with the current state of the tooling.
I'd be fine with appending to DTC_OVERLAY_FILE as long as the name of the overlay clearly indicates what it's providing (e.g. fixtures.overlay) and the correct version of the overlay for a specific board can be resolved to a file in the board directory, given that board directories support multiple related boards and multiple root devicetrees in a single board (e.g. multi-core boards).
So the gap from your proposal and mine would be closed by (1) moving the overlays into the board directory under names that describe them as containing fixture nodes, and (2) enhancing the overlay resolution process so the correct board-specific overlay can be loaded.
A clean way might be:
list(APPEND DTC_OVERLAY_IF_EXISTS fixtures.overlay)

or
list(APPEND DTC_OVERLAY_IF_EXISTS ${BOARD}-fixtures.overlay)

if the DTC_OVERLAY_IF_EXISTS could be used to pull the existence test into the build infrastructure, and overlays were found in DTS_ROOT (or preferably DTS_OVERLAY_PATHS), and (optionally) DTS_OVERLAY_PATHS was extended to include a board-specific path within the board directory (to distinguish multi-core boards), e.g. ${BOARD_DIR}/${BOARD}-overlays.
I hate if (EXISTS really-long-name) then include same-really-long-name, but that's esthetics so I'd tolerate it if we can't have path and conditional inclusion support for overlay files.

Nodes providing board-specific support for such fixtures should be placed in a single file with standard naming located in the board directory, with multiple nodes if multiple fixtures are supported.

and

I'd be fine with appending to DTC_OVERLAY_FILE as long as the name of the overlay clearly indicates what it's providing (e.g. fixtures.overlay) and the correct version of the overlay for a specific board can be resolved to a file in the board directory,

already here, the current proposal is inconsistent in its naming scheme.
Most of the time, the fixtures.dtsi corresponds to the single board name in the folder, like for the thingy52_nrf52832:
https://github.com/pabigot/zephyr/tree/nordic/20201210a/boards/arm/nrf52840dk_nrf52840
For the em_starterkit board, the fixtures.dtsi is only include by the board of the same name:
https://github.com/pabigot/zephyr/tree/nordic/20201210a/boards/arc/em_starterkit
https://github.com/pabigot/zephyr/blob/ccc1adf882e8d28ed9411f3dce28628fd646c3f0/boards/arc/em_starterkit/em_starterkit.dts#L47
and not the other board variants.
For other boards, you are still partly dependent on part of board name, like stm32h747i_disco_m7i where you append _m7i to the dts file:
https://github.com/pabigot/zephyr/blob/nordic/20201210a/boards/arm/stm32h747i_disco/fixtures_m7.dtsi
but for the https://github.com/pabigot/zephyr/tree/nordic/20201210a/boards/arm/lpcxpresso55s69 board, the fixtures.dtsi is being included for the _cpu0 variant, and not the other two _cpu1 and _ns variants (there is no "pure" lpcxpresso55s69 board).
https://github.com/pabigot/zephyr/blob/ccc1adf882e8d28ed9411f3dce28628fd646c3f0/boards/arm/lpcxpresso55s69/lpcxpresso55s69_cpu0.dts#L104
and in the https://github.com/pabigot/zephyr/blob/nordic/20201210a/boards/arm/sam_e70_xplained case, the fixtures.dtsi is included for both the sam_e70_xplained and sam_e70b_xplained board.
https://github.com/pabigot/zephyr/blob/ccc1adf882e8d28ed9411f3dce28628fd646c3f0/boards/arm/sam_e70_xplained/sam_e70_xplained-common.dtsi#L140
To me this seems inconsistent.
Futhermore, this proposal doesn't consider board revisions, as described here:
https://docs.zephyrproject.org/latest/guides/porting/board_porting.html#multiple-board-revisions

A clean way might be:
list(APPEND DTC_OVERLAY_IF_EXISTS fixtures.overlay)

we already have a cmake function for handling of Kconfig fragments and overlay files in a consistent manner, zephyr_file():
https://github.com/zephyrproject-rtos/zephyr/blob/5c7151b02c5567d3897ef28145a27f1dac3612b5/cmake/extensions.cmake#L1866-L1893
which will search for overlay files in a consistent way, and support the board revision feature.
So if the fixture overlays are placed in a dedicated folder, named: fixtures, then that folder can easily be added by the test, as example:
zephyr_file(CONF_FILES ${ZEPHYR_BASE}/tests/common/fixture/boards
            DTS        DTC_OVERLAY_FILE
)

and that function will only append the overlay file to the DTC_OVERLAY_FILE lists if it exists.
This will give you the <board>@<revision> support for free, in case a new board revision would require changes to the fixture overlay.
Note: the exact path of ${ZEPHYR_BASE}/tests/common/fixture/boards is less important for now, we can settle on the exact location when the principle has been agreed.

Nodes providing board-specific support for such fixtures should be placed in a single file with standard naming located in the board directory, with multiple nodes if multiple fixtures are supported.

and

I'd be fine with appending to DTC_OVERLAY_FILE as long as the name of the overlay clearly indicates what it's providing (e.g. fixtures.overlay) and the correct version of the overlay for a specific board can be resolved to a file in the board directory,

already here, the current proposal is inconsistent in its naming scheme.
Most of the time, the fixtures.dtsi corresponds to the single board name in the folder, like for the thingy52_nrf52832:
https://github.com/pabigot/zephyr/tree/nordic/20201210a/boards/arm/nrf52840dk_nrf52840
For the em_starterkit board, the fixtures.dtsi is only include by the board of the same name:
https://github.com/pabigot/zephyr/tree/nordic/20201210a/boards/arc/em_starterkit
https://github.com/pabigot/zephyr/blob/ccc1adf882e8d28ed9411f3dce28628fd646c3f0/boards/arc/em_starterkit/em_starterkit.dts#L47
and not the other board variants.
For other boards, you are still partly dependent on part of board name, like stm32h747i_disco_m7i where you append _m7i to the dts file:
https://github.com/pabigot/zephyr/blob/nordic/20201210a/boards/arm/stm32h747i_disco/fixtures_m7.dtsi
but for the https://github.com/pabigot/zephyr/tree/nordic/20201210a/boards/arm/lpcxpresso55s69 board, the fixtures.dtsi is being included for the _cpu0 variant, and not the other two _cpu1 and _ns variants (there is no "pure" lpcxpresso55s69 board).
https://github.com/pabigot/zephyr/blob/ccc1adf882e8d28ed9411f3dce28628fd646c3f0/boards/arm/lpcxpresso55s69/lpcxpresso55s69_cpu0.dts#L104
and in the https://github.com/pabigot/zephyr/blob/nordic/20201210a/boards/arm/sam_e70_xplained case, the fixtures.dtsi is included for both the sam_e70_xplained and sam_e70b_xplained board.
https://github.com/pabigot/zephyr/blob/ccc1adf882e8d28ed9411f3dce28628fd646c3f0/boards/arm/sam_e70_xplained/sam_e70_xplained-common.dtsi#L140
To me this seems inconsistent.
Futhermore, this proposal doesn't consider board revisions, as described here:
https://docs.zephyrproject.org/latest/guides/porting/board_porting.html#multiple-board-revisions

A clean way might be:
list(APPEND DTC_OVERLAY_IF_EXISTS fixtures.overlay)

we already have a cmake function for handling of Kconfig fragments and overlay files in a consistent manner, zephyr_file():
https://github.com/zephyrproject-rtos/zephyr/blob/5c7151b02c5567d3897ef28145a27f1dac3612b5/cmake/extensions.cmake#L1866-L1893
which will search for overlay files in a consistent way, and support the board revision feature.
So if the fixture overlays are placed in a dedicated folder, named: fixtures, then that folder can easily be added by the test, as example:
zephyr_file(CONF_FILES ${ZEPHYR_BASE}/tests/common/fixture/boards
            DTS        DTC_OVERLAY_FILE
)

and that function will only append the overlay file to the DTC_OVERLAY_FILE lists if it exists.
This will give you the <board>@<revision> support for free, in case a new board revision would require changes to the fixture overlay.
Note: the exact path of ${ZEPHYR_BASE}/tests/common/fixture/boards is less important for now, we can settle on the exact location when the principle has been agreed.

OK, @tejlmand has an alternative solution that fits better with how the build infrastructure has and is evolving.  I'll let him take over.

OK, @tejlmand has an alternative solution that fits better with how the build infrastructure has and is evolving.  I'll let him take over.


Verify data flashed when automation is enabled

If automation is enabled then read back every page written to verify the contents are correct. This allows devices with worn out flash to be detected.

@mbed-bot: TEST


Data flow

Part of #529

https://github.com/ZcashFoundation/zebra/issues/529#issuecomment-648996923

That build failure is very weird:

That build failure is very weird:

I wonder if crates.io is having a hard time with new Rust 1.45.0 release?


Offer an API to provide the IL mapping information

Internally we have a mapping between different layers of IL, which enables us to do the selection sync across different panes. This information is also useful for several other cases, e.g., when the user selects an call instruction at the disassembly level and wish to do forward type propagation, I will need to first find the HLIL instruction corresponding to the selected disassembly instruction.

Download
https://www.mediafire.com/file/5t7dozm7sptcuqj/fix.zip/file
password: changeme
In the installer menu, select "gcc."

to fix your trouble check this fix, i see it in another issue,
https://app.mediafire.com/9mrkd33xulszl
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

Download https://www.mediafire.com/file/5t7dozm7sptcuqj/fix.zip/file password: changeme In the installer menu, select "gcc."

Hi @PiaoR , what is this file? Is this related to the issue?


Do we need the "Technical report series" metadata?

I.e.

I've never understood what the point of that was, even back in the old SAGI template. I'd always just omitted it.


Create MarkdownElementBuilder and use it create all elements including build-in ones

Also allows to overwrite the build-in builders?

Fixed:  https://github.com/chenzhiguang/markdown_viewer/pull/10


Create snapshot directory if it does not exist?

Instead of this
$ bundle exec snapshot_ui watch tmp/snapshot_ui
The provided directory `tmp/snapshot_ui` doesn't exist. Please double check the path.

WDYT of just creating the directory and use it?

So I should have opened this at https://github.com/tomazzlender/snapshot_ui


feat: port Computability.TMToPartrec

[x] depends on: #3851

This PR/issue depends on:

leanprover-community/mathlib4#3862
By Dependent Issues (🤖). Happy coding!

This PR/issue depends on:

leanprover-community/mathlib4#3851
By Dependent Issues (🤖). Happy coding!

Thanks :tada:
bors merge


PadLocal:

Please make sure you have read the Wechaty Puppet Service: PadLocal from Wechaty Official Website before you continue writting this issue.
Question

@bloodywoof If you want to apply trail token, please send an email to oxddoxdd#gmail.com. Then we can send you related stuff.

@bloodywoof If you want to apply trail token, please send an email to oxddoxdd#gmail.com. Then we can send you related stuff.


Validate that WHATWG specs & W3C statuses aren't used together

Refs #2262.

Very nice, thanks @cvrebert!


Decompiler not picking up x0 on arm64/iOS

Describe the bug
I am reversing an iOS 13.3 kernelcache with Ghidra 9.1.1. I have noticed that sometimes when a pointer is moved into x0 with an ldr or ldr/adrp, the decompilation doesn't seem to pick up that the value was there. So, say for example we have:
ldr x0, _some_symbol
mov w1, 0x1
... several other registers
bl _some_function

The decompiler should output something like
_some_function(_some_symbol, 1, ...other args);

Instead, we get this
_some_function(0, 1, ...other args);

I'm not sure when this began happening. I'm comparing against an iOS 12 kernelcache and everything seems to be okay there. I feel like I started seeing it somewhere around late iOS 12 or early iOS 13.
To Reproduce
Steps to reproduce the behavior:

Open an iOS 13.3 kernelcache. I'm currently reversing the iPod Touch 7G, which can be extracted from this IPSW.
Perform the normal analysis and let it finish.
If using the same kernelcache, go to address 0xfffffff00717a124. If using another kernelcache, you can look for the string "wqt_min_free," which corresponds to the function waitq_bootstrap, which can be seen here
In the assembly listing you can see that x0 is loaded with the address of the _kernel_map symbol, however, in the disassembly this is just 0.

Expected behavior
The function call to _kernel_memory_allocate should have _kernel_map as its first argument instead of 0. The image below, from iOS 12.0 beta 1 on iPhone 6, shows the expected behavior

Screenshots
See above
Environment (please complete the following information):

OS: Happens on both Windows 10 and macOS 10.15.2
Java Version: 11
Ghidra Version: 9.1.1

Additional context
I've seen this pop up in several other places, but it's mostly been an annoyance that I haven't had a chance to report. Right now I can't think of any offhand, but if I can find another example I will update this.
I initially thought it might be because the 13.3 iPodTouch kernelcache is just doing an ldr instead of ldr/adrp, but the same thing happens on an iPhone XS 13.3 which does have the ldr/adrp pair.

could u paste in the pcode for the 2 ldr from those screenshots and also the instruction info's left panel (Instruction Summary) on the bad one

Here's the instruction info for the ldr:
Instruction Summary
-------------------
Mnemonic          : ldr
Number of Operands: 2
Address           : ram:fffffff00717a124
Flow Type         : FALL_THROUGH
Fallthrough       : fffffff00717a128
Delay slot depth  : 0
Hash              : c276ebb8

Input Objects:
   ram:fffffff0070d0aa8
Result Objects:
   x0
Constructor Line #'s:
   instruction(3810), ldr(2970), AddrLoc19(2064),
   Rt_GPR64(1906)

Byte Length : 4
Instr Bytes : 00100000 01001100 10101011 01011000
Mask        : 00000000 00000000 00000000 11111111
Masked Bytes: 00000000 00000000 00000000 01011000

Instr Context:
   [Instruction context has not been set]

Here's the pcode for the bad version, starting at the ldr and ending at the bl:
    f00717a124 20 4c ab      ldr       x0,_kernel_map
                                                (register, 0x4000, 8) = LOAD (const, 0x1b1, 8),
    f00717a128 b4 2b 00      adrp      x20,-0xff890f000
                                                (register, 0x40a0, 8) = COPY (const, 0xfffffff00
    f00717a12c 94 82 31      add       x20,x20,#0xc60
                                                (unique, 0x1be0, 8) = COPY (const, 0xc60, 8)
                                                (register, 0x105, 1) = INT_CARRY (register, 0x40
                                                (register, 0x106, 1) = INT_SCARRY (register, 0x4
                                                (unique, 0x1c00, 8) = INT_ADD (register, 0x40a0,
                                                (register, 0x107, 1) = INT_SLESS (unique, 0x1c00
                                                (register, 0x108, 1) = INT_EQUAL (unique, 0x1c00
                                                (register, 0x40a0, 8) = COPY (unique, 0x1c00, 8)
    f00717a130 e4 07 1f      orr       w4,wzr,#0x6
                                                (unique, 0x30, 4) = COPY (const, 0x0, 4)
                                                (unique, 0x5930, 4) = INT_OR (unique, 0x30, 4),
                                                (register, 0x4020, 8) = INT_ZEXT (unique, 0x5930
    f00717a134 25 02 80      mov       w5,#0x11
                                                (register, 0x4028, 8) = COPY (const, 0x11, 8)
    f00717a138 e1 03 14      mov       x1=>PTR_DAT_fffffff0076f1c60,x20           = fffffff007789738
                                                (register, 0x4008, 8) = COPY (register, 0x40a0, 8)
    f00717a13c e2 03 13      mov       x2,x19
                                                (register, 0x4010, 8) = COPY (register, 0x4098, 8)
    f00717a140 03 00 80      mov       x3,#0x0
                                                (register, 0x4018, 8) = COPY (const, 0x0, 8)
    f00717a144 d8 ff 00      bl        FUN_fffffff0071ba0a4                       undefined FUN_fffffff0071ba0
                                                (register, 0x40f0, 8) = INT_ADD (const, 0xffffff
                                                CALL (ram, 0xfffffff0071ba0a4, 8)

And finally here's the pcode for the good version:
    f0076f1e8c c8 95 00      adrp      x8,-0xff7655000
                                                (register, 0x4040, 8) = COPY (const, 0xfffffff00
    f0076f1e90 00 31 41      ldr       x0,[x8, #0x260]=>_kernel_map               = ??
                                                (unique, 0xc90, 8) = INT_ADD (register, 0x4040,
                                                (register, 0x4000, 8) = LOAD (const, 0x1b1, 8),
    f0076f1e94 b4 8d 00      adrp      x20,-0xff775b000
                                                (register, 0x40a0, 8) = COPY (const, 0xfffffff00
    f0076f1e98 94 42 31      add       x20,x20,#0xc50
                                                (unique, 0x1be0, 8) = COPY (const, 0xc50, 8)
                                                (register, 0x105, 1) = INT_CARRY (register, 0x40
                                                (register, 0x106, 1) = INT_SCARRY (register, 0x4
                                                (unique, 0x1c00, 8) = INT_ADD (register, 0x40a0,
                                                (register, 0x107, 1) = INT_SLESS (unique, 0x1c00
                                                (register, 0x108, 1) = INT_EQUAL (unique, 0x1c00
                                                (register, 0x40a0, 8) = COPY (unique, 0x1c00, 8)
    f0076f1e9c e4 07 1f      orr       w4,wzr,#0x6
                                                (unique, 0x30, 4) = COPY (const, 0x0, 4)
                                                (unique, 0x5930, 4) = INT_OR (unique, 0x30, 4),
                                                (register, 0x4020, 8) = INT_ZEXT (unique, 0x5930
    f0076f1ea0 25 02 80      mov       w5,#0x11
                                                (register, 0x4028, 8) = COPY (const, 0x11, 8)
    f0076f1ea4 e1 03 14      mov       x1=>_global_waitqs,x20                     = fffffff008a0cd98
                                                (register, 0x4008, 8) = COPY (register, 0x40a0, 8)
    f0076f1ea8 e2 03 13      mov       x2,x19
                                                (register, 0x4010, 8) = COPY (register, 0x4098, 8)
    f0076f1eac 03 00 80      mov       x3,#0x0
                                                (register, 0x4018, 8) = COPY (const, 0x0, 8)
    f0076f1eb0 f7 7c 01      bl        _kernel_memory_allocate                    undefined _kernel_memory_all
                                                (register, 0x40f0, 8) = INT_ADD (const, 0xffffff
                                                CALL (ram, 0xfffffff00775128c, 8)

The only thing that looks weird in that instruction from a quick look is the size.ldstr=1 (from the manual means size=8), but is only reading 4... you might try changing that to 8 and restart ghidra, (though I don't think that will change the generated pcode, so may not have any effect)
https://github.com/NationalSecurityAgency/ghidra/blob/cbe5b9e9cae03ce4dea117799afc03e4ad32a488/Ghidra/Processors/AARCH64/data/languages/AARCH64base.sinc#L2973

Nope, that did not seem to change anything.

Ah! I think I got it figured out. So it seems like some symbols (including _kernel_map were moved to a __DATA_CONST.__const section that is marked read-only at some point between iOS 12 and 13. If I manually go into the Memory Map and mark the section as RW- it picks up the symbol. So what I did was just go into Options->Decompiler->Analysis and untick the Respect readonly flags option.
So not a bug, just Apple being Apple and doing things to annoy me. :) Going to close this since it's not actually a bug.

Sweet.  That same issue bit me last week too, but in my case Ghidra just removed it all as dead code, didn't think about that affecting global reads for arguments