TestIngressGateway503DuringRuleChange fails

https://github.com/istio/istio/pull/5417 causes this test to fail. It appears that this code exposes a bug in createGatewayHTTPFilterChainOpts in gateway.go, since for the last rule set in the test, no filter opts are returned for a valid port 80 listener. However, it's not clear why adding mixer filter should cause the test to fail because the size of the listener filters is set even before the mixer filter plugin is called.
buildGatewayListeners: gateways after merging: &{map[istio-ingressgateway.istio-system.svc.cluster.local:true] map[80:[port:<number:80 protocol:"HTTP" name:"http" > hosts:"uk.bookinfo.com" ] 443:[port:<number:443 protocol:"HTTPS" name:"https" > hosts:"uk.bookinfo.com" tls:<mode:SIMPLE server_certificate:"/etc/certs/cert-chain.pem" private_key:"/etc/certs/key.pem" ca_certificates:"/etc/certs/root-cert.pem" > ]]}
2018-05-16T22:39:48.738923Z	info	OnOutboundListener HTTP
2018-05-16T22:39:48.739022Z	warn	FilterChains are empty!

omitting listener "http_<IP_ADDRESS>_80" due to: must have more than 0 chains, and the same number of chains in each of:
listener: 0; &v2.Listener{Name:"http_<IP_ADDRESS>_80", Address:core.Address{Address:(*core.Address_SocketAddress)(0xc420d0a110)}, FilterChains:[]listener.FilterChain{}, UseOriginalDst:nil, PerConnectionBufferLimitBytes:nil, Metadata:(*core.Metadata)(nil), DeprecatedV1:(*v2.Listener_DeprecatedV1)(nil), DrainType:0, ListenerFilters:[]listener.ListenerFilter(nil), Transparent:nil, Freebind:nil, TcpFastOpenQueueLength:nil}
opts: 0; v1alpha3.buildListenerOpts{env:model.Environment{ServiceDiscovery:(*aggregate.Controller)(0xc4204001e0), ServiceAccounts:(*aggregate.Controller)(0xc4204001e0), IstioConfigStore:(*model.istioConfigStore)(0xc42081ea10), Mesh:(*v1alpha1.MeshConfig)(0xc420317800), MixerSAN:[]string(nil)}, proxy:model.Proxy{ClusterID:"Kubernetes", Type:"router", IPAddress:"<IP_ADDRESS>", ID:"istio-ingressgateway-bb8766f79-68ktm.istio-system", Domain:"istio-system.svc.cluster.local", Metadata:map[string]string{"POD_NAME":"istio-ingressgateway-bb8766f79-68ktm", "istio":"sidecar"}}, proxyInstances:[]*model.ServiceInstance(nil), ip:"<IP_ADDRESS>", port:80, protocol:"HTTP", bindToPort:true, filterChainOpts:[]*v1alpha3.filterChainOpts{}}
chain: 0; []plugin.FilterChain{}

Repeating the same test on release-0.8 shows exactly the same lack of routes due to temporary virtualservices being empty. So the behavior at least is the same but test fails on mixer branch, must be some other reason for the failure.
2018-05-17T03:38:30.498236Z     info    buildGatewayListeners: gateways after merging: &{map[istio-ingressgateway.istio-system.svc.cluster.local:true] map[80:[port:<number:80 protocol:"HT
TP" name:"http" > hosts:"uk.bookinfo.com" ] 443:[port:<number:443 protocol:"HTTPS" name:"https" > hosts:"uk.bookinfo.com" tls:<mode:SIMPLE server_certificate:"/etc/certs/cert-chain.pem" p
rivate_key:"/etc/certs/key.pem" ca_certificates:"/etc/certs/root-cert.pem" > ]]}
2018-05-17T03:38:30.498624Z     info    Servers: [port:<number:443 protocol:"HTTPS" name:"https" > hosts:"uk.bookinfo.com" tls:<mode:SIMPLE server_certificate:"/etc/certs/cert-chain.pem"
private_key:"/etc/certs/key.pem" ca_certificates:"/etc/certs/root-cert.pem" > ]
2018-05-17T03:38:30.498766Z     info    hosts=[uk.bookinfo.com], virtualServices=[]
2018-05-17T03:38:30.498861Z     info    constructed http route config for port 443 with no vhosts; omitting
2018-05-17T03:38:30.498940Z     info    omitting HTTP listeners for port &{443 %!d(string=HTTPS) %!d(string=https)} filter chain 0 due to no routes
2018-05-17T03:38:30.499018Z     warn    FilterChains are empty!
2018-05-17T03:38:30.499123Z     info    buildGatewayListeners: gateways after merging: &{map[istio-ingressgateway.istio-system.svc.cluster.local:true] map[80:[port:<number:80 protocol:"HT
TP" name:"http" > hosts:"uk.bookinfo.com" ] 443:[port:<number:443 protocol:"HTTPS" name:"https" > hosts:"uk.bookinfo.com" tls:<mode:SIMPLE server_certificate:"/etc/certs/cert-chain.pem" p
rivate_key:"/etc/certs/key.pem" ca_certificates:"/etc/certs/root-cert.pem" > ]]}
2018-05-17T03:38:30.499448Z     info    Servers: [port:<number:80 protocol:"HTTP" name:"http" > hosts:"uk.bookinfo.com" ]
2018-05-17T03:38:30.499531Z     info    hosts=[uk.bookinfo.com], virtualServices=[]
2018-05-17T03:38:30.499604Z     info    constructed http route config for port 80 with no vhosts; omitting
2018-05-17T03:38:30.499668Z     info    omitting HTTP listeners for port &{80 %!d(string=HTTP) %!d(string=http)} filter chain 0 due to no routes
2018-05-17T03:38:30.499712Z     warn    FilterChains are empty!

we need to fix this before the final 0.8 release as the 503 gateway test is disabled..
I encountered this bug in a different scenario. Seems like adding a HTTPS port in the gateway triggers some of this behavior as well.

I'm not sure that's the same issue - my PR hit this as well and the fix is among the commits. The symptom for the issue you mention is seeing "cannot set both HTTP and network filters" (because setting listener type based on protocol was wrong in some parts of the code). With that fix in, I nevertheless still see some other problem in the 503.

The reason for the gaps is envoy crash, now moved to separate issue: https://github.com/istio/istio/issues/5694
There's still the question of virtualservices going missing leading to temporary missing listeners.

Ingress gateway mixer filter has been disabled until this is resolved.

This is not blocking 0.8.0 release but @qiwzhang and folks are helping to diagnose and see if it can still be fixed in 0.8.0. If not, Ingress gateway mixer filter will have to be enabled in either 0.8.1 (if it happens) or 1.0.0.

Original issue has been fixed, closing this.


Users should be able to delete their worlds

important to only include worlds in the query where the user is the owner.

the commit ddbc0dc solved the issue


bug Fix gc rowset bug

Proposed changes
Write rowset:
add to pending_rowset_set
write rowset file to tablet dir
add to tablet_version_map
erase from pending_rowset_set
Original gc rowsets:
list tablet dir to get rowset_files
take snapshot of tablet_version_map
reclaim rowset_files not in tablet_version_map_snapshot and not in pending_rowset_set
Bug happens:
[W] add to pending_rowset_set
[W] write rowset file to tablet dir
[GC] list tablet dir to get rowset_files
[GC] take snapshot of tablet_version_map
[W] add to tablet_version_map
[W] erase from pending_rowset_set
// Now rowset is neither in tablet_version_map_snapshot or pending_rowset_set
[GC] reclaim rowset_files not in tablet_version_map_snapshot and not in pending_rowset_set
Fixed gc rowsets:
list tablet dir to get rowset_files
filter out rowset_files in pending_rowset_set
take snapshot of tablet_version_map
reclaim rowset_files not in tablet_version_map_snapshot
Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


koord-scheduler: add ElasticQuota Guaranteed usage

Ⅰ. Describe what this PR does
add ElasticQuotaGuaranteeUsage featuregate go guarantee usage

Ⅱ. Does this pull request fix one issue?

Ⅲ. Describe how to verify it
Ⅳ. Special notes for reviews
V. Checklist

[ ] I have written necessary docs and comments
[ ] I have added necessary unit tests and integration tests
[ ] All checks passed in make test

/lgtm

/approve


File Manager: Tabs information should be user-specific between sessions

If you login as a non-admin webmin site use and use filemin, when you log in as a different user (e.g. root), it tries to load the previous path opened in filemin when you open it (I assume set via cookie which isn't cleared on logout?), but returns an error as it can't find it, disclosing the path the last user was looking at.
I realise this isn't a big deal as it requires two different users to be using the same PC and browser, however as I frequently flick between the admin account and site user accounts, I've noticed it and I thought I'd better bring it up in case it is just part of a bigger issue.

Hi,
It sounds as very possible issue. I will try add user checking. It happens because of new feature - tabs.
Thanks for reporting!

That was hard.
Version 18.01 will make it possible.
Thanks for the idea.


Fix bug for dmon-agent collectd crash

File "/opt/dmon-agent/dmon-agent.py", line 157, in post
aux.configureComponent(settingsDict, collectdTemp, collectdConf)
File "/opt/dmon-agent/pyUtil.py", line 271, in configureComponent
subprocess.Popen('echo >> ' +filePath) # TODO fix this
File "/usr/lib/python2.7/subprocess.py", line 679, in init
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

Added shell=True to Popen to force execution using shell.


Add an option to return each row as an object keyed by column name

This is one approach for resolving #24

Got thinking about the name, and my original choice didn't seem ideal in the event someday in the future someone wants to add a different format. So instead of rowObjects: true, I think rowFormat: 'object' is more extensible.

Great! When will the new version release?

Just published v1.2.0 with option rowFormat: 'object'


Pretty print JSON/XML payload when sayAndMakeRequest(Request)

Pretty prints the payload when Content-Type is either JSON or XML. I'm not overly happy with the PayloadUtils class, but gets the job done..
It's might also be problem that ObjectMapper and XmlMapper instances are not user-configurable, which practically makes testing models with, for example, Java 8 DateTime fields much harder (support via Jackson JSR-310 datatype module).

doctester » doctester #59 SUCCESS
This pull request looks good
(what's this?)

We'll have a look asap. Thanks!

Any update on this? :)

Thanks for reminding :) Sometimes it's just too many PRs :D
Very nice feature and well implemented!

I just tried to upgrade Ninja's doctester 1.1.6 to 1.1.7 (just released with this feature). Unfortunately there is one test that breaks now (encoding issue). Would you mind having a look? Just check out Ninja and upgrade doctester in the main pom to 1.1.7. Thanks!

Will look into it today!

Okay the problem basically seems to be:

We want a pretty-printed XML output, but the default Stax parser (SJSXP provided by the JDK) doesn't support e.g. output indentation. And that's why Woodstox parser was introduced as a dependency - like dataformat-xml repo suggests.
Having Doctester depend on Woodstox then seems to also affect Ninja's XmlMapper instance (exist in same classpath?) and therefore Ninja's XML rendering: Woodstox impl doesn't render empty xmlns nor escape the > character. Ninja's test cases however assume both of these.

We could of course use non-indented xml ouput, but introducing Woodstox or even Aalto dependency and behaviour directly to Ninja might be good solution in the long term - as even dataformat-xml seems to assume a native Stax2 compliant implementation.

Very good response! You are right - it makes a lot of sense to add woodstox to Ninja as well. Btw - Group id of woodstox has changed to com.fasterxml.woodstox (and version 5.0.0 - https://github.com/FasterXML/woodstox).
I actually did not know that ">" MAY be escaped (according to the xml specs...)...
Would you mind creating a bug for that (or even add woodstox to Ninja?) :) Thanks!

Ah, good catch (the version)! I'll do PRs for Doctester and Ninja updates & fixes.

Wohooo! Cool :)


Fails on Windows

I'm using relative paths
        jsdoc : {
            dist : {
                src: ['appv2/**/*.js'],
                options: {
                    destination: 'doc'
                }
            }
        }

And getting the following error:
E:\Users\md\Documents\Visual Studio 2010\Projects\MyProject>grunt jsdoc
Running "jsdoc:dist" (jsdoc) task
>> An error occurs in jsdoc process:
>> 'E:\Users\md\Documents\Visual' is not recognized as an internal or external c
ommand,
>> operable program or batch file.
Warning: jsdoc failure Use --force to continue.

Aborted due to warnings.

As a note for others researching this issue, I discovered this error, and after debugging it looked like it was needing Java on my path to run.

An error occurs in jsdoc process:
'java' is not recognized as an internal or external command,
operable program or batch file.

@xavisavvy you can resolve the problem?

@xavisavvy don't you have either an old version of the plugin ? or the deprecated package ?
Could you post your package.json


Unable to boot XFCE4. CTRL+ALT+FORWARD brings me to chromeos developer console

chronos@localhost / $ sudo edit-chroot -all
name: xenial
encrypted: no
Entering /mnt/stateful_partition/crouton/chroots/xenial...
WARNING: Unable to register core dump handler.
crouton: version 1-20190902190258~master:9a594a66
release: xenial
architecture: amd64
xmethod: xiwi
targets: xiwi,xfce
host: version 12607.82.0 (Official Build) stable-channel caroline
kernel: Linux localhost 3.18.0-19838-gc647e70c954a #1 SMP PREEMPT Thu Jan 9 23:22:43 PST 2020 x86_64 x86_64 x86_64 GNU/Linux
freon: yes
Unmounting /mnt/stateful_partition/crouton/chroots/xenial...```

#### Please describe your issue:
sudo startxfce4 does not book to XFCE. Also, even before attempting 'start XFCE4' I'm able to CTRL+ALT+FORWARD which brings me to chromeos developer console where it's prompting me for localhost username and pw. Previously, this would only get me to XFCE after running 'startxfce4'

#### If known, describe the steps to reproduce the issue:
Step 1: CTRL+ALT+FORWARD brings me to chromeos developer console
Step 2: Go to crosh, type shell, then type 'sudo startxfce4'
Step 3: Receive the following:
Markers: (--) probed, (**) from config file, (==) default setting,
        (++) from command line, (!!) notice, (II) informational,
        (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(++) Log file: "/tmp/Xorg.crouton.1.log", Time: Wed Jan 22 21:04:38 2020
(++) Using config file: "/etc/X11/xorg-dummy.conf"
(==) Using system config directory "/usr/share/X11/xorg.conf.d"
/usr/bin/xinit: XFree86_VT property unexpectedly has 0 items instead of 1
crouton: version 1-20190902190258~master:9a594a66
release: xenial
architecture: amd64
xmethod: xiwi
targets: xiwi,xfce
host: version 12607.82.0 (Official Build) stable-channel caroline
kernel: Linux localhost 3.18.0-19838-gc647e70c954a #1 SMP PREEMPT Thu Jan 9 23:22:43 PST 2020 x86_64 x86_64 x86_64 GNU/Linux
freon: yes
Error: not connected.
Cannot connect to extension, retrying...
Error: not connected.
Cannot connect to extension, retrying...
Error: not connected.
Cannot connect to extension, retrying...
Error: not connected.
Cannot connect to extension, retrying...
Error: not connected.
Cannot connect to extension, retrying...
Error: not connected.
Cannot connect to extension, retrying...
Error: not connected.
Cannot connect to extension, retrying...
Error: not connected.
Cannot connect to extension, retrying...
Error: not connected.
Cannot connect to extension, retrying...
Error: not connected.
Cannot connect to extension, retrying...
Unable to start display, make sure the crouton extension is installed
and enabled, and up to date. (download from https://goo.gl/OVQOEt)
Running exit commands...
/usr/bin/xinit: connection to X server lost

waiting for X server to shut down Hangup
Hangup
Hangup
(II) Server terminated successfully (0). Closing log file.

Unmounting /mnt/stateful_partition/crouton/chroots/xenial...
Sending SIGTERM to processes under /mnt/stateful_partition/crouton/chroots/xenial...

@DennisLfromGA Your guidance did the trick. Thanks for the help


Flat 4.1.0 is deprecated causing an npm warn

Prerequisites

[x] Checked that your issue hasn't already been filed by cross-referencing issues with the faq label
[x] Checked next-gen ES issues and syntax problems by using the same environment and/or transpiler configuration without Mocha to ensure it isn't just a feature that actually isn't supported in the environment in question or a bug in your code.
[x] 'Smoke tested' the code to be tested by running it outside the real test suite to get a better sense of whether the problem is in the code under test, your usage of Mocha, or Mocha itself
[x] Ensured that there is no discrepancy between the locally and globally installed versions of Mocha. You can find them with: node node_modules/.bin/mocha --version(Local) and mocha --version(Global). We recommend that you not install Mocha globally.

Description
when installing mocha 8.1.1, npm warns that flat 4.1.0 is deprecated.
Steps to Reproduce
run npm i -g  mocha
Expected behavior: [What you expect to happen]
No warns
Actual behavior: [What actually happens]
npm WARN deprecated<EMAIL_ADDRESS>Fixed a prototype pollution security issue in 4.1.0, please upgrade to ^4.1.1 or ^5.0.1.
Reproduces how often: [What percentage of the time does it reproduce?]
all the time
Versions
mocha 8.1.1
node 12
ubuntu bionic

Issue opened in yargs-unparser:  https://github.com/yargs/yargs-unparser/issues/67


Validate Function not working

Hi,
I was trying the extension but when I tried to validate the configuration JSON I always received the following message:
Request failed, status code: 401 See console log for details
If I'm not wrong there is no way the review log for this extension so I am stucked.

I think I found the solution.
The 401 happens when JSON file to validate has a lot of content (more than 5000 lines). In these case, I think there is an API Limitation and the request fails. With shorter JSON, everything works fine.


[BUG]: Stratify notebooks reponses differ from testing sheet

Describe the bug

For all responses there are 2 additional properties, Im sure this is fine though and the document needs to be updated:

model = stratifiy(
...
concepts_to_stratify=None, #If none given, will stratify all concepts.
params_to_stratify= None #If none given, will stratify all parameters.
)

For the following, Im not sure if the actual response is correct or not
[2.3]
Prompt: "Stratify my model by the ages young and old where young and old can become old, but old cannot become young"
Testing Document:
model = stratify(
    template_model=model,
    key= "Age",
    strata=['young', 'old'],
    structure= [['young', 'old']],
    directed=False,
    cartesian_control=False,
    modify_names=True
)

Actual Response
model = stratify(
    template_model=model,
    key= "Age",
    strata=['young', 'old'],
    structure= [['young', 'old'], ['old', 'old']],
    directed=True,
    cartesian_control=False,
    modify_names=True,
    concepts_to_stratify=None, #If none given, will stratify all concepts.
    params_to_stratify= None #If none given, will stratify all parameters.
)

LLM responses do not need to match test doc responses


docs: prepare instability-example for publish

reword README
set msrv to 1.64
fix feature flags in example
fix licenses (remove NOTICES, use The Ratatui Developers)
remove CODEOWNERS file

Oops - sorry - wrong remote - I meant this to go to the instability remote, but didn't check. Apologies


cf org  doesn't work in 6.18.1

Command
cf org <org name> should return the organisation details but they are empty in 6.18.1
CLI Version
6.18.1
Error
With cf 6.18.0 and before:
$ cf org paas_user_research
Getting info for org paas_user_research as admin...
OK

paas_user_research:
                      domains:        cloudapps.digital
                      quota:          paas_demo (10240M memory limit, Unlimited instance memory limit, 1000 routes, 10 services, paid services allowed, unlimited app instance limit)
                      spaces:         sandbox
                      space quotas:

With cf 6.18.1:
14:35:31:aws:ruby-2.2.3:bin(master)$ cf org paas_user_research
Getting info for org paas_user_research as admin...
OK

paas_user_research:
                      domains:
                      quota:           (0M memory limit, 0M instance memory limit, 0 routes, 0 services, paid services disallowed, unlimited app instance limit)
                      spaces:
                      space quotas:

CC API Endpoint Version
API version: 2.54.0
CF Trace
$ CF_TRACE=true cf org paas_user_research

REQUEST: [2016-06-08T14:40:13+01:00]
GET /v2/organizations?q=name%3Apaas_user_research HTTP/1.1
Host: api.cloud.service.gov.uk
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Content-Type: application/json
User-Agent: go-cli 6.18.1+a1103f0 / darwin

RESPONSE: [2016-06-08T14:40:13+01:00]
HTTP/1.1 200 OK
Content-Length: 1530
Connection: keep-alive
Content-Type: application/json;charset=utf-8
Date: Wed, 08 Jun 2016 13:40:13 GMT
Server: nginx
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: 9eed4c83-4d83-4981-bb71-6aa7bf08c8c3

{
  "total_results": 1,
  "total_pages": 1,
  "prev_url": null,
  "next_url": null,
  "resources": [
    {
      "metadata": {
        "guid": "49a23332-8cec-43d8-837c-89098d570c70",
        "url": "/v2/organizations/49a23332-8cec-43d8-837c-89098d570c70",
        "created_at": "2016-06-08T12:54:10Z",
        "updated_at": "2016-06-08T13:10:58Z"
      },
      "entity": {
        "name": "paas_user_research",
        "billing_enabled": false,
        "quota_definition_guid": "f9bbc5cb-1d8f-4532-bb0f-5b556e581ae8",
        "status": "active",
        "quota_definition_url": "/v2/quota_definitions/f9bbc5cb-1d8f-4532-bb0f-5b556e581ae8",
        "spaces_url": "/v2/organizations/49a23332-8cec-43d8-837c-89098d570c70/spaces",
        "domains_url": "/v2/organizations/49a23332-8cec-43d8-837c-89098d570c70/domains",
        "private_domains_url": "/v2/organizations/49a23332-8cec-43d8-837c-89098d570c70/private_domains",
        "users_url": "/v2/organizations/49a23332-8cec-43d8-837c-89098d570c70/users",
        "managers_url": "/v2/organizations/49a23332-8cec-43d8-837c-89098d570c70/managers",
        "billing_managers_url": "/v2/organizations/49a23332-8cec-43d8-837c-89098d570c70/billing_managers",
        "auditors_url": "/v2/organizations/49a23332-8cec-43d8-837c-89098d570c70/auditors",
        "app_events_url": "/v2/organizations/49a23332-8cec-43d8-837c-89098d570c70/app_events",
        "space_quota_definitions_url": "/v2/organizations/49a23332-8cec-43d8-837c-89098d570c70/space_quota_definitions"
      }
    }
  ]
}
Getting info for org paas_user_research as admin...
OK

paas_user_research:
                      domains:
                      quota:           (0M memory limit, 0M instance memory limit, 0 routes, 0 services, paid services disallowed, unlimited app instance limit)
                      spaces:
                      space quotas:

Platform Details
Mac OS X 10.11.5 and Ubuntu 14.04.4 64-bit
Shell
Terminal

Hi @saliceti
We fixed this in the following story, will be included in the next release:
https://www.pivotaltracker.com/story/show/119658109
Regards,
Dies Koper
CF CLI PM

We have fixed this in cf CLI 6.19.0, released today.
https://github.com/cloudfoundry/cli#downloads
Regards,
Dies Koper
CF CLI PM

I confirm this is working now. Thanks!


Can't build with base >= 4.7

Perhaps I'm doing something dumb, but I can't build any of the code (here is chapter7/poly but others don't build as well)
$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.8.4
$ cabal --version
cabal-install version <IP_ADDRESS>
using version <IP_ADDRESS> of the Cabal library
$ cabal install
Resolving dependencies...
cabal: Could not resolve dependencies:
trying: poly-<IP_ADDRESS> (user goal)
next goal: base (dependency of poly-<IP_ADDRESS>)
rejecting: base-<IP_ADDRESS>/installed-bfd... (conflict: poly => base>=4.6 && <4.7)
rejecting: base-<IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>,
<IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>,
<IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS> (global constraint requires installed instance)
Dependency tree exhaustively searched.

Note: when using a sandbox, all packages are required to have consistent
dependencies. Try reinstalling/unregistering the offending packages or
recreating the sandbox.

Is there a greater reason for not supporting GHC 7.8?

Actually it seems it's because repline doesn't support base 4.7.

I bumped the upper bound on repline and reuploaded to hackage. Let me know if that fixes your problem.
https://hackage.haskell.org/package/repline-<IP_ADDRESS>

Thank you. By also changing the bounds of the packages here they run fine.

Great. Thanks for reporting.


Enhancement: song lists

so i enjoy modding the game because its simple and easy, after learning how on your engine it still was, except when you boot up a song without either the Inst or voices it straight up crashes the game, I just recommend making so they aren't required to play the levels as a lot of mods use only one and delete the other.
thanks!

...isn't there a needsVoices option in the song json?

...isn't there a needsVoices option in the song json?

exactly. Plus, tutorial doesn't use that option, and the game doesnt crash.


WIP: Use Pipenv instead of pip and requirements.txt

I will note that Pipfile/Pipenv was incredibly unstable for the Requests project and slowed down our CI builds.

I am just trying it out. While working on this, I ran into a pretty serious bug: https://github.com/kennethreitz/pipenv/issues/443


Support Emoji stickers - Restore grab title from file option

First of all allow me to express all my gratitude for this wonderful hidden gem (your plugin)
I personally use MAKE.md to view my vault so emoji stickers are shown next to the note title.
This little code makes it so that if there's a "sticker: emoji//" property it will be grabbed and put in the title.
I also have to add, for full disclosure, that obsidian also takes the sticker property and uses it as a small logo on top of the note title natively, but that functionality seemed to hard to implement for me at the moment.
Limitations: only supports emoji - lucide or other sticker images will not be rendered and will be ignored
Attached are images of how obsidian handles emojis and how they are handled with this pr in the exported html

Thank you, this is an awesome contribution. I'll look over the code when I have a chance and merge it in

Thank you, this is an awesome contribution. I'll look over the code when I have a chance and merge it in

You're welcome!
I would be happy to help with the sticker-as-logo part of the code but I feel like it should be built upon the "filename as title" feature that you said you were gonna rework in #292 so let me know how I can help with that feature first

P. S. I just noticed I might have worked on an older version of the plugin which didn't support custom title property. I will add it in tomorrow.

This updated commits account for the custom title property.

Another small oversight fixed (hopefully the last)

Awesome! I will try and look over this today

Fixed title grabbing from file.
The previous solution was setting the tile after the data heading like this
`
New solution writes it this way instead
🙅 GetTitle
If you want also the id to match the gettitle (which means the field id will be set by the gettitle function output too)
I had to work on this pr and make this a draft because now as you can see the emoji and the title are rendered like a single phrase but I will split the gettitle output to use the emoji as a separate logo instead so that it matches the obsidian output.
I haven't encountered any bugs but feel free to let me know if you manage to reproduce any of the old ones

Everything is working as intended now

Thank you again for all this work, I'll be sure to look over and merge today :)

Thank you again for all this work, I'll be sure to look over and merge today :)

You're welcome. What else do you think it's needed to get to the 1.4.7 release?

@7Mik: You're welcome. What else do you think it's needed to get to the 1.7.4 release?

Honestly, I just need to fix one bug, and then merge all these changes together, do testing, and fix any more bugs that might occur after integration.

@7Mik: You're welcome. What else do you think it's needed to get to the 1.7.4 release?

Honestly, I just need to fix one bug, and then merge all these changes together, do testing, and fix any more bugs that might occur after integration.

If you need help for anything count me in!

If you need help for anything count me in!

Thank you! I will certainly let you know! Right now I am going to focus on merging everything actually and then fix that bug afterwards because it will make continued work on the plugin much easier once everything is not split into two working versions

I cannot test this PR easily because the branch is named master which conflicts with the main master branch in my repo, so I will go ahead and merge it and look over the changes from the master branch. I am fairly confident that there is nothing glaringly wrong with this PR.

@7Mik Hey, so I am testing this and it does not work for me. MAKE.md seems to set the sticker property to an emoji, however you are trying to parse an int and it always says invalid sticker. I am not sure why we can't just grab the content direcly from the sticker property and put it in front of the title? Is the way you're doing it using like the unicode characters or something?

@7Mik Hey, so I am testing this and it does not work for me. MAKE.md seems to set the sticker property to an emoji, however you are trying to parse an int and it always says invalid sticker. I am not sure why we can't just grab the content direcly from the sticker property and put it in front of the title? Is the way you're doing it using like the unicode characters or something?

Very weird, I have exported my whole vault and it worked flawlessly both for the sticker in the file tree and in the title.
What is your YAML looking like after you choose an emoji? Keep in mind that lucide and other stickers aren't supported

---
text_area_test: hello
text_test: there
banner: "[[test jpg.jpg]]"
banner_y: 0.5
sticker: ⭐
title: Test Document
---

--- text_area_test: hello text_test: there banner: "[[test jpg.jpg]]" banner_y: 0.5 sticker: ⭐ title: Test Document ---

That is very weird because my sticker property looks like this: sticker: emoji//1f645
What version of MAKE.md are you on?

Ahhhh, I actually manually added the sticker property instead of picking it through MAKE.md's interface. But MAKE.md still interprets it correctly. A normal emoji should be detected and accepted if the user writes it manually. Especially since I might like to use this feature without using MAKE.md

I will just change it so it will only try to parse the property if it begins with "emoji//"

I will just change it so it will only try to parse the property if it begins with "emoji//"

And if it's not emoji//?

I will just insert whatever happens to be there as text content (to disallow html injection)

I will just insert whatever happens to be there as text content (to disallow html injection).
Edit: And actually since I render the title as markdown that will automatically sanitize any input. So I can just insert anything that happens to be there.

Try and test with lucide emoji since I think it might bug it out

Ah yes it does, but I beleive I can actually make it work with lucide

Lucide is now working:

However two more things to note:

Files with an icon are no longer sorted correctly because the icon is being used to sort by
The MAKE.md inline title / header is completely broken even with styles included

Good job on making lucide work. About the file sorting you could just make a ::before element to store the emoji since they are already passed as two different variables into the filetree.js
I don't really understand why is it broken as for me it's just working fine, how can I reproduce it on my side?

Hmm I just disabled MAKE.md and it's still showing the make.md header stuff so I think it might just be something not updating actually.

Ahh it's actually the stickerlogo that is being added by the plugin. It's being added as text content rather than HTML content so it breaks when an SVG is added to it instead of an emoji.

Ahh it's actually the stickerlogo that is being added by the plugin. It's being added as text content rather than HTML content so it breaks when an SVG is added to it instead of an emoji.

Because you're treating the lucide emoji as an svg?

Yes, do you know of a better way?

Yes, do you know of a better way? I am just using the getIcon function from obsidian.

I didn't know of such function, unfortunately as far as I know lucide are only usable trough svg.
That's why I decided only to support native emoji. People might have all sorts of different plugins using lucide fontawesome or any custom emoji so supporting every type of emoji will be a huge pain.

I was thinking it could be useful to add a "default folder icon" and a "default file icon" setting so users could mimik the way MAKE.md has icons in the file tree easily. But override it with the sticker property (or with MAKE.md itself).


Git and Version Control Systems

OSSU previously listed 6 different resources to learn Git. Too many, in my humble opinion. The current curriculum doesn't cover version control systems at all. (In particular, version control is not part of the "About This Course" description for either Software Engineering: Introduction or Software Development Capstone Project).
I recommend that we include one course on Git in Core Applications, after Software Engineering: Introduction. I recommend How to Use Git and GitHub from Udacity.

Apologies that I am about to steer this topic into talking about what I'm envisioning for some future version of OSSU, but I will eventually address your suggestion.
After I had to manage an employee at my company who didn't understand Git, despite having a Bachelor's in CS, I have very much come around to the idea of integrating some notion of basic tooling into the requirements of the OSSU curriculum.
That would mean not just version control systems, but also things like development environments, APIs, libraries, transpilers, common databases, deployment and workflow processes... of course the trouble with these is that they aren't timeless like most of the curriculum, they are constantly changing, and the particulars are often highly domain-specific.
One of my goals is to clearly delineate the (relatively) "timeless" portions of the curriculum -- theory that will even outlast our lifetimes, which we can call "Category I" -- from parts that are "industry practice" for the near future but far from timeless, "Category II" -- and further from parts that are so ephemeral, they might be outdated by the time you finish the curriculum, "Category III".
Decentralized snapshot-based version control like Git or Mercurial is just graph theory. It becomes significantly easier to understand once one has internalized this. Decentralized snapshot-based version control is also, undoubtedly, industry practice for now, and maybe the next decade or more. (An alternative to snapshot-based would be patch-based.)
Perhaps an elegant way to handle this would be to somehow integrate teaching graph theory with teaching Git. That could mean having Category I content embedded into Category II content, or vice versa. I personally prefer the former: Category I content embedded into Category II content, as then it would be elegant to embed Category I and II content inside Category III content, like so:
MODULE
UNIT 1: Introduce project to solve real-world problem, student attempts a solution using what they know already (Category III)
UNIT 2: Learn new tools that improve student's solution/workflow (Category II)
UNIT 3..n: Learn theory needed to better understand tools and problem domain (Category I)
UNIT n+1: Student solves the problem using newfound knowledge (integration of categories)
Unit 1 would be some project devised by OSSU. Unit 2 would be some recommended tutorials, or maybe short course, such as the Git course you recommended. Units 3 until "n" (however many courses it takes) would be the traditional coursework we already have. And for Unit "n" (the last unit), students would apply everything they've learned, and get graded on the elegance of their solution by their peers at OSSU. An advantage of this approach is that it shows the student how theory is a tool to improve practice -- a tool of tools.
This kind of highly-integrated curriculum program is a long way off for us, but it's something I've been thinking about for the past month or so. It's not going to happen overnight; whatever v8 of OSSU CS looks like, it will probably only be an incremental improvement on what we have now.
On that note, I'm not really convinced that simply sticking a git course into Core Applications solves this in the best possible way even in the short-term. Think of Core Applications as "domains of computer science" -- git is definitely not a domain, but just a tool everyone needs to use whether they understand it or not. In the shortest-term, I think we should just find the best git tutorial on the Internet and link to it somewhere, such as on the CONTRIBUTING page. For the next curriculum version, I hope we can develop some required projects for students, for which they will need to learn and use version control.

@hanjiexi very interesting idea! We certainly can go in that direction.
About the "industry practice" courses, I like to mix these concepts with more theoretical stuff but a recurring thought that always comes to my mind is:

It'd be really great to have a web engineering course, with 80 ~ 90% "industry practice" and 10 ~ 20% CS stuff.

This course should start with the UI layer - UI/UX > HTML > CSS > JS, and maybe finish with one creating their own distributed server with a raspberry pi cluster :)
We should still work on a more "language-level" in such course, so less attention for frameworks, but a deep understand of "how to create such frameworks" so to speak.
I'll add more comments about the Computer Science course later. But one implication of the Web Engineering (WE) course will be to keep the CS one really focused on CS stuff, and the more "practical contents" (industry practice) will stay in the WE.
The format @hanjiexi suggested is still worth for all OSSU courses for sure.
Another idea is to split the CS course and each Advanced CS section will become an independent repository and the CS course will be the Intro + Core section only.

@waciumawanjohi The next OSSU CS update will include two recommended git resources (a book and a course, the one you recommended). Finding a good place to put it is awkward in the current structure; for now it's on the contributing page but in a future update hopefully we can give it a better home by having better coverage of software tooling.
@ericdouglas Those are some great ideas, and my thinking is similar especially on this:

Another idea is to split the CS course and each Advanced CS section will become an independent repository and the CS course will be the Intro + Core section only.

But let's talk about it in a separate topic: #422

One more thing: I talked to the teacher of Software Engineering: Introduction who confirmed that it will have students use git for the course project. They will probably either integrate a git tutorial somewhere into the course sequence, or link to one that they recommend, but since the course isn't out yet they have time to figure out what they're going to do.


Unable to start udev on Alpine Edison base images

udev won't be able to start properly.
always hit Segmentation fault when executing udevd.

Will be fixed in the hostOS


isKeyJustPressed & isButtonJustPressed does not trigger consistently at '1.11.0' with gdx-lwjgl3-glfw-awt-macos

Not using the gdx-lwjgl3-glfw-awt library and using JVM opts "-XstartOnFirstThread" fixes the issue.

[ X] macOS

It seems like -XstartOnFirstThread is the only really reliable way to have LWJGL3 work on MacOS. The StartOnFirstThreadHelper class from Guacamole may make distributing executable JARs easier -- just start your main() method with if (StartOnFirstThreadHelper.startNewJvmIfRequired()) return; and that file takes care of restarting with the necessary argument if it isn't already there. Distributing with Packr or jpackage means users don't need a JDK installed (which can be tricky on MacOS, because of x64 and ARM64 both being used), and both of those can specify -XstartOnFirstThread as an argument when launched via the .app . The need for -XstartOnFirstThread is mostly an issue during development, when JARs are being sent around but not yet packaged with JREs for distribution. Hopefully the class from Guacamole (which really just needs that file, not the library) can help.

I am planning to create a debug UI/editor UI as a second window on top / besides my libgdx app.
So I would be interested to fix these issues also. Where would I have to look ?

I don't think this is as easy to fix as you'd like it to be, but the best place to start looking is the original PR that introduced the extension (https://github.com/libgdx/libgdx/pull/6772).
Apart from that, you could try removing the gdx-lwjgl3-glfw-awt-macos extension and doing this at the beginning of your DesktopLauncher instead:
if (SharedLibraryLoader.isMac && Gdx.app == null) {
    org.lwjgl.system.Configuration.GLFW_LIBRARY_NAME.set("glfw_async");
}

This should have the same effect as our extension, but is offered by LWJGL 3 itself. So they might be able to help you figure out what is causing this.


[draft] support targz-ref conversion and runtime

Converter
Related PR for nydus-image builder: https://github.com/dragonflyoss/image-service/pull/858
To implement pull tar.gz lazily, the converter workflow will be:

Create the nydus blob layer (application/vnd.oci.image.layer.nydus.blob.v1) with the tar-like format:

image.blob.meta: nydus blob meta
image.boot: nydus bootstrap

| <image.blob.meta> | <image.blob.meta> header | <image.boot> | <image.boot> header |

Merge all <image.boot> into a final bootstrap (application/vnd.oci.image.layer.v1.tar+gzip).
Acceld/Nydusify support converting all nydus blob layers and bootstrap layer into OCI artifact manifest like:

{
  "mediaType": "application/vnd.oci.artifact.manifest.v1+json",
  "artifactType": "application/vnd.nydus.ref.v1",
  "blobs": [
    {
      "mediaType": "application/vnd.oci.image.layer.nydus.blob.v1",
      "size": 123,
      "digest": "sha256:87923725d74f4bfb94c9e86d64170f7521aad8221a5de834851470ca142da630",
      "annotations": {
        "containerd.io/snapshot/nydus-blob": "true"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.layer.nydus.blob.v1",
      "size": 456,
      "digest": "sha256:87923725d74f4bfb94c9e86d64170f7521aad8221a5de834851470ca142da630",
      "annotations": {
        "containerd.io/snapshot/nydus-blob": "true"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:aec98c9e3dce739877b8f5fe1cddd339de1db2b36c20995d76f6265056dbdb08",
      "size": 789,
      "annotations": {
        "containerd.io/snapshot/nydus-bootstrap": "true"
      }
    }
  ],
  "subject": {
    "mediaType": "application/vnd.oci.image.manifest.v1+json",
    "size": 123,
    "digest": "sha256:cc06a2839488b8bd2a2b99dcdc03d5cfd818eed72ad08ef3cc197aac64c0d0a0"
  },
  "annotations": {
    "containerd.io/snapshot/nydus-fs-version": "6"
  }
}

The subject field reference the original OCI image, https://github.com/opencontainers/image-spec/blob/main/artifact.md.
Perhaps the legacy registry does not support OCI artifact, we need to make it works by using a compatible image manifest format.

Runtime

Range fetch <image.blob.meta> data from the remote registry layer by layer and then write to the blob cache directory.
Pull the final bootstrap data (in tar.gz format) and unpack it to the snapshot directory.
Boot nydusd and mount the final bootstrap.

Could we also add support for "blob.digest" files too?


added 'new' and 'version' feature

@shubhodeep9
I have added 'new' and 'version' commands in it . As mentioned in issue #13 .
for 'new' I have just did git clone ... but I didn't checked whether git is installed or not on that operating system. I'll search more about it and can you please describe me what you mean by adding a "new controller" .

Hi @Adi2612 first of all thanks for the PR.
Let's start by git issue.
the thing I have in mind is, instead of cloning the repo, let's download the repo zip in /temp or equivalent they have in Windows. and unzip in the current location with the given app name.
So for example:
torn new sampleApp
should download repo zip to /temp and then unzip it in working directory.

Secondly, torn controller User should create a UserController with sample template code in Controller directory.

Hi @Adi2612 first of all thanks for the PR.
Let's start by git issue.
the thing I have in mind is, instead of cloning the repo, let's download the repo zip in /temp or equivalent they have in Windows. and unzip in the current location with the given app name.
So for example:
torn new sampleApp
should download repo zip to /temp and then unzip it in working directory.

@shubhodeep9
Yup, It is a great idea and thus it will also solves its dependency on git.
Can U tell me more why my commits have failed all the tests / checks.

@Adi2612 for now ignore the tests.
The travis has been failing for a while.

@shubhodeep9 , I have made changes in for "new" command . Please review it.

hey @shubhodeep9  created function for them.

@Adi2612 will you be working on controller too?

yup , I would love to work on that but yet I don't understand what U mean by creating a new controller . Can you please describe it more and also explain what will be its uses.

@Adi2612 will you be working on controller too?

yup , I would love to work on that but yet I don't understand what U mean by creating a new controller . Can you please describe it more and also explain what will be its uses.

Let's have the discussion on #13
meanwhile I'll merge this PR


WIP: feat(danger): Block PR if yarn audit has high or critical

Summary

Related Issues or PRs

How To Test

Screenshots (optional)

Warnings

:warning:

Changes were made to package.json, but not to yarn.lock - Perhaps you need to run yarn install?

  Generated by :no_entry_sign: dangerJS against aec57bc23f2b7849ca68b315b76958b908f5e202


imagePullSecrets missing in deployment.yaml

Describe the bug:
When deploying the cert-manager helm chart with imagePullSecrets set in values.yaml, the container image fails to download.
Expected behaviour:
Images should download without issue with the docker registry secrets configured.
Steps to reproduce the bug:
1- copy the container image in a registry that requires login credentials
2- kubectl create secret docker-registry ...
3- reference that registry in values.yaml image.repository
4- set the secret name in values.yaml imagePullSecrets
5- helm install
Anything else we need to know?:
Environment details::

Kubernetes version: any version
Cloud-provider/provisioner: any
cert-manager version: 1.7.0
Install method: helm

To fix this, add the missing imagePullSecrets under the container spec
/kind bug

The imagePullSecrets values are interpolated to each components service account template (see i.e here) not deployment template.
I've just tested it with cert-manager v1.7 with a values file that has image pull secret specified and I do see it added to the service account specs as expected. Maybe you've set the service account creation to false (i.e here)?
Perhaps we should add a note about this to the docs.

Thanks for the information, I did not check the service account template and you're right that it should address the issue. Closing this ticket...
Thank you


Proposal: instead of not rendering unknown views, fall back to View and continue to render children

I think our approach to not render unknown views and not rendering children is the wrong. I believe instead we should convert to a View or some other generic view, still apply modifiers, and continue rendering children.
This of course is a post v0.3 thing but let's discuss if this is in fact a better approach.
This would be consistent with the web:

I would do this by using Group when the element doesn't exist. Group automatically applies its modifiers to each child.
We should probably still log the error. It could be something like this:
<nonexistent style="foregroundStyle(.orange);">
  <Text>Hello, world!</Text>
</nonexistent>

would render as:
Group {
  Text("Hello, world!")
}
.foregroundStyle(.orange)
.task {
  logger.log(.warning, "...")
}

with a message like this:
Unknown element `<nonexistent>`. The app may not render as expected.

It would be nice if we had some debug information passed on all elements. I think LiveView does this for a jump-to-definition feature. It could use this info to log the file/line number of the unknown element.

That works

Brian Cardarella
Founder, DockYard
On Tue, Aug 20, 2024 at 1:43 PM Carson Katri @.***>
wrote:

I would do this by using Group when the element doesn't exist. Group
automatically applies its modifiers to each child.
We should probably still log the error. It could be something like this:

Hello, world!
would render as:
Group {
Text("Hello, world!")}.foregroundStyle(.orange).task {
logger.log(.warning, "...")}
with a message like this:
Unknown element <nonexistent>. The app may not render as expected.
It would be nice if we had some debug information passed on all elements.
I think LiveView does this for a jump-to-definition feature. It could use
this info to log the file/line number of the unknown element.
—
Reply to this email directly, view it on GitHub
https://github.com/liveview-native/liveview-client-swiftui/issues/1413#issuecomment-2299407066,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAEQXAPOZXCJWBATNXRAJLZSN55NAVCNFSM6AAAAABMYYBC4KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJZGQYDOMBWGY
.
You are receiving this because you authored the thread.Message ID:
@.***
com>

Currently, this is catching elements such as <Style> and <iframe>, which seems incorrect. Should we just manually exclude these tag names?
'<csrf-token>' is not a known element.
'<Style>' is not a known element.
'<iframe>' is not a known element.
'<nonexistent>' is not a known element.

I suspect we'll need to split between HEAD and BODY here to really manage this concern properly. Thoughts?

Yeah, that makes sense to me. The iframe may still need to be handled separately, since I don't think it'd be placed in the head.

I can change the plug to inject into whatever part of the docuemnt we want, it's really only in dev environment so render perf isn't an issue if I were parse the documents and walk into a specific part


Problem with the Adapter Web Port

Hello,
My Problem im not getting fixed is the Web port delivers a Upgrade Required when accessing.
Further in the console of the Webbrowser it shows an error 426.
Versions:

Adapter version: V0.12.5
JS-Controller version: 4.0.24
Node version: v16.19.1
Operating system: Linux

The Adapter does Not Provider Andy Content unser the Port it is just for the ocpp connection


cursor position displaced

cursor position is displaced by one step to the right (in the editor window, not talking about the preview).

Do you use such special unicode characters? How can I reproduce it ?

yes, indeed, I am using special characters as it often happens in the diversity of languages outside the Englisch bubble. Also looks like the above mentioned problem has to do with encoding and special characters as the latter were rendered badly in the pdf. Have no idea how to reproduce it. PS: using asciidoxfx on windows 10.

Hi, I think I have understood the problem now. It occurs if you copy text into asciidocfx from another text editor like Word. gosh - it would be helpful to get malformed special characters highlithed.

I also have this problem and it seems related to certain (but not all) Unicode characters. For example, the check mark (U+2713) character has this problem.
It can be simply reproduced by typing the following comment in AsciidoxFX:
// After adding the ✓ symbol, the cursor is in the wrong place :-(

To really understand the problem, just try to edit the last part of this sentence (e.g. try to change the word "is" to "was") and you'll probably notice that you didn't end up with what you expected.


avoid materializing some special constants

Whether using @jit or op-by-op mode, the fact that we're evaluating Python means that we might end up materializing some large regularly-structured constants, like the outputs of np.arange or np.eye. (We have a separate strategy for dealing with np.zeros, np.ones, and np.full based on creating zero-stride ndarrays and treating them in a special way, though this PR unifies their treatment.) That can be wasteful and make both compile- and run-times slower.
Instead, when using @jit we'd like code for generating these constants to be staged into the computation, and when calling functions op-by-op in many cases we'd probably like to execute code to generate the constant in device memory rather than materializing the constant on the host and transferring it.
A key motivating example is PRNG. Our count-based PRNG design means to generate 1e7 random values we map a hash function over a count vector of size 1e7. In the master branch, that means we create an array of length 1e7 on the CPU by calling onp.arange(1e7), then we ship that to the device for the actual computation. Pretty dumb!
To avoid that, all we need is a bit of laziness. If we create lazy constant values, rather than materializing the constants outright, they have a chance to persist in their un-materialized form until they encounter a jit trace (or, for op-by-op mode, a computation to be applied on the device).
This PR adds lazy constant values as well as a lax.tie_in primitive for ensuring that constant values are hooked into the staged computation. It also updates the PRNG code to take advantage of these lazy constants, avoiding the pathology described above.
TODO:

[x] do some simple benchmarking done
[x] add unit tests
[x] tune the cutoff in xla.py instantiate_device_constant for when to materialize-and-transfer vs when to build-on-device for lazy constant arguments passed to computations done

Note for reviewer: a few commits up I experimented with making lax.full a primitive, but I didn't like the power:weight and so in bdc9e92f I opted for some type-testing in the traceable instead. Feedback welcome.

Here's some benchmarking using this PRNG script (originally by @robieta). For the CPU test I ran it on my desktop, and for the GPU test I ran it on a Cloud VM with a P100.
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals

from functools import partial
from timeit import default_timer

import numpy as np
from jax import lax
from jax import jit
from jax import numpy as jax_numpy
from jax import random as jax_random

n_list = [1e3, 5e3, 1e4, 2e4, 5e4, 1e5, 5e5, 1e6, 2e6]
n_list = [int(i) for i in n_list]

key = jax_random.PRNGKey(0)
print("             Compile time   ( per 1k n)   Run time")
for n in n_list:
  x = np.arange(n, dtype=np.int32)
  key, subkey = jax_random.split(key)

  start_time = default_timer()
  y = jax_random.uniform(subkey, [n])
  run_and_compile_time = default_timer() - start_time

  key, subkey = jax_random.split(key)

  start_time = default_timer()
  z = jax_random.uniform(subkey, [n])
  run_time = default_timer() - start_time

  approx_compile_time = run_and_compile_time - run_time

  print("n = {} {:12.4f}  {:11.4f}    {:8.4f}".format(
      str(n).ljust(8), approx_compile_time, approx_compile_time / n * 1000,
      run_time))

CPU@master
             Compile time   ( per 1k n)   Run time
n = 1000           0.0850       0.0850      0.1208
n = 5000           0.0606       0.0121      0.1120
n = 10000          0.0590       0.0059      0.1128
n = 20000          0.0586       0.0029      0.1133
n = 50000          0.0511       0.0010      0.1255
n = 100000         0.0737       0.0007      0.1207
n = 500000         0.0703       0.0001      0.1772
n = 1000000        0.0621       0.0001      0.2009
n = 2000000        0.0651       0.0000      0.2701

CPU@lazy-constants
             Compile time   ( per 1k n)   Run time
n = 1000           0.0183       0.0183      0.1276
n = 5000          -0.0016      -0.0003      0.1355
n = 10000         -0.0042      -0.0004      0.1536
n = 20000          0.0068       0.0003      0.1363
n = 50000         -0.0007      -0.0000      0.1341
n = 100000        -0.0083      -0.0001      0.1603
n = 500000         0.0138       0.0000      0.1473
n = 1000000        0.0075       0.0000      0.1532
n = 2000000        0.0068       0.0000      0.1511

GPU@master
             Compile time   ( per 1k n)   Run time
n = 1000           0.1414       0.1414      0.1839
n = 5000           0.0982       0.0196      0.1895
n = 10000          0.0852       0.0085      0.1940
n = 20000          0.0861       0.0043      0.1932
n = 50000          0.0541       0.0011      0.2351
n = 100000         0.0939       0.0009      0.2039
n = 500000         0.0832       0.0002      0.2478
n = 1000000        0.0918       0.0001      0.3019
n = 2000000        0.1047       0.0001      0.4055

GPU@lazy-constants
             Compile time   ( per 1k n)   Run time
n = 1000           0.1062       0.1062      0.2101
n = 5000           0.0579       0.0116      0.2155
n = 10000          0.0601       0.0060      0.2146
n = 20000          0.0575       0.0029      0.2198
n = 50000          0.0528       0.0011      0.2201
n = 100000         0.0810       0.0008      0.1906
n = 500000         0.0591       0.0001      0.1896
n = 1000000        0.0295       0.0000      0.2226
n = 2000000        0.0579       0.0000      0.2178

These seem like big wins, especially the GPU run time.

I did some timings on that P100 cloud machine to see where the cutoff is for materialize-and-transfer versus build-on-device.
                xfer time   inst time
n = 1000           0.0006       0.0043
n = 5000           0.0005       0.0044
n = 10000          0.0005       0.0044
n = 20000          0.0005       0.0048
n = 50000          0.0006       0.0049
n = 100000         0.0007       0.0049
n = 500000         0.0012       0.0049
n = 1000000        0.0044       0.0049
n = 2000000        0.0085       0.0050
n = 5000000        0.0232       0.0047
n = 10000000       0.0670       0.0051

Seems like the cutoff is around a size of 1e6. That might get lower as we improve our execution overheads.

The CPU timings on my desktop look similar in terms of cutoff point:
                xfer time      inst time
jax/lib/xla_bridge.py:143: UserWarning: No GPU found, falling back to CPU.
  warnings.warn('No GPU found, falling back to CPU.')
n = 1000           0.0002       0.0044
n = 5000           0.0002       0.0042
n = 10000          0.0003       0.0045
n = 20000          0.0002       0.0044
n = 50000          0.0004       0.0046
n = 100000         0.0004       0.0049
n = 500000         0.0027       0.0071
n = 1000000        0.0056       0.0077
n = 2000000        0.0113       0.0083
n = 5000000        0.0184       0.0111


Multi-cloud unit-tests 5 failing

After running npm run crm-sync:test:use-cases tests on the org with person accounts setup following results are received:
  42 passing (26m)
  5 failing

  1) Progressive resolution B2C Commerce Customer creation scenarios via the B2CContactProcess API
       creates a Contact that includes additional non-resolution specific field values:

       -- expected the Contact object to have the property Description
      + expected - actual

      -false
      +true

      at /Users/osapishchuk/IdeaProjects/b2c-crm-sync/test/_use-cases/_common/_validateContactProperties.js:31:16
      at Array.forEach (<anonymous>)
      at _validateContactProperties (test/_use-cases/_common/_validateContactProperties.js:28:23)
      at Object._executeAndVerifyB2CProcessResult [as executeAndVerifyB2CProcessResult] (test/_use-cases/_common/_executeAndVerifyB2CProcessResult.js:30:5)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (internal/process/task_queues.js:95:5)
      at async Context.<anonymous> (test/_use-cases/sfdc/b2cContactProcess.create.live.test.js:248:18)

  2) Progressive resolution of a B2C Commerce Customer update scenarios via the B2CContactProcess API
       allows direct updates by B2C CustomerList, Email, and LastName:

       -- expected the Contact object to have the property Description
      + expected - actual

      -false
      +true

      at /Users/osapishchuk/IdeaProjects/b2c-crm-sync/test/_use-cases/_common/_validateContactProperties.js:31:16
      at Array.forEach (<anonymous>)
      at _validateContactProperties (test/_use-cases/_common/_validateContactProperties.js:28:23)
      at Object._executeAndVerifyB2CProcessResult [as executeAndVerifyB2CProcessResult] (test/_use-cases/_common/_executeAndVerifyB2CProcessResult.js:30:5)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (internal/process/task_queues.js:95:5)
      at async Context.<anonymous> (test/_use-cases/sfdc/b2cContactProcess.update.live.test.js:155:25)

  3) Progressive resolution of a B2C Commerce Customer update scenarios via the B2CContactProcess API
       allows direct updates by B2C CustomerID:

       -- expected the Contact object to have the property Description
      + expected - actual

      -false
      +true

      at /Users/osapishchuk/IdeaProjects/b2c-crm-sync/test/_use-cases/_common/_validateContactProperties.js:31:16
      at Array.forEach (<anonymous>)
      at _validateContactProperties (test/_use-cases/_common/_validateContactProperties.js:28:23)
      at Object._executeAndVerifyB2CProcessResult [as executeAndVerifyB2CProcessResult] (test/_use-cases/_common/_executeAndVerifyB2CProcessResult.js:30:5)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (internal/process/task_queues.js:95:5)
      at async Context.<anonymous> (test/_use-cases/sfdc/b2cContactProcess.update.live.test.js:195:25)

  4) Progressive resolution of a B2C Commerce Customer update scenarios via the B2CContactProcess API
       allows direct updates by B2C CustomerList ID and B2C CustomerNo:

       -- expected the Contact object to have the property Description
      + expected - actual

      -false
      +true

      at /Users/osapishchuk/IdeaProjects/b2c-crm-sync/test/_use-cases/_common/_validateContactProperties.js:31:16
      at Array.forEach (<anonymous>)
      at _validateContactProperties (test/_use-cases/_common/_validateContactProperties.js:28:23)
      at Object._executeAndVerifyB2CProcessResult [as executeAndVerifyB2CProcessResult] (test/_use-cases/_common/_executeAndVerifyB2CProcessResult.js:30:5)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (internal/process/task_queues.js:95:5)
      at async Context.<anonymous> (test/_use-cases/sfdc/b2cContactProcess.update.live.test.js:236:25)

  5) Progressive resolution of a B2C Commerce Customer update scenarios via the B2CContactProcess API
       allows direct updates by ContactID:

       -- expected the Contact object to have the property Description
      + expected - actual

      -false
      +true

      at /Users/osapishchuk/IdeaProjects/b2c-crm-sync/test/_use-cases/_common/_validateContactProperties.js:31:16
      at Array.forEach (<anonymous>)
      at _validateContactProperties (test/_use-cases/_common/_validateContactProperties.js:28:23)
      at Object._executeAndVerifyB2CProcessResult [as executeAndVerifyB2CProcessResult] (test/_use-cases/_common/_executeAndVerifyB2CProcessResult.js:30:5)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (internal/process/task_queues.js:95:5)
      at async Context.<anonymous> (test/_use-cases/sfdc/b2cContactProcess.update.live.test.js:275:25)

Following duplicate rules are on the instnace:

This is my configuration for "B2C Commerce: Standard Person Accounts" duplicate rule


Add EXPLAIN preview to the diagnostics

See http://www.postgresql.org/docs/9.2/static/sql-explain.html. I don't think it's going to be too crazy to fetch the EXPLAIN plan behind a Linq query.
Might make an extension method off of IQueryable instead. This might get a little awkward when we introduce Include()

I can give that a go, if no one is looking into it (at least for non include queries). Should we add the optional parameters as well?

I don't think the code would be much different for the Includes. Just make sure we centralize the code for creating a DocumentQuery inside of the MartenQueryable, and you'd get the NpgsqlCommand object.
I'm drawing a blank, what do you mean about optional parameters?


Smart link is much slower than the Unity version

Hi,
With the same model, this version of the mesh decimator seems much slower than the Unity version.
I haven't profiled it yet but I was just scanning the code and in the Unity version, in the smart link code, when looping over the borderIndexCount I notice you generate a vertex hash and then break out of the inner loop. You aren't doing that in this non-Unity version though.
I have a model with several meshes and 64k triangles in each mesh and this double loop over the borderIndexCount seems very slow; it takes maybe 5 mins to simplify the model.
I'll try to do some tests and actual profiling, but it was definitely in that inner loop every time I broke into the debugger.
Cheers,
Sam

Hi @samizzo !
This is correct. A few weeks ago I made a new release of UnityMeshSimplifier with huge optimizations (with some instances having up to 15 times faster decimation). And I worked on getting the same optimizations here to MeshDecimator but was having problems with some tests I made just before the release of it.
Then I haven't had time to deal with it yet. But if you are interested the optimizations are available in the development branch. The changes can be seen here: https://github.com/Whinarn/MeshDecimator/compare/master...development
Once I have some available time I will look into the problems I noticed and deal with them in order to make a release.
The problems I noticed were different results of the decimation before the optimizations had been applied to after, so I must have made a mistake somewhere, because the intention was never to adjust the algorithm. So that you know. But if you are not noticing any issues on your side, you are free to use the code available in the development branch until I make a proper release. Be aware that it hasn't been properly tested yet though.
I hope this helps you. Thanks!

Ahh ok! That makes sense! Thanks for the info. I'll close this issue then.


Remove KeyboardInputViewController.shared

KeyboardInputViewController.shared was added as a convenience when moving to KeyboardKit 5, to avoid having to change many external interfaces.
However, when implementing multi-platform support, many parts of the library must be disabled for macOS and watchOS, since the shared controller doesn't exist on these platforms.
Over time, after releasing to KeyboardKit 6, limit the usage of KeyboardInputViewController.shared and enable the affected types for more platforms.
Eventually, if possible, KeyboardInputViewController.shared should be removed in KeyboardKit 7.

This has been pushed to the kk7 branch.


[MACHINE LEARNING] Using C++ to create an Intelligent android app

Topic Suggestion
Writing sample(s):
Samples.
Proposal Submission
Proposed title of article
Using C++ to create an Intelligent android app
Proposed article introduction
We are going to build on this article and learn how to create a native machine learning android app using the NDK. We will look at creating a model, loading a heart dataset, splitting it, and then making a prediction.

I'm still researching on this. More will be talked about. We may also explore the Neural Networks API.

Key takeaways

Creating a model using the JNI
Loading a dataset using the JNI
Processing the dataset and checking how to set random seeds to preserve the states using C++
Making predictions from the dataset

Article quality
It is easy to follow through, and this is not much explored.
References
N/A

Sounds like a helpful topic - let's please be sure it adds value beyond what is in any official docs and/or what is covered in other blog sites. (the articles should go beyond a basic explanation - and it is always best to reference any EngEd article and build upon it). @auxesis
Please be attentive to grammar/readability and make sure that you put your article through a thorough editing review prior to submitting it for final approval. (There are some great free tools that we reference in EngEd resources.)
ANY ARTICLE SUBMITTED WITH GLARING ERRORS WILL BE IMMEDIATELY CLOSED.
Please be sure to double-check that it does not overlap with any existing EngEd articles, articles on other blog sites, or any incoming EngEd topic suggestions (if you haven't already) to avoid any potential article closure, please reference any relevant EngEd articles in yours. - Approved


ZHA: aqara magnetic door sensor aq2 changes zigbee type

After pairing the lumi.sensor_magnet.aq2 with ZHA, it consists of a thermometer (sensor.lumi_lumi_sensor_magnet_aq2_0321adb5_1_1026) and a binary_sensor (binary_sensor.lumi_lumi_sensor_magnet_aq2_0321adb5_1_6). Then after a restart of HA (or after one day without restart, not sure) the thermometer is lost and the binary_sensor became a switch (switch.zha_0321adb5_1_6). But the switch still reflects the state of the binary_sensor.

Are you still running locally modified code? If so, it’s going to make this tough to reproduce. I have a bunch of these and I’m not experiencing this. I’m going to need screen shots of the before and after for the device card in the zha config panel to start and the signature for the device. The temperature sensors were purposefully removed from these because they aren’t accurate and xiaomi’s own hub doesn’t use it.

a screenshot like this? (but it's from 2 different sensors)
and yes I have partially modified code. what USB stick do you use?

HUSBZB-1 for the stick, that should be irrelevant though. Your issue is that the quirk is applied on one and not the other. The real question is why? Get the signature for the one that doesn’t. My guess is that the “bad” one failed to read model and manufacturer during joining based on the entity ids the sensors have (zha_0321...) after you collect the signature:
Remove the device using the zha config panel. Press the button after issuing the command and watch the logs for the message that the device left the network. Wait about 2 minutes and restart HA. Then pair the device again following the pairing instructions for Xiaomi devices (you have to press the button every few seconds until the device joins). I will say that you’re running an outdated version of HA / ZHA and there were some changes added to improve the Xiaomi pairing experience.

They devices pair fine. Then after a restart the binary_sensor becomes a switch. The first screenshot is before the restart, the second screenshot after restart. Same device.
So that means that the device is not stored correctly in the .storage after shutdown.

It’s losing the quirk application. Manufacturer and model come from the zigbee db and have absolutely nothing to do with anything in .storage. All of this happens in Zigpy at startup before the device is even given to HA. The code is here: https://github.com/zigpy/zigpy/blob/e536d03d94bb020c00e726d71fb0d301d6ad9d53/zigpy/application.py#L54
This spits out matching logs when attempting to apply quirks. You should have a section of logs that explain exactly why the quirk isn’t applied.

When I stop HA, sometimes I see an exception in SQL handling (sqlalchemy) telling that the database was opened in a different thread then the one writing to it. But its not showing which HA component is causing this.
These are the endpoints in zigbee.db:

The clusters:

The attributes:

I don’t know what to tell ya. It’s obvious that the quirk isn’t applying. There should be logs that state why. All of this happens outside of HA before the device is handed off to HA. The quirks matching logs are where I would start. As for your db issue... have you modified anything there?  That SQL error is concerning but I’ve never seen it and no one else is reporting it.

switch (switch.zha_0321adb5_1_6)

Judging by the new entity id, it lost model and manufacturer, but it is in the zigbee.db check the logs during zha startup, it would tell you why the quirks are not applied.
No other component but zha (zigpy) uses the Zigbee DB. Are you sure it is not because you opened database in SQL browser?

the logs during startup:
2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 5 value: b'lumi.sensor_magnet.aq2' 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 1 value: 3 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 7 value: 3 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 0 value: 1 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 65281 value: b'\x01!\xd1\x0b\x03(%\x04!\xa8\x13\x05!\xdf\x00\x06$\x01\x00\x00\x00\x00\n!\x858d\x10\x00' 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 0 value: 1 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 0 value: 0 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 0 value: 100 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 7 value: 200 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 0 value: 1008 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 0 value: 2504 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 33 value: 160 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 32 value: 29 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 65281 value: b'\x01!\xc7\x0b\x03(#\x04!\xa8\x13\x05!a\x00\x06$\x05\x00\x01\x00\x00\n!\xdcdd\x10\x01' 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 3 value: 11272 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 4 value: 47324 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 0 value: 4092 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 32 value: 28 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 0 value: 3886 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 0 value: 2555 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 33 value: 145 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 0 value: 0 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 0 value: 0 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 0 value: 254 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 32 value: 27 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 33 value: 130 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 1291 value: 4 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 3 value: 24939 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 4 value: 24701 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 7 value: 370 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 32 value: 28 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.appdb] Attribute id: 33 value: 200 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Considering <class 'zigpy.quirks.smartthings.SmartthingsMultiPurposeSensor'> 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Fail because endpoint list mismatch: dict_keys([1]) {1, 2} ... 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Considering <class 'zhaquirks.xiaomi.aqara.magnet_aq2.MagnetAQ2'> 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Fail because endpoint list mismatch: dict_keys([1]) {1, 2} 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Considering <class 'zhaquirks.xiaomi.aqara.motion_aq2.MotionAQ2'> 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Fail because endpoint list mismatch: dict_keys([1]) {1, 2} 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Considering <class 'zhaquirks.xiaomi.aqara.motion_aq2b.MotionAQ2'> 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Fail because endpoint list mismatch: dict_keys([1]) {1, 2} 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Considering <class 'zhaquirks.xiaomi.aqara.remote_b286acn01.RemoteB286ACN01'> 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Fail because endpoint list mismatch: dict_keys([1, 2, 3]) {1, 2} 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Considering <class 'zhaquirks.xiaomi.aqara.sensor_swit.SwitchAQ3V2'> 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Fail because endpoint list mismatch: dict_keys([1]) {1, 2} 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Considering <class 'zhaquirks.xiaomi.aqara.sensor_switch_aq3.SwitchAQ3'> 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Fail because endpoint list mismatch: dict_keys([1]) {1, 2} 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Considering <class 'zhaquirks.xiaomi.aqara.switch_aq2.SwitchAQ2'> 2019-08-04 15:10:34 DEBUG (MainThread) [zigpy.quirks] Fail because endpoint list mismatch: dict_keys([1]) {1, 2}

What version are you running? This is not the latest upstream. If you want to report an issue you would need to reproduce it with the upstream version. The easiest way to ensure there's no other "versions" interference is to run HA in docker with official image

Ok I understand.
So I have to close this?

You don’t “have” to do anything... it’s just nearly impossible for us to help you when we’re trying to diagnose issues with a system that isn’t in a known state. so, you have .94 of HA modified, the newest Zigpy and quirks 0.0.19? Are you sure you’ve always had the venv active when installing libs?

Upss...wait...I have one zigpy package from pip3 and one within HA and they are not the same version?

That’s also concerning...
I’m trying to help, but I really don’t want to chase something that may not be an issue... why not create another venv and start it using the dB and config from this one? That should help you determine if the venv you have is messed up or if the issue is localized to your custom setup.

I have no venv, and I uninstalled zhaquirks and zigpy from pip3. They were not used anyway. So I don't have the newest zigpy and quirks. Now I have .94 with bugs in zigpy which are fixed meanwhile, yes?
Is it correct that during startup, zigpy expects 1 endpoint for the aqara device, but it finds 2 from zigbee.db so it does not apply aq2 quirks?

That is a good question. The device only has one endpoint. Your DB screenshot only showed 1 Endpoint... so the log doesn’t correlate to the DB which is odd.

That was the wrong device. The reason is that the model is not matching. The device has model 'lumi.sensor_magnet.aq2', but the signatur is model None. In https://github.com/zigpy/zigpy/blob/dev/zigpy/quirks/xiaomi/init.py, there is no model defined for AqaraOpenCloseSensor.

Huh? The signature is only model none of something screwed up the dB or the device didn’t pair correctly. Your screenshots clearly show the IEEE for the device and that data in the database so I’m not sure what you’re trying to say...

It is this code in zigpy/quirks/init.py getdevice():
       if not all([device[eid].model == sig[eid].get('model', device[eid].model) for eid in sig.keys()]):             _LOGGER.debug("Fail because model mismatch on at least one endpoint")             continue
The loop over device[eid] is wrong, it should be a loop over endpoints. But "eip" is the index into the signatures attributes.
But ok this is old code, in the current zigpy version is it completly different.

If that were true your screenshots (which clearly show a match previously - included again below so you can see the one I mean) would be impossible

and we would have had a ton of reports of this not working in .94 which we didn’t. At this point I don’t think I can help with this “issue” any longer.

The device pairs fine but when restoring from zigbee.db, the model ist lost so the quirks is not applied.
But I see now that the current code is completely different and the model moved from the endpoint to the device. So we cannot do much here.

Your log shows quirks matching for a device with two endpoints. But your DB snapshot clearly shows only a single endpoint. Which one is wrong? DB snapshot also clearly shows manufacturer and model attributes in the database, but it fails to load it?
This is quite severe bug and I'm sure we would have heard from others if that was the case, but ATM everything points it to being specific to your environment.

earlier you mentioned warnings about concurrent access to zigbee db from different threads. This could cause the issue you are seeing if zigpy fails to save manufacturer/model to DB. You would need to figure out that 1st. But on the flip side, your DB screenshot showed that data was saved successfuly, so it is not clear.

Remove the device from HA (to force full discovery) and join it again. Make sure the database is not accessed by anything else, like sql browser etc.
take a note during the discovery the number of endpoints being detected and whether each endpoints reads successfuly model & manufacturer info
Make a copy of  zigbee.db
run sqlite zigbee-db-copy.db .dump | grep 21:ad:d5 and confirm it saved the information for all endpoints

Hello, thank you for your advise. The log output was wrong, it was not from the aqara device, it was some other device. But you cannot see this in the log.
The problem was in appdb.yp this code:
                        if cluster == Basic.cluster_id and attrid == 5:
                            if isinstance(value, bytes):
                                value = value.split(b'\x00')[0]
                                dev.model = value.decode().strip()
                            else:
                                dev.model = value

This is the up to date version and it fixed the problem that sometimes there is a type mismatch or in the name there are hex '0'. I adapted the above code and now it works fine.
(in the old version the condition "if bytes or not" was missing)
Thank you again.

you must have a "soup" of zigpy versions. HA 0.94.x uses zigpy-homeassistant==0.3.3 and that version of zigpy stores model & manufacturer attributes as bytes and https://github.com/zigpy/zigpy/blob/f66500ba612d8193dddb57bf2b2f2fed35739a61/zigpy/appdb.py#L225-L230 splits the bytes by NULL char and decodes it as ASCII. This definitely would be reported by others as that particular sensor is quite popular and used by many. I also have this sensor and did not have any problem with it during 0.94 version development.

I had zigpy-homeassistant 0.7 installed with pip3. I thought installing newer packages with pip3 would fix bugs in older versions. I removed 0.7 now.


[FLINK-5160] Fix SecurityContextTest#testCreateInsecureHadoopCt on Wi…

This PR resolves in issue in the SecurityUtilsTest class, which tries to call getUserName() on com.sun.security.auth.module.NTSystem. However, as documented here this method does not exist; on Windows it is called getName().

merging


Document.execCommand() marked obsolete

Request type

[ ] Please close this issue, I accidentally submitted it without adding any details
[ ] New documentation
[x] Correction or update

Details
HI, I wonder why Document.execCommand() is marked obsolete. I've tried to find the source of this info and can not find any confirmation that it is obsolete.  Plus, a lot of MDN articles refer to  execCommand without a mention that it is obsolete. Can you please clarify if it is ok to so still use it or not?
Link: https://developer.mozilla.org/en-US/docs/Web/API/Document/execCommand

Hey, thanks for your issue.
It was marked obsolete in https://github.com/mdn/browser-compat-data/pull/5518 by @sideshowbarker.
See also the big red banner here https://w3c.github.io/editing/docs/execCommand/


How is v0.33.0 supporting Vs2022 as mentioned in the release notes?

Hi,
thank you for the new release v0.33.0!
As a git tfs user, I was pleased to find out that a new v0.33.0 came out, reading in the release notes that support for VS2022 has been added. Unfortunately, this isn't quite the case, as far as I remember, the distribution needs to contain a corresponding GitTfs.Vs20xx folder, for 2022 there is none. This is the output of git tfs version with the old GIT_TFS_CLIENT setting pointing to 2019, it again claims that vs2022 is supported:
$ git tfs version
Found matching Visual Studio version at C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional
git-tfs version <IP_ADDRESS> (TFS client library <IP_ADDRESS> (MS)) (64-bit)
C:\tools\gittfs\git-tfs.exe

Note: If you want to force git-tfs to use another version of the tfs client library,
set the environment variable `GIT_TFS_CLIENT` with the desired version (ie: '2015' for Visual Studio 2015,...)
Supported version: 2022, 2019, 2017, 2015

Setting GIT_TFS_CLIENT to 2022 will give this error:
git tfs fetch
Unable to load TFS version specified in GIT_TFS_CLIENT (2022)!
Could not load file or assembly 'GitTfs.Vs2022' or one of its dependencies. The system cannot find the file specified.

which seems quite logical, since there is no such file GitTfs.Vs2022.dll on my entire PC.
Is there some VS2022 component that can be added to the VS2022 installation, or how to solve this?
Best regards,
Lucian

We forgot to update the build script so the VS2022 was not packaged 😭 Sorry about that!
I just fixed in with #1480
You could dowload the dev version artifact with VS2022 support here: https://ci.appveyor.com/project/pmiossec/git-tfs-v2qcm/builds/49355018/artifacts (that's the only difference with the v0.33.0 released).

Thanks Philippe for the quick response (also @choco 👍🏻) and help, it works!

Fixed in v0.34


custom network connection error

I'm running nginx-proxy to automatically config a reverse proxy.
One requirement is defining a network for each service.
web:
  restart: always
  build: .
  container_name: mysite
  command: npm start
  links:
   - db
  ports:
   - "3000:3000"
   - "5858:5858"
   - "35729:35729"
  environment:
   - NODE_ENV=development
   - VIRTUAL_HOST=mysite.local
  volumes:
   - ./:/opt/mean.js
   - /opt/mean.js/node_modules
   - /opt/mean.js/public
   - /opt/mean.js/uploads
  net: my_network

db:
  image: mongo:3.2
  container_name: mysite_db
  restart: always
  ports:
   - "27017:27017"
  volumes:
   - db:/data/db
  net: my_network

The problem comes when my web container isn't able to connect to my db container.
Could not connect to MongoDB!
What am I missing?

I'm assuming your referring to Docker containers?

Yes

@soupman99 I haven't tested the docker-compose file with a pre-configured network setup so I don't know how it affects the containers but you should check a few things to see where the problem is: In the meanjs web container - what environment variables do you have? you should have something like DB_1_PORT_27017_TCP_ADDR1  which will map to the network IP address of the db container

@soupman99 can this issue be closed?

@soupman99 can this issue be closed?


Can't get to work addError()

const a = form.querySelector('input[name="email"]'); pristine.addError(a, 'asd');
It gives me:

Uncaught TypeError: Cannot read property 'push' of undefined
at f.addError (pristine.min.js:1)

How should i use addError()?
Thanks!

Got stuck on same problem. But there is a (dirty) solution:
Problem
Pristine creates an object property on input that you pass as addError argument. That object has errors array, or at least it should have, but it's not initialized.
Solution
Manually initialize errors array before using addError
Sample implementation
const a = form.querySelector('input[name="email"]');
if (a?.pristine?.errors === undefined) a.pristine.errors = [];
pristine.addError(a, "asd");

Thanks, it works!
I think that  this is a bug that needs to be fixed by simply initialiting the errors array on creation.

Yeah, it definitely should be fixed "inside" Pristine, not like this, but for now it gets the job done 😄


[SchemaRegistry] AvroEncoder design

include Josh's design + discuss with Adam about his ideas
discuss with architects

original gist with feedback here: https://gist.github.com/swathipil/f3a5b7f9b224f652403866261990fa22

re-design here: https://gist.github.com/swathipil/9f835a8006932766f67c36189433a822


Add support for brotli encoding

Brotli encoding is a relatively new compression algorithm which may be better than gzip, and is supported by all major browsers according to Wikipedia. This patch adds support for ecstatic to serve brotli-encoded files (using the .br extension) if the browser supports it.
Some things to note:

Firefox and Chromium, and probably other browsers, only support brotli when using https, except that Chromium allows it from localhost, so it can be tested by running locally.
I am unaware of any method for testing if a file is in fact brotli-encoded (as opposed to simply having a .br extension) like the way we can check for gzip encoding. If anyone knows how to do this, it should be added.
As I've written it here, ecstatic by default tries to serve brotli, next tries gzip, then falls to un-encoded. I've set brotli on by default and ahead of gzip because it looks like brotli performs better (see article linked above).

@thornjad pretty sure adding fs.stat calls before responding will make ecstatic slower, regardless of the optimized encoding. Perhaps this should be opt-in?

ahh you did: https://github.com/jfhbrook/node-ecstatic/pull/228/files#diff-b2b5a88fb51675f1aa1065c093dce1eeR451

I believe the stat call currently happens with gzip, fwiw

possibly relevant https://github.com/jfhbrook/node-ecstatic/pull/195/files

How does this look to everyone? Are there any objections?

No immediate concerns, but I'd like to take a closer look and kick the tires on this just the same

Documentation added! Thank you for reviewing!

Very weird. Yes I can reproduce this on Linux, I'm going to see if I get the same on an entirely fresh install on Windows. The weird part is that I don't get the same test failures on my fork.

ok word so it's probably not a windows-specific issue. You don't have to try to repro on windows specifically in that case, imo.
and yeah, frickin' weird right??

Ready for a new kicker? I just pulled master into the master on my fork, and? No error. I'm going to try to investigate this now if I have enough time.

wondering if it has to do with line endings

y'know \n vs \r\n, git pulls some shenanigans on text files to make them happy this way

also, check for accidentally uncommitted things, maybe you have a delta or a new file somewhere

With some experimentation, it looks like in those two test cases, where brotli is not accepted and where brotli is not enabled, it isn't falling back on gzip like it should. For example, when I go to test/public/brotli in firefox (which doesn't support brotli without https), I get the uncompressed index file.
As for differences, I used cat lib/ecstatic.js | grep "^M" and found nothing, and diff and comm find no differences with my fork. I can look more tomorrow, I really don't know what's up but really want to find out.

Possible there's a race condition?

I've found the problem, and it's frustratingly simple! Both of those tests are attempting to serve test/public/brotli/index.html.gz, which exist in my local fork. However, *.gz is in the .gitignore, so those local files were never even tracked. Since I added the entire test/public/brotli directory at once, I didn't even notice. Getting rid of that line in the .gitignore fixes it all!
Because of #230, how should I submit this change? Should it be a new PR?


delete operation throws build error

I created a generic repository that gets a PersistentContainer injected (I followed you instructions up to there):
public class BaseRepository<T>: Repository where T: ManagedObjectConvertible {

	public var persistentContainer: PersistentContainer

	init(persistentContainer: PersistentContainer) {

		self.persistentContainer = persistentContainer
	}

	public func allItems() throws -> [T] {

		try self.persistentContainer.perform { context in

			try context.fetch(T.all).map { try $0.decode() }
		}
	}

	public func create(_ item: T) throws {
		try self.persistentContainer.perform { context in

			try context.insert(item)
		}
	}

	public func remove(_ item: T) throws {
		try self.persistentContainer.perform { context in

			try context.delete(item)
		}
	}
}

However, the "remove" function where the delete operation on the context is used, throws a build error:
Cannot convert value of type 'T' to expected argument type 'ManagedObject<PlainObject>'

I tried around, but I cannot figure out how to fix this. Do you have any ideas?
The calling part looks like this (a repository factory):
	public func makeRepository<R>(for storyType: StoreType) -> R where R: Repository {

		switch storyType {
			case .country:
				return BaseRepository<Country>(persistentContainer: persistentContainer) as! R

			case .location:
				return BaseRepository<Location>(persistentContainer: persistentContainer) as! R
		}
	}

where
public protocol Repository {
	associatedtype T: ManagedObjectConvertible

	var persistentContainer: PersistentContainer { get }

	func allItems() throws -> [T]
}

Thanks for your support!

Thanks for your explanation. Makes things much clearer to me.
I now changed the repo this way (according to suggestion #1):
	public func remove(_ item: T) throws {
		try self.persistentContainer.perform { context in

			guard let uuid = item.uuid else { return }

			try context.delete(request: T.all.where(\T.uuid == uuid))
		}

I'm calling that from a test:
	func testStore() throws {

		let store = Store(type: .inMemory,
						  seeder: nil)

		let repo: BaseRepository<Country> = store.makeRepository(for: .country)

		let allBefore = try repo.allItems()
		XCTAssertEqual(allBefore.count, 0)

		let new: Country = .init()
		try repo.create(new)

		let allAfter = try repo.allItems()
		XCTAssertEqual(allAfter.count, 1)
		XCTAssertEqual(allAfter[0], new)

		try repo.remove(new)

		let allRemaining = try repo.allItems()
		XCTAssertEqual(allRemaining.count, 0)
	}

This fails with a fatal:
Thread 1: Fatal error: unsafelyUnwrapped of nil optional

that happens here:
extension ManagedObjectConvertible {
    static func attribute(_ keyPath: PartialKeyPath<Self>) -> Attribute<Self> {
        self.attributes.first(where: { $0.keyPath == keyPath }).unsafelyUnwrapped
    }
...

because self.attributes.first() returns nil which obviously cannot be unsafelyUnwrapped.
Am I still doing something wrong? The uuid attribute is optional for the CoreData NSManagedObject, as well as for the domain item; but even it is not for the domain item, the error persists.

Hm, checked it, but it seems to be as required - or do I miss something:
public struct Country: Codable, Equatable, IdentifiedItem {

	public var uuid: UUID?
	public var name: String
}

extension Country: ManagedObjectConvertible {

	public init() {
		self.uuid = .init()
		self.name = .init()
	}

	public static let entityName = "Country"

	public static let attributes: Set<Attribute<Country>> = [

		.init(\.uuid, "uuid"),
		.init(\.name, "name"),
	]

	public struct Relations {
		let locations = ToManyRelation<Location>("locations")
	}

	public static let relations = Relations()
}

oh, i see - give me some time to investigate the problem, i will return to you soon

Great, thank you!

Well, looks good. Changed my code according to the changes in the pull request, and everything's working fine!
Thanks a lot for your support!

You are welcome 🤝


example broken?

Ben reports the example/simpulation.py is failing. It is working fine for me on OSX. It may be package version dependent?
@bstabler - could you do a pip freeze and capture output and I will see if I can reproduce.

My pandas was 0.17.1 and your pandas is 0.18.  I just upgraded to 0.18 but now pandas is failing with an numexpr import error.  I think the issue is related to https://github.com/pydata/pandas/issues/12489.  I'm investigating.

An uninstall of numexpr and a new install of numexpr 2.5.1 appears to have worked.  I can now successfully run the example.


Support human play

Hi!
As a card game enthusiast and AI researcher, I'd like to play LOCM as human vs human or human vs bot. I've been trying to figure out how to allow human play but haven't reached a solution yet. Simply allowing someone to input the action strings would be enough.
My current tentative is having (i) a server process that reads keyboard inputs and puts them into a shared queue; and (ii) a player process that prints the next queue element (or hangs until there's any). I hoped to see the game state via the browser visualization and act via the server process.
I did remove the time limits, but my custom java referee still gives timeout error to the player process. Do you have any insights of better approaches or why timeout error persists?

Hi @ronaldosvieira!
That makes two of us. It'd be really hard to incorporate it within the CodinGame infrastructure, as the rendering engine expects to get the full replay at once (it allows things like going backwards).
On the other hand, both engines (not referees) don't have such restrictions. I don't know how we could reuse the current implementation of UI to make human play possible, but I'll definitely think about it.
A very simple CLI interface can be made using a basic CLI Nim player. The only currently missing part would be the user input -> action parser.

I see. Thanks!
I'll let you know if I have any progress on this.


fix(appl-883): fixes Search results displayed too closely as though overlapping on search page

This PR resolves APPL-883
Description
Recently we've added nationality / birth year info label to artist search results. The gap between search results became too small when there is such label.

Before adding disambiguating info
Before this change
After

PR Checklist

[x] I have tested my changes on iOS and Android.
[x] I hid my changes behind a feature flag, or they don't need one.
[x] I have included screenshots or videos, or I have not changed the UI.
[x] I have added tests, or my changes don't require any.
[x] I added an app state migration, or my changes do not require one.
[x] I have documented any follow-up work that this PR will require, or it does not require any.
[x] I have added a changelog entry below, or my changes do not require one.

To the reviewers 👀

[ ] I would like at least one of the reviewers to run this PR on the simulator or device.

Changelog updates
#nochangelog
Need help with something? Have a look at our docs, or get in touch with us.

🙏🏽