feat(#10): --json

closes #10

PR-Codex overview
This PR enhances the CLI tool by adding an option to output JSON files along with CSV files. It also introduces a new flag to control the JSON file creation.
Detailed summary

Updated README to reflect changes in CLI tool options
Added print flag in index.js to conditionally output JSON files
Modified file writing logic to handle JSON file creation based on the print flag

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

@rultor merge

@rultor merge

@h1alexbel OK, I'll try to merge now. You can check the progress of the merge here

@rultor merge

@h1alexbel Oops, I failed. You can see the full log here (spent 7min)
up to date, audited 275 packages in 1s

46 packages are looking for funding
  run `npm fund` for details

5 vulnerabilities (1 moderate, 3 high, 1 critical)

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.
\u001b[?25l(⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂) ⠴ idealTree:repo: sill idealTree buildDeps\u001b[0m\u001b[K
(⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂⠂) ⠦ idealTree:repo: sill idealTree buildDeps\u001b[0m\u001b[K
(#########⠂⠂⠂⠂⠂⠂⠂⠂⠂) ⠧ idealTree: timing idealTree Completed in 407ms\u001b[0m\u001b[K
(##################) ⠋ reify:@dprint/darwin-arm64: timing reifyNode:node_module\u001b[0m\u001b[K
(##################) ⠴ reify:@dprint/darwin-arm64: timing reifyNode:node_module\u001b[0m\u001b[K
(##################) ⠦ reify:@dprint/darwin-arm64: timing reifyNode:node_module\u001b[0m\u001b[K
(##################) ⠦ reify:@dprint/darwin-arm64: timing reifyNode:node_module\u001b[0m\u001b[K
(##################) ⠦ reify:@dprint/darwin-arm64: timing reifyNode:node_module\u001b[0m\u001b[K
(##################) ⠏ reify:@dprint/darwin-arm64: timing reifyNode:node_module\u001b[0m\u001b[K

\u001b[K\u001b[?25h
up to date, audited 275 packages in 1s

46 packages are looking for funding
  run `npm fund` for details

5 vulnerabilities (1 moderate, 3 high, 1 critical)

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.

><EMAIL_ADDRESS>test
> grunt --no-color

Running "mochacli:test" (mochacli) task

\u001b[0m\u001b[0m
\u001b[0m  Test case for compiled.js\u001b[0m
  \u001b[32m  ✓\u001b[0m\u001b[90m compiles query from parts\u001b[0m

\u001b[0m  Test case for to-json.js\u001b[0m
  \u001b[32m  ✓\u001b[0m\u001b[90m creates .json file and writes results as JSON\u001b[0m
  \u001b[31m  1) "after each" hook for "creates .json file and writes results as JSON"\u001b[0m

\u001b[0m  Test case for tokens.js\u001b[0m
test.json file saved
  \u001b[32m  ✓\u001b[0m\u001b[90m fetches one token from file\u001b[0m
  \u001b[32m  ✓\u001b[0m\u001b[90m fetches a few tokens from file\u001b[0m

\u001b[92m \u001b[0m\u001b[32m 4 passing\u001b[0m\u001b[90m (20ms)\u001b[0m
\u001b[31m  1 failing\u001b[0m

\u001b[0m  1) Test case for to-json.js
       "after each" hook for "creates .json file and writes results as JSON":
\u001b[0m\u001b[31m     Error: ENOENT: no such file or directory, unlink 'test.json'\u001b[0m\u001b[90m
      at Object.unlinkSync (node:fs:1881:3)
      at Context.<anonymous> (test/to-json.test.js:31:10)
      at process.processImmediate (node:internal/timers:476:21)
\u001b[0m

Warning:  Use --force to continue.

Aborted due to warnings.
container 5b7e1466cdfcb85ea5a83136f84faec066d2750834f0cf53bc8eaa8cb18bf33d is dead
Wed May 22 13:25:18 UTC 2024

@rultor merge

@rultor merge

@h1alexbel OK, I'll try to merge now. You can check the progress of the merge here


selectionRequired: false leads to locked input field

2.07.WIP
multiple lists on one screen,
Everything works using "selectionRequired: true" with the same code
Switchting to selectionRequired: false
Typing in fields with results in unconsistent behaviour. Typing possible in the beginning for a few characters, no results, then input is locked.

Please, check latest version from master branch (2.0.7 RC).

Released v2.0.7. Should be more stable. Sorry


Correction to security advisory (ReDos)

Can we correct the advisories on normalize-url for ReDos vulnerability CVE-2021-33502, to reflect the updated CVE on NIST? The github and npm advisories state that all versions prior to 4.5.1 are vulnerable, but NIST and the release log say that only 4.3.0 to 4.5.0 should be flagged.
Github Advisory: https://github.com/advisories/GHSA-px4h-xg32-q955
NPM Advisory: https://www.npmjs.com/advisories/1755
Ref: https://nvd.nist.gov/vuln/detail/CVE-2021-33502
I don't know if it's a manual process at all. It'd close a lot of vulns for a lot of people if we can get it corrected.

I just contacted<EMAIL_ADDRESS>- let's see if they do update https://snyk.io/vuln/SNYK-JS-NORMALIZEURL-1296539 . CVE on nist.org was already updated: https://nvd.nist.gov/vuln/detail/CVE-2021-33502#VulnChangeHistorySection

It looks like it has been corrected on the github advisory now. Thanks to everyone involved :hugs:

It looks like, that GitHub picked up the change from nist. I still see the wrong version numbers on snyk.

Snyk now also displays a corrected version number range - just a little bit different to the NIST and GitHub one.


carbon-apimgt-6.5.349 Build npm issue

Using

RHEL 7.7
openjdk-<IP_ADDRESS>-2.b14.el7.x86_64
mvn 3.2.5
node-v12.20.1-linux-x64
codebase : carbon-apimgt-6.5.349

While building code I ended up with
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 06:21 min
[INFO] Finished at: 2021-01-08T09:13:26+05:30
[INFO] Final Memory: 371M/1484M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.5.0:exec (npm install (initialize)) on project org.wso2.carbon.apimgt.publisher.feature: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :org.wso2.carbon.apimgt.publisher.feature

Tried different node version like node-v10.13.0-linux-x64
and 'npm cache clean --force' in 'features/apimgt/org.wso2.carbon.apimgt.publisher.feature/src/main/resources/publisher' and tried to rebuild but things are same.
Any suggestion please.

Hi @sarkark ,
Please raise API Manager product-related issues in this repository
Can you please provide the npm build debug file or full error log in the build ?
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.5.0:exec (npm install (initialize)) on project org.wso2.carbon.apimgt.publisher.feature: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]

the above error log only provide the information about the mojo plugin exception and the feature name (Publisher)

Hi @sarkark ,
Please raise API Manager product-related issues in this repository
Can you please provide the npm build debug file or full error log in the build ?
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.5.0:exec (npm install (initialize)) on project org.wso2.carbon.apimgt.publisher.feature: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]

the above error log only provide the information about the mojo plugin exception and the feature name (Publisher)


Generic errors on web UI for Create New Alias, Add Recipient, Deactivate Alias - Insufficient debug info

Behaviour
When I operate on the web interface on aliases and recipients, I get generic errors:

Error
An error has occurred, please try again later

Container logs are showing:

"DELETE /api/v1/allowed-recipients/c499911b-026c-4931-8204-a9a3b1345381 HTTP/1.1" 401 41 "https://anonaddy.mydomain.com/recipients" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36"
"POST /api/v1/recipients HTTP/1.1" 401 41 "https://anonaddy.mydomain.com/recipients" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36"
"DELETE /api/v1/active-aliases/10dfc096-75d5-41b0-b0f1-d80864d34c7c HTTP/1.1" 401 41 "https://anonaddy.mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36"

So apparently the requests are not authorised. But I have no clue how to understand why they are not authorised. I correctly logged into the web interface.
Probably it's something wrong in my set up, but no clue where to look. I'm going crazy, the only options are to randomly change settings. I've also wiped and re-initiated the DB from scratch, to no avail.
I would need some more debug info, but APP_DEBUG=true and I don't know what I can do more.
Steps to reproduce this issue

Log in to the web interface
deactivate aliases
create aliases
add recipients
do stuff and watch errors coming up, but no info on the reason

Image Release: 0.13.9-r0

After tinkering a lot today with different settings, I have realised that the reported errors occur with image tags starting from 0.13.3 upwards.
More specifically, I tested the "create new alias" function on the web UI and it works with image tags 0.11.2 and 0.12.3, while it returns an error with image tags 0.13.3, 0.13.4, 0.13.7 and 0.13.9.
So I am a bit less convinced that this is down to some misconfiguration on my setup.
Hopefully I'll have time to tinker a bit more tomorrow.

No chance. I couldn't find any further clue as to what is causing the issue. I'm providing here a full bug report, in the hope that someone can either point me in the right direction or find a bug in the docker image.
Behaviour
Steps to reproduce this issue

Deploy anonaddy docker image tag 0.13.3 or higher starting from an empty database.
Register on the web UI with the ANONADDY_ADMIN_USERNAME
Verify email address & login
click on "Create New Alias" in Aliases
Click "Create Alias"

Expected behaviour
The alias is created successfully.
Even in case of error, the container logs in debug mode provide some more info on where the error occurred than a simple HTTP code.
Actual behaviour
A red popup appears in the lower right corner:

Error
An error has occurred, please try again later

Configuration

Docker version (type docker --version) : Docker version 20.10.21, build baeda1f
Docker compose version if applicable (type docker-compose --version) : Docker Compose version v2.12.2
Platform (Debian 9, Ubuntu 18.04, ...) : Canonical-Ubuntu-22.04-Minimal-aarch64-2022.08.16-0
System info (type uname -a) : Linux email4ward01 5.15.0-1021-oracle # 27-Ubuntu SMP Fri Oct 14 20:04:20 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
Include all necessary configuration files : docker-compose.yml, .env, ...
config files.zip
deployed with the following commands:

docker stack deploy -c docker-compose.db.single.yml anonaddy_db
docker stack deploy -c docker-compose.anonaddy.single.yml Anonaddy

Docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.9.1-docker)
  compose: Docker Compose (Docker Inc., v2.12.2)

Server:
 Containers: 6
  Running: 5
  Paused: 0
  Stopped: 1
 Images: 38
 Server Version: 20.10.21
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: journald
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: active
  NodeID: k0qsu4qv27ncgiusbpk5fv5sq
  Is Manager: true
  ClusterID: nbuhooquvm4tgvian9voockw2
  Managers: 3
  Nodes: 3
  Default Address Pool: <IP_ADDRESS>/8
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 5
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: <IP_ADDRESS>
  Manager Addresses:
   <IP_ADDRESS>:2377
   <IP_ADDRESS>:2377
   <IP_ADDRESS>:2377
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 1c90a442489720eec95342e1789ee8a5e1b9536f
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.15.0-1021-oracle
 Operating System: Ubuntu 22.04.1 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 1
 Total Memory: 1.92GiB
 Name: email4ward01
 ID: JR2T:77DW:NZPP:IMO3:UY5I:R3ZW:LKBN:ICL6:MASZ:CWBO:OYBA:BUU6
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: true
 Insecure Registries:
  <IP_ADDRESS>/8
 Live Restore Enabled: false

Logs
"POST /api/v1/aliases HTTP/1.1" 401 41 "https://anonaddy.mydomain.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36"

no logs in /docker/data/Anonaddy/data/storage/logs/

Have you read this issue - https://github.com/anonaddy/docker/issues/150
and made sure to add the right value for SANCTUM_STATEFUL_DOMAINS?

@willbrowningme no, I didn't see issue #150.
So I have now set SANCTUM_STATEFUL_DOMAINS=anonaddy.mydomain.com (i.e. the domain name I put in my browser to reach the web UI) and now it works!!! :-)
I have also realised that my APP_URL variable wasn't quite correct. By using a Cloudflare tunnel I reach the web UI by putting https://anonaddy.mydomain.com in my browser; then Cloudflare tunnel acts as a rev proxy and calls the actual web UI at http://localhost:8000. So I changed it with the correct version, but it didn't help without also setting SANCTUM_STATEFUL_DOMAINS. I also set APP_URL with a completely wrong domain, but it didn't seem to make any difference for good or bad. I am not quite sure what APP_URL is needed for.
Many thanks for your help!
I'd suggest adding something to the documentation about SANCTUM_STATEFUL_DOMAINS. Or maybe you're thinking of some fix that doesn't require an additional variable and maybe figures out the correct setup based on APP_URL...

So I have now set SANCTUM_STATEFUL_DOMAINS=anonaddy.mydomain.com (i.e. the domain name I put in my browser to reach the web UI) and now it works!!! :-)

That's strange, SANCTUM_STATEFUL_DOMAINS is already set with: https://github.com/anonaddy/docker/blob/aac8ad22f7a1960eae0ef6f597928a4adc6279b8/rootfs/etc/cont-init.d/13-config-anonaddy.sh#L64
So if ANONADDY_DOMAIN is correct it should work.

@crazy-max
In my case ANONADDY_DOMAIN=mydomain.com.
As per the documentation:

ANONADDY_DOMAIN: Root domain to receive email from

So, in my case I want anonaddy to send emails from @mydomain.com or @*.mydomain.com.
However SANCTUM_STATEFUL_DOMAINS=anonaddy.mydomain.com, which is the FQDN I am using for the web app.
That's because I have to point my browser to https://anonaddy.mydomain.com to log into the web UI, not to https://mydomain.com.
So I think this explains.
In my case I didn't want to point my browser to https://mydomain.com (i.e. ANONADDY_DOMAIN) to reach the web UI.
With the test domain I am curently using to test & tinker with Anonady, https://mydomain.com is already being used for another web site.
But even with a production domain, I am not sure I would always want to have Anonaddy's web UI to respond on the root domain (like in https://mydomain.com), since it could also be taken by something else.

That makes sense. Wonder if we should not infer SANCTUM_STATEFUL_DOMAINS from APP_URL instead.

@crazy-max I think that would make a lot of sense.

@crazy-max thanks a lot!
I positive and negative tested image tag 0.13.11 only playing with APP_URL and not setting SANCTUM_STATEFUL_DOMAINS.
It seems to be working as expected.
APP_URL needs to include the correct port. E.g. if you set APP_URL=http://anonaddy.mydomain.com but then you reach the web app by http://anonaddy.mydomain.com:8000 you'll get the errors mentioned above. You'll need to set APP_URL=http://anonaddy.mydomain.com:8000 to make it work in this scenario.
On the other hand, it doesn't seem to matter whether you set APP_URL=http://anonaddy.mydomain.com but you reach the web app by https://anonaddy.mydomain.com. Or viceversa. The web app will work ok regardless of http or https.
But you do need to put either http:// or https:// at the beginning of APP_URL. Setting APP_URL=anonaddy.mydomain.com when the web app is reachable by http(s)://anonaddy.mydomain.com will make those red box errors pop up again.
Overall, this all makes sense to me. By following the documentation and the examples it should be sufficiently intuitive to get things right.

I needed to remove the http from my APP_URL to get it working without explicitly setting SANCTUM_STATEFUL_DOMAINS.


How do you detect a pinch and zoom?

I don't mean any region change, but specifically if a user uses pinch and zoom.
the only handler I can see that fires on pinch and zoom is onRegionChangeComplete. There seem to be user-driven handlers for onPanDrag, and onPress, and onLongPress. Is there a most appropriate handler for pinch/zoom? Can you perhaps do something with one of these other handlers to know that was a user initiated Pinch and Zoom?

try checking for change in delta in onRegionChangeComplete ?

@gilbox This would not work for people who are using methods like animateToCoordinate since methods like that invoke onRegionChangeComplete  too.
I would also like to have event that invokes only when human has actually zoom or pinched the map :)

+1
This feature'd be great. I'm using custom image markers, and i'd like to resize them according to user zoom in /out.

@seppemarotta did you figure out how to make the markers zoom with the map by any chance

much like this:  https://github.com/airbnb/react-native-maps/issues/487 to my understanding this is simply not supported at the moment

@fmelp
I'm using this
this.map.fitToSuppliedMarkers([id], animated);
Where id is the id of the Marker and animated is a boolean.

+1

@fmelp , hey can you express in a bit more detail what you're doing? I am looking for something like a handler that fires when the user attempts to pinch and zoom. Is there a way to use this in that way?

Has this been solved? can we detect when a user explicitly pinches and zooms?

@the-simian I use the onPanDrag prop — works great. Pretty sure it also covers explicit pinch & zoom, as well as — obviously — panning.

+1

Any Update? Need Help.

+1

Any updates on this one? Want to be able to differentiate between a user moving on the map and using the animateToCoordinate method. Using onPanDrag required scrollEnabled to be disabled on iOS, so we weren't able to use that.

You can try using https://github.com/kmagiera/react-native-gesture-handler

this work with IOS, but android not fired this events(

Anyone has a solution already for this ? I tried to detect double tap by assuming that when you press on map, onPress will be triggered but it also does not work.

also looking for a solution to this. I need to know when a user pinch zooms to render specific markers.

@the-simian I use the onPanDrag prop — works great. Pretty sure it also covers explicit pinch & zoom, as well as — obviously — panning.

where is the zoom attribute in the onPanDrag callback? I tried calling getCamera inside but it was way too slow to call it for every single onPanDrag event.

also looking for a solution to this. I need to know when a user pinch zooms to render specific markers.

@the-simian I use the onPanDrag prop — works great. Pretty sure it also covers explicit pinch & zoom, as well as — obviously — panning.

where is the zoom attribute in the onPanDrag callback? I tried calling getCamera inside but it was way too slow to call it for every single onPanDrag event.


rkt 0.7.0 fetch fails when an ACI compressed with xz

rkt fetch/run always fails when an ACI compressed with xz.
$  rkt version
rkt version 0.7.0
appc version 0.6.1

$ sudo rkt --insecure-skip-verify fetch test.aci
error copying image: exit status 1

I found a commit that caused this issue using git-bisect.
https://github.com/coreos/rkt/commit/7fbfb358d46084d8a3ca93365a208eb79cf2003e
This commit adds code that calling aci.ManifestFromImage before calling WriteACI.
ManifestFromImage reads ACI file partially.
Currently, XZ decompressor implementation is executing xz command at goroutine.
ManifestFromImage partially reads ACI file.
But xz command is still executing after ManifestFromImage exited.
Therefore, file position is changed by xz command that was executed at ManifestFromImage, and cannot read begining of file at WriteACI.

@kazuki Thanks for your report and analysis! I can reproduce this easily with a quite big ACI having the manifest at the start of the tar as this helps keeping more than one xz process running.
I'm testing some patches (for both appc/spec and rkt) that should fix this. I also discovered another bug while looking at this. I'll link the related PRs here when ready.

Blocked on #1272


Add -n flag to govendor sync

Instead of actually pulling in the vendored updates, when the -n flag is specified govendor sync will only show you what's outdated compared to upstream without updating anything locally.  This will allow a developer to cherrypick upstream updates without syncing the entire tree.

In hindsight, I could have probably omitted status and just have sync -n.

!  Actually, I think we talked past each other on this one: I was hoping govendor sync -n would do the network IO to figure out what is out of date and only print something if it is out of date.  For instance, running:
$ govendor sync -n
fetch "github.com/armon/circbuf"
fetch "github.com/armon/go-metrics"
fetch "github.com/armon/go-radix"
fetch "github.com/bgentry/speakeasy"
fetch "github.com/hashicorp/errwrap"
fetch "github.com/hashicorp/go-msgpack/codec"
fetch "github.com/hashicorp/go-multierror"
fetch "github.com/hashicorp/go-syslog"
fetch "github.com/hashicorp/go.net/internal/iana"
fetch "github.com/hashicorp/go.net/ipv4"
fetch "github.com/hashicorp/go.net/ipv6"
fetch "github.com/hashicorp/logutils"
fetch "github.com/hashicorp/mdns"
fetch "github.com/mattn/go-isatty"
fetch "github.com/miekg/dns"
fetch "github.com/mitchellh/cli"
fetch "github.com/mitchellh/mapstructure"
fetch "github.com/ryanuber/columnize"
fetch "golang.org/x/sys/unix"

That output isn't terribly useful beyond telling me what needs to happen, but I could do that with something like govendor list -no-status +v | xargs -n1 printf "govendor fetch %s\n".  The ideal is to have govendor sync -n pull down the sha and compare it with what's in vendor.json and emit something if the version is newer.  For a project that is tracking master I expect this to change constantly, but for something that's tracking a branch, I wouldn't expect this to change unless someone has committed to a branch.
Make sense?

Are you asking for something similar to: https://github.com/kardianos/govendor/issues/122 ?
Because "sync" doesn't update packages, it only pulls down existing packages at the existing revision if they don't match the checksum (modified or missing).

@kardianos Oh... hrm.  I thought govendor fetch pulled down what was upstream ala go get ..  I was using govendor sync -n to find a list of what to update and then pulling each update via govendor fetch pkgspec, but that doesn't look like it was doing what I thought it was doing.

There is currently no way in the govendor tool to determine which libraries could be updating, without actually updating them with govendor fetch. (issue #122)
govendor sync is designed primarily for users who do not wish to store vendor source code in the same repo, and thus need a way to pull down the exact revision (git hash) stored in the vendor.json file.
govendor sync never touches remotes to determine if something needs to be pulled down. It only uses the stored checksum in the vendor.json folder and compares that with the computed checksum (SHA1 of filename, import path, file content). This way both missing and manually modified code will be discovered.
Your above command will update GOPATH, but it won't update do a proper comparison. There is currently no command that will do that. Again, #122 should be resolved for this. If you have a preference for a UX for the command to do that, please comment there.


dependencies missing in environment

We need to keep the environment.yml updated for dependencies so devs have a unified env.
ImportError: Unable to find a usable engine; tried using: 'pyarrow', 'fastparquet'.
pyarrow or fastparquet is required for parquet support

If the install needs more than conda, we should document that as well.

ah, i get confused about which is authoritative. It's in requirements.txt but not environment.yml

i think this was resolved with 88000b642a5524ba30e9ad6d18f87a5c7e80fd85


Removed slider component

Closes #82 #108
This component is no longer used in the builder as it has been replaced
by a new JS component.
CR @piensaenpixel cc @xavijam

ping @piensaenpixel

🇪🇸


scrollMessages quite buggy on Safari

This seems to have to do something with a filled localStorage. After clearing my localStorage, I'm not able to reproduce this.

The scrollMessages is very buggy when there are a lot of messages in the chat window. We should limit the amount of messages rendered in this window


Add role columns from User Registration List to the User model

Completion Criteria:

[ ] User model has fields for eligible_sponsor and eligible_manager
[ ] User model is populated when user registration csv is loaded
[ ] Factory for user has the ability to create eligible_sponsor and eligible_manager users

Hey team! Please add your planning poker estimate with Zenhub @bess @hectorcorrea @JaymeeH @jrgriffiniii @leefaisonr


[2862] Get protocol from client config base url

This PR is for issue 2862.

Fixes #2862

Thanks. I will start looking at the test code tonight.

fwiw you can test http directly by directly hitting the controller.

@rabbah how can I determine the controller hostname from within tests?

https://github.com/apache/incubator-openwhisk/blob/master/tests/src/test/scala/common/WhiskProperties.java line 229

@jsanda, as pointed out above, you can use WhiskProperties.getApiHostForClient().

I have been away for the past several days. I will update this PR tomorrow.

Closing this one as the CLI has moved to incubator-openwhisk-cli and incubator-openwhisk-client-go.


Error in Installing Bi-Mamba layer

Hi, when I install the environment at step 4, there comes the error as below:

Would please help me?

It seems that there is a problem with the ninja version. You can try reinstalling it with the following command.
pip uninstall ninja
pip install ninja==<IP_ADDRESS>

It seems that there is a problem with the ninja version. You can try reinstalling it with the following command. pip uninstall ninja pip install ninja==<IP_ADDRESS>

谢谢回复，现在环境安装成功了，但是在跑VRDS测试的时候又报了下面的错误：

能麻烦再帮忙看下吗？

第一个问题可以参考这个回复 #6 第二个图中框出的警告不影响结果

非常感谢，可以跑起来了。


VictimTracker - victims.json

Hello
I banned a guy but VictimTracker does not pop up for me
Do I have to change the victims.json? I tried to put in steamID but did not worked..
my victims.json looks like:
{"victims":[]}
can som1 help me plz? :)
thx

can u please make a release fix?
I tried to install from source but I cant open Titan
PS installin everything and says 'titan is rdy to use' but it does not open

@Marc3842h  When I try to open the solution file I get the following message: One or more projects in the solution were not loaded correctly. Can you please help me?

@fiskerface Until the feature is stablized, a official release won't be published on the releases tab. Some people may be friendly however and send you their own prebuilt versions of the latest Titan source code.
@45656456kawai Not yet as I'm unable to track progress on that issue as nobody has bothered to open a GitHub issue regarding it.
@trillex1337 Please read the build guide before asking for any help, as you wouldn't have tried to open the solution file if you would've read it.
Existing issues are not a place for asking for help. I will be locking this conversation to prevent more questions. If you find another issue or have more questions, please open a new issue.


First integration to the IATI Validator

First integration to the IATI Validator
See https://api-doc.dataworkbench.io/?version=latest for latest docs.
Objective: use IATI Validator to spot errors prior to parsing.
User story 001
iati.cloud picks up publishers and XML data from Registry. A new intermediate function will kick in which will look at the IATI Validator for any errors. If a critical error is returned by IATI Validator no parsing will be performed.
User story 002
iati.cloud performs US001, parsing commences. Once activity is stored in iati.cloud, the feedback from IATI Validator are stored in db and made accessible via the activity end-point.

https://api-doc.dataworkbench.io/?version=latest


Sitemap has links for not existing content pages

The automatically created sitemap at https://manjaro.org/sitemap.xml since #42 contains some links to pages, that don't exist:

These are markdown files in the content directory but as they are not linked and document driven mode is deactivated the pages are not generated (and also not meant to exist).
This could be a bug in the Sitemap Nuxt module.

@nurRiyad, do you have an idea if this is a bug in the Nuxt module or there is some configuration issue with our page?

I will check. Give me some time. Hopefully in the next 48h

There is a way to exclude content folder from generating sitemap. I use that and exclude the /products/device/** not to add in the sitemap. Fixed #45

Fixed by #45.


Update ToggleDarkMode.svelte

Light mode / Dark mode doesn't get saved #6
fix / cleaning required
sorry but forget to remove console.log

sorry for late reply but can i ask what test this code is failing upon it is working fine on my browser.
Thank you

and i also fix the padding issue because there is a change in padding while toggle

https://user-images.githubusercontent.com/43876655/154859433-311b3307-510b-4685-9f23-ce051d625a4b.mp4

If I checkout your branch and run yarn run build it's failing due to compiling issues. Also, if you look at your changed files, you see that your function setMode never gets called.
I fixed this.

i dont know i tried and test on chrome firefox and opera work as suppose to.How did you fixed it ?

Take a look at my Commit I call setMode() and remove some unnecessary brackets.

sorry for keeping comment bracket were inuse that was a anonymous function it will run on every re-render.
last comment


Add additional logging around search_controller's ES query building

Problem

Currently we only log the query itself, but not the process that led to the query being constructed. We are seeing queries get sent with sizes of up to 1800 but have no way of telling why 1800 was chosen for the size.
Description

Add logging to the _get_query_slice method. Log at each branch and be sure to log the variables that lead to each branch as well. The idea is to log as much as is necessary to understand how the method is working in production to produce the from and size results that are being sent to ES.
In addition, log some basic search facts like the term and the serializer data.
Be sure to introduce a "trace" variable so that we can easily follow a particular search request. This can just be a uuidv4 generated at the top of the search method and passed around and added to the logs.
Implementation

[ ] 🙋 I would be interested in implementing this feature.

Fixed by https://github.com/WordPress/openverse-api/pull/777

Re-opening for consideration, since some of the logging in #777 had to be reverted while triaging the API stability. Does it make sense to try to reintroduce some of the logging before revisiting the search controller refactor?

Agreed, Zack, that we should add the logging back to the search controller.

@WordPress/openverse-api Does this issue still make sense to implement as it reads today? I'm wondering if we still find value in this particular approach or if we'd want to do something more like labelling specific queries and making it possible to read specific query type times and such. Given we've already identified the most time-intensive parts of the search controller (dead link filtering and very deep pagination), should we add logging around that kind of thing specifically rather than more general logs?

I think we can lower this issue's priority and look into Kibana, which should have everything built-in to see what is happening inside Elastichsearch. The additional logging around dead link filtering and pagination sounds good to have in the short term as well.

which should have everything built-in to see what is happening inside Elastichsearch

Can you elaborate what you mean by this? I'm not familiar with using Kibana in this way. Is there a documentation I could read about it? Is Kibana equipped to do meta-analytics of our ES cluster?


base64DataUrl not working first time, but yes subsequents IE9

So the error is a 'simple' as that, first time you select a file and try to read as base64DataUrl it fails, no error in the catch. Then if you try again the SAME file, it works correctly.
Any idea where to look? Flash is installed and the shim is loading correctly.

I did some debugging, the problem is that the first time Flash is not setup in the element, after the first click, canceling or selecting a file, Flash starts working.

Related to #674


[REQ] Add new language!

Acknowledgement of preliminary instructions

[X] I have read and acknowledged the preliminary instructions.

What idea do you have?
I lived in Uzbekistan for many years and am fluent in the language and its grammar. Among the teenagers around me, many played Roblox, especially R6 games, but not all of them had powerful computers. I introduced them to Bloxstrap, but since many didn’t speak English, I had to explain everything and translate the functions for them. Bloxstrap became popular in my neighborhood, but most users struggled with the language barrier. I would like to request the addition of an Uzbek translation, and I’m willing to personally handle the translation work. It’s difficult to explain each function to every user individually.

We do translations through Crowdin. Go talk to pizzaboxer about being added.


Add column offset to fix mypy linter pointing to wrong column number (off by one)

Closes #14978
Before:

After:

The tests are failing, you may need to adjust the expected output.


Add support for keyed nodes reuse

@TimLariviere , could you please tell me if i go right way in my attemps to fix #713 ?

@TimLariviere , could you please take another look at PR? I think i have resolved issues you mentioned

One thing that will greatly help for this PR is unit testing.
Could you add tests in the Fabulous.XamarinForms.Tests projects that only call the updateCollectionGeneric, please?
With a bunch a realistic cases such as:

No keyed controls & no change between 2 calls to updateCollectionGeneric
No keyed controls & remove control at end of list between 2 calls to updateCollectionGeneric (should work today)
No keyed controls & remove control at start of list between 2 calls to updateCollectionGeneric (should not work correctly)
2nd control with a key & remove 1st one at start of list between 2 calls to updateCollectionGeneric (should work correctly with the new algorithm)

Example of unit test (not actual code)
let previous =
    [
        View.Label(text = "Label 1")
        View.Label(text = "Label 2")
    ]

let current =
    [
        View.Label(text = "Label 1")
        View.Label(text = "Label 2")
    ]

let mutable mockUpdateCallCount = 0
let mockUpdate _ _ =
    mockUpdateCallCount <- mockUpdateCallCount + 1

do updateCollectionGeneric previous current ... mockUpdate ...

mockCreateCallCount |> should equal 0
mockUpdateCallCount |> should equal 2
// And so on

With those tests, we'll be able to tell if the new algorithm correctly works, as well confirmed previous codes still work as expected.

@TimLariviere , do i need to create  actual XF controls in tests? Cause i need somehow to mock create function

do i need to create actual XF controls in tests? Cause i need somehow to mock create function

No. By mocking the create function, we can ignore Xamarin.Forms.
We don't want to test if the create function does its work, but if it's called when expected.

Ok. i have made a little bunch of tests. Looks like if we need more, would be good to use something like FsCheck

@vshapenko I will add in this PR another file with a set of non-regression tests and new use cases that should be supported by the new algorithm.
We will be able to fix the code after that.

And I must say I don't have much experience using FsCheck, so I don't really see how we can use it here.
So you have a specific idea, don't hesitate to try it out here.

Maybe there is no need in FsCheck yet.  I do not yet fully understand the conditions that should be met to mark test passed on different set of views transformations and updates.

@vshapenko I've merged the non-regression tests from #725 into this PR.
The tests using the new key attribute are missing, I will add them here.
To start, we will need to make the existing tests pass.

Existing tests look ok

@TimLariviere , what else do we need to safely merge this PR?

@vshapenko I wrote some new tests making use of key. Some are failing.
I'll need to clean up the tests a bit and I will push so we can fix the algorithm.

There's 6 tests failing in the ones I've just added (keyed and mixed use cases).
Also I figured out that we can reuse elements directly like the keyed ones, if the instance is the same (identical).
I had to rewrite the algorithm a bit to facilitate its testing.
Now updateCollectionGeneric became updateCollectionGenericInternal and don't directly interact with a list of Xamarin.Forms controls but instead gives the operations to move from the previous state to the new one.
E.g.
[
    View.Label(text = "First time")
    View.Label(key = "Title", text = "Welcome")
    View.Button(text = "Click here")
]
-->
[
    View.Label(key = "Title", text = "Welcome")
    View.Button(text = "Click here")
]

Results in
Update the previous label keyed Title with the new label keyed Title
Update the previous button with the new button (since it has a compatible type)
Remove the label "First time" (since it couldn't be reused)

updateCollectionGeneric applies the operations to the actual Xamarin.Forms list, so it doesn't change its usage.
I hope the tests are understandable with their naming convention.
Don't hesitate to ask questions.

@TimLariviere , looks like your version of algorithm tries to store previous view elements. This would not work this way, i think. My vision is that we operate keys and tie keys to 'TTarget rather than 'T. Otherwise, it will bring alot of inconsistent behaviour, as we see in tests.

I don't think storing 'T instead 'TTarget should make a difference.
In Fabulous, a necessary condition is that the current UI ('TTarget) should always match the previous state ('T) so when selecting a 'T we can find the associated 'TTarget (they're using the same index in prevColl / targetColl).
So we can abstract the real controls list and only work with the previous/current ViewElements.

Note: I had to keep a copy of the targetColl to avoid accessing an already updated list.
let previousTargetColl = List(targetColl)

Though I believe we can achieve the same without this copy.

I think, currently root of all evils is at  following code:
 let prevChildOpt = match prevCollOpt with ValueNone -> ValueNone | ValueSome coll when i < n -> ValueSome coll.[i] | _ -> ValueNone

Yes, we should not use index anymore.
How I imagine it:
We make 2 sets from the previous collection: one with all the keyed elements, one with the rest ("the reusable pool").
For each element (curr) inside the new collection:

if prevColl has an element where identical prev curr = true

move to the correct index but do not update (ViewElement is immutable so same ref = same values)

else if curr has a key

try to get it from the keyed list

update it

otherwise try to get an element from the reusable pool where canReuse prev curr = true

update it (then remove the reused element from the pool)

else

create it

else

try to get an element from the reusable pool where canReuse prev curr = true

update it (then remove the reused element from the pool)

else

create it

@TimLariviere , there are some strange cases in tests like
Given previous state = 1k / current state = 2k, updateCollectionGeneric should Update[1k -> 2k]
For test to pass, they should have same automationId, i think, else  we dont get canReuse=true

That's a tricky case, and maybe it's wishful thinking, but 1k has no matching counterpart so it can be "demoted" into the reusable pool.
And since 2k is a brand new key, it can't find matching keys in the keyed list, so it will tap into the reusable pool, finding 1k.
One way I can think of handling that is: before adding a previous keyed element inside the keyed list, we check that a matching key exists in the new collection. if exists, add to keyed list, if not, add to reusable pool.
Might not be the most performant way but we'll see how to improve once we cover all the cases.

Ok, for now i have only 4 tests failing. Most cases are about "replace". In our logic it should be create& remove i believe

Ok, for now i have only 4 tests failing. Most cases are about "replace". In our logic it should be create& remove i believe

Yes, I agree.

@TimLariviere , please, take a look to changes, all tests are passing now, but i am in doubt about mockRemove and remove implementation

And you were right about the "replace" tests, it should be "create + remove".
I changed them.

@TimLariviere , all tests are passing right now. I think we do not need "create+remove" logic in tests.

Looks good.
I cleaned up the code a little, adding comments and reformatting to follow same style than the rest of Fabulous.
I also added an identicalElements list so we don't need to remove an identical element (without a key) from reusableElements at the end.

For current logic it looks like it is not needed, because we clear rest on create, and remove just does not take place because it is useless in such scenario. So, we make create and do not need remove because there is nothing to remove

Even though I agree with you that it doesn't change anything for the algorithm, I prefer to have create + remove instead of just create.
It makes the intent clearer. We can't reuse the previous element, so we create a new one and delete the previous one.

Tested the samples.
All work except TicTacToe. Will take a look later.

Looks good.
I cleaned up the code a little, adding comments and reformatting to follow same style than the rest of Fabulous.
I also added an identicalElements list so we don't need to remove an identical element (without a key) from reusableElements at the end.

For current logic it looks like it is not needed, because we clear rest on create, and remove just does not take place because it is useless in such scenario. So, we make create and do not need remove because there is nothing to remove

Even though I agree with you that it doesn't change anything for the algorithm, I prefer to have create + remove instead of just create.
It makes the intent clearer. We can't reuse the previous element, so we create a new one and delete the previous one.

delete is a  "fake" operation in such cases, because create already acts as create+ replace. If create was using insert instead .[i]<- x, then deletion would be needed

Yes absolutely. This is just needed for a "complete picture" in the tests.
updateCollectionGeneric completely ignores it.

@vshapenko Ok, finally got it working!
I had to fix 2 issues to make TicTacToe work correctly:

When applied to ObservableCollection, the algorithm could potentially throw, even though it should have been fine.

That's because of the way Xamarin.Forms reacts to changes in ObservableCollection.
When we're moving an element from index X to Y (where X > Y), we set the index Y with the updated control and left X because it will be replaced by something else later.
Though Xamarin.Forms doesn't like having the same ref twice at different indexes, and it will try to delete the ref at X while we're trying to set at Y.
This resulted in Xamarin.Forms trying to update ObservableCollection while it was already being updated. Thus crashing.
I had to change the algorithm a bit to use RemoveAt/Insert instead of setting at a specific index.

Updating attached properties (e.g. Grid.Column) was no longer working

Attached properties in Fabulous are tricky.
In the case of Grid, the attached properties are to be applied to the Children items but this property is defined in the inherited class Layout<'T>.
So the way CodeGen generated code required us to reprocess the "derived" attached properties (Grid => Layout<'T>) independently.
This was possible because the previous algorithm acted predictability by only using indexes.
It was not the case anymore.
So I changed the way attached properties were updated to make it possible to update them while executing updateCollectionGeneric.

That rabbit hole is deeper than i thought

Well, for the attached properties, it was bound to happen I guess.
What was implemented always felt more like workaround than proper support.
It's slightly better now at least.

Maybe would be worth to implement Replace method in observable collection in xf. This will give us one render pass instead of two in some cases

@TimLariviere , what else do we need to merge this PR? If needed, would write a bunch of docs about keys

@vshapenko Yes, only the documentation remains.
It would be great if you write some docs about key, why is it needed, how to use it, what it will do concretely.
You can put that in https://fsprojects.github.io/Fabulous/Fabulous.XamarinForms/views-perf.html

@TimLariviere , i have added documentation draft, may be some samples would be needed.

@TimLariviere , updated documentation a little bit. Hope the purpose of key became more understandable


Allow the input to be empty in case of a pipe.

I don't want this to error out. The search query can be provided as part of the context you pass in:
 2024-09-29 10:13:02 ⌚ (☸ |kind-apache:N/A) Guillermos-MacBook-Pro in /tmp
○ → cat context.txt | chatgpt
you must specify your query

Make sure there is a delta in the history; should still throw an error when there is none. Perhaps a more specific error though?

Fixed here:
https://github.com/kardolus/chatgpt-cli/commit/2bf30f406d9a5984d105d9b4ef6e47bb0185565d


[SPARK-20239][Core] Improve HistoryServer's ACL mechanism

What changes were proposed in this pull request?
Current SHS (Spark History Server) two different ACLs:

ACL of base URL, it is controlled by "spark.acls.enabled" or "spark.ui.acls.enabled", and with this enabled, only user configured with "spark.admin.acls" (or group) or "spark.ui.view.acls" (or group), or the user who started SHS could list all the applications, otherwise none of them can be listed. This will also affect REST APIs which listing the summary of all apps and one app.
Per application ACL. This is controlled by "spark.history.ui.acls.enabled". With this enabled only history admin user and user/group who ran this app can access the details of this app.

With this two ACLs, we may encounter several unexpected behaviors:

if base URL's ACL (spark.acls.enable) is enabled but user A has no view permission. User "A" cannot see the app list but could still access details of it's own app.
if ACLs of base URL (spark.acls.enable) is disabled. Then user "A" could see the summary of all the apps, even some apps didn't run by user "A", but can only access its own app's details.
if ACLs of base URL (spark.acls.enable) is disabled, then user "A" could download any application's event log, even it is not run by user "A".
history admin ACL (if it is not also configured in spark.admin.acls) has no permission to list all apps.
The changes of Live UI's ACL will affect History UI's ACL which share the same conf file.

The unexpected behaviors is mainly because we have two different ACLs, ideally we should have only one to manage all.
So to improve SHS's ACL mechanism, here in this PR proposed to:

Unify two different ACLs into one ("spark.history.ui.acl.enable").
Do not enable SecurityFilter in the base URL to do "all or nothing" filter, leverage app base ACL to partially list apps which can be accessed by user.

With this PR:

Admin user could see/download the list of all applications, as well as application details.
Normal user could see/download the list of applications accessible by him, as well as the details.

How was this patch tested?
New UTs are added, also verified in real cluster.
CC @tgravescs @vanzin please help to review, this PR changes the semantics you did previously. Thanks a lot.

Test build #75641 has started for PR 17582 at commit bc1e53a.

Test build #75641 has finished for PR 17582 at commit bc1e53a.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75641/
Test FAILed.

Merged build finished. Test FAILed.

Test build #75643 has started for PR 17582 at commit e56c388.

Test build #75643 has finished for PR 17582 at commit e56c388.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75643/
Test PASSed.

Sorry but I'm confused by the explanation in the description.  I didn't completely follow what problems you are seeing that aren't intended and I don't understand how you are proposing to fix.  Can you please describe the design you are proposing in more detail?
On the description can you please clarify each of your bullets? For instance:

if base URL's ACL (spark.acls.enable) is enabled but user A has no view permission. User "A" cannot see the app list but could still access details of it's own app.

Are you saying user A is not in the list of acls or is?  if they have no view permission then they shouldn't be able to see the app.  I don't understnad what you mean by "could still access details of it's own app"?  Is this user A's application (meaning they started it) and hence he would automatically be in the acl list?
Clarifying the other bullets would be helpful as well.

user configured with "spark.admin.acls" (or group) or "spark.ui.view.acls" (or group), or the user who started SHS could list all the applications, otherwise none of them can be listed

So to me this is the only bug; which means that maybe ACLs on the listing itself shouldn't ever be applied, and this PR should be a lot simpler, right?
Most of it seem to be dealing with filtering the list of apps so that only applications the user can see are shown. I wonder if that's necessary, since the only thing that's showing is the existence of the application, not any data about it that could be considered sensitive.
There's also a minor thing that the listing being different for different users might cause confusion; but if there's a good reason for filtering, then that concern can be overridden. I'm just not sure there is a good reason for it.

@tgravescs sorry for the confuse.

if base URL's ACL (spark.acls.enable) is enabled but user A has no view permission. User "A" cannot see the app list but could still access details of it's own app.

Here actually has two list of acls, one is controlled by spark.acls.enabled, if user "A" is not added to this acl list, then user "A" cannot see the app list (/<history-server-url>/api/v1/applications). But if this app is run by user "A", then user "A" could still see the details of app, like (/<history-server-url>/api/v1/applications/<app-id>/jobs), this acl is controlled by "spark.history.ui.acls.enabled", and user "A" is automatically in the acl list (because of run by him).

if ACLs of base URL (spark.acls.enable) is disabled. Then user "A" could see the summary of all the apps, even some apps didn't run by user "A", but can only access its own app's details.

If "spark.acls.enabled" is disabled, then SecurityFilter is not worked, so user "A" could access /<history-server-url>/api/v1/applications, which means user "A" could see all the applications even not run by him.
This /<history-server-url>/api/v1/applications doesn't touch spark.history.ui.acls.enabled.

if ACLs of base URL (spark.acls.enable) is disabled, then user "A" could download any application's event log, even it is not run by user "A".

This is the same issue as above. /<history-server-url>/api/v1/applications/<app-id>/logs is only controlled by "spark.acls.enable", not "spark.history.ui.acls.enable". So anyone could download any even logs if "spark.acls.enable" is disabled.
So basically what I fixed is that:

disable the work of spark.acls.enable, which means SecurityFilter is not checked.
Using spark.history.ui.acls.enable to filter applications, application summary and application log based on users who run the app.

So the result of my PR is:

history admin user could see/download/access any apps.
normal user could see/download/access apps run by him.

@vanzin your suggestion is to only disable ACLs on the listing, that definitely simplifies the fix, but IMO that "all or nothing" solution is not so ideal:

any user could list all the apps, though cannot access the details if it is not run by him. For the sensitivity, is it better to even not show the apps not run by him?
currently if ACLs on listing is disabled, anyone could download event log, which on the other hand expose the security hole to other users.

So IMO filtering based on users is better than "all or nothing" solution. Also it doesn't increase the code complex much.

@tgravescs @vanzin do you have any comment on this JIRA?
A compromise is that any user could see all the app list but detailed information is still controlled by per app ACLs. But we should also fix event log download issue, currently anyone could download the event log if "spark.acls.enable" is disabled, even it is not permitted by HDFS. This is definitely a security hole should be fixed.
What do you think?

Sorry again the wording above and all the different configs are a bit confusing to me as to what the real issues are here.

Here actually has two list of acls, one is controlled by spark.acls.enabled, if user "A" is not added
to this acl list, then user "A" cannot see the app list (//api/v1/applications). But if this app is run by user "A", then user "A" could still see the details of app, like (//api/v1/applications//jobs), this acl is controlled by "spark.history.ui.acls.enabled", and user "A" is automatically in the acl list (because of run by him).

You are mixing things here.  You say that if user "A" is not added to acl list he cannot see the app list. This is broken then and I assume only applies to rest api not UI?  But I'm not sure what that has to do with your second sentence, if user "A" ran the app then of course he can see the details of the app, that is intended. I'm not sure what that has to do with the first issue?  If you don't have spark.history.ui.acls.enabled then it is up to what the user set.  Generally in any secure environment you should set spark.history.ui.acls.enabled=true and it should enforce acls no matter what user set.  It might help for you to describe these in terms of configs.  Which exact configs are set on the history server and which exact configs are set on the application side and which exact apis are being used (Rest vs Web UI).
so all the urls you list are the REST API, is this only an issue with rest api or the actual web UI as well?  It sounds like things are definitely broke there but I'm not sure it requires changing the configs just fixing the things that are broken.
Its supposed to be that if spark.history.ui.acls.enable is enabled it doesn't matter what the setting of spark.acls.enable is, acls should always be enforced on the history server.  see the description: https://spark.apache.org/docs/latest/monitoring.html
Certain UI's don't have information that should be sensitive. I thought the list of applications was one of those things, all users should be able to see the entire list of applications.  Nothing sensitive there, but once you look at the application details that should be acl'd.  If someone added something sensitive then it should be protected or it should be moved from that page.
My opinions on your response to @vanzin

No, there shouldn't be sensitive information there and many times a user is looking for a job run by say a headless user or other user.  I guess you could filter only the jobs that user has acls to but that makes it more complicated.  Do you have a concrete reason it should be protected?  Note that this follow how other Hadoop UI's work.

That is just broken, event log should be protected.

Been following this but haven't had time to do a proper review, but @tgravescs since you brought up the UI vs API thing, as of 2.0 the UI gets it's list from the API so that's where the security has to be handled.

@tgravescs , with the changes of history UI, REST API and web UI are now mixed. The base URL to list all the apps is through REST API.
The key problem here is that in History Server we could have configured two ACLs (spark.acls.enable here and spark.history.ui.acls.enable here), this two ACLs checks different URLs, for example:
/api/v1/applications
/api/v1/applications/<app-id>
/api/v1/applications/<app-id>/logs

are controlled by spark.acls.enable. And web UI will use this API to list and show apps on the UI. If this is enabled, then only SHS owner, admin/view user groups could see the apps list on the SHS's base web UI.
And all the other URLs (web UI and REST API) for application details are controlled by spark.history.ui.acls.enable.
If we configured differently for this two ACLs, then we will get some unexpected behaviors.
So what I here fixed is to unify the ACL and offer the right behavior.

No, there shouldn't be sensitive information there and many times a user is looking for a job run by say a headless user or other user. I guess you could filter only the jobs that user has acls to but that makes it more complicated. Do you have a concrete reason it should be protected? Note that this follow how other Hadoop UI's work.

We worked with customers and they wish to filter and list apps based on the login users. But I don't have strong opinion it. I could change to what you suggested.

so we should definitely fix the /api/v1/applications//logs to go through the acls.  It looks like it should be protected in ApiRootResource.java. You have the app id so it needs to do something like the withSparkUI to get the acls included in that application.
Like I mentioned the listing (/api/v1/applications) and /api/v1/applications/ (which is same info I believe as listing) were intentionally left open.  I don't really see a reason to change that but if other people have a use case for it then perhaps we should make which pages are protected by acls configurable.
on the history server I would expect spark.acls.enable=false and spark.history.ui.acls.enable=true, I can see where that could be confusing, perhaps we should document this better. spark.acls.enable on the history UI really is protecting the root UI, not the app level ui's.  We could explicitly turn this off.

Thanks @tgravescs for your reply.

on the history server I would expect spark.acls.enable=false and spark.history.ui.acls.enable=true, I can see where that could be confusing, perhaps we should document this better. spark.acls.enable on the history UI really is protecting the root UI, not the app level ui's. We could explicitly turn this off.

This could be happened when history server and spark application shares same configuration file. That's why in our internal test the behavior is not expected.

Test build #75934 has started for PR 17582 at commit a8fb1e0.

Test build #75934 has finished for PR 17582 at commit a8fb1e0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75934/
Test PASSed.

Merged build finished. Test PASSed.

Test build #75967 has started for PR 17582 at commit 68c9d83.

Test build #75970 has started for PR 17582 at commit 4b3781f.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75970/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75967/
Test FAILed.

Jenkins, retest this please.

Test build #75977 has started for PR 17582 at commit 4b3781f.

Test build #75977 has finished for PR 17582 at commit 4b3781f.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75977/
Test FAILed.

retest this please

Test build #75998 has started for PR 17582 at commit 4b3781f.

@jerryshao is the PR description still accurate? It seems you're not really implementing 2 anymore.

Test build #75998 has finished for PR 17582 at commit 4b3781f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75998/
Test PASSed.

Just update the description, please review again @vanzin , thanks!

changes lgtm. Did you file a jira to track changing to not use withSparkUI?  If user is downloading because the file is huge and takes a long time to render or causes history server to have issue this would hurt that use case.   We could wait and see if someone has that use case too.

Thanks @tgravescs for your comments. Do you think it is a good idea to read out ACLs when mergeApplicationListing  in here and keep in applications, so that we don't need to load SparkUI to check ACLs when downloading event logs?

As @vanzin said I think this is fine for now to get this fixed quickly, but filing a follow up jira makes sense.    Actually this might be good to get into the 2.1.1 release if they are going to spin another rc.

OK, thanks @tgravescs .

LGTM. Merging to master / 2.2, will try 2.1 and 2.0 too.

No luck with 2.1, please file a separate PR.

What about branch 2.0, do we also need to backport to it @vanzin ?

It would be good, but maybe the 2.1 backport will merge cleanly to 2.0.

OK, let me try it, thanks.


User reporting to have issues with rom uploads

No more information for now, except that they were not able to add even one rom with the manager.

I really need more feedback than this to look about it, since it just works as is.
There is just some lacks of form upload error related to how Dropzone is actually integrated,  that hide error messages, see #28
I strongly think your dumb feedback is just errors about the file format / file extensions rejected by the Manifest rules

http://raspbian-france.fr/recalbox-raspberry-pi-emulateur-retro-gaming/ looks like there is an other dumb feedback. I will ask what happenend.

They won't know anything like i said with #28, upload errors are almost hided since there is a dumb redirection at end of the upload process


The "@" char prefix in Loader="LoadOrders" is not required

Because the reference to LoadOrders in TemplatedList is automatically recognized, the "@" prefix is not required. Best regards.

@dotnet-policy-service agree

@dotnet-policy-service agree

@dotnet-policy-service agree
De: dotnet-policy-service[bot] @.>
Enviado el: martes, 25 de abril de 2023 08:27 p. m.
Para: dotnet-presentations/blazor-workshop @.>
CC: asereware @.>; Mention @.>
Asunto: Re: [dotnet-presentations/blazor-workshop] The "@" char prefix in Loader="LoadOrders" is not required (PR #362)
@asereware https://github.com/asereware  please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
@dotnet-policy-service agree [company="{your company}"]
Options:

(default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.

@dotnet-policy-service agree

(when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.

@dotnet-policy-service agree company="Microsoft"
Contributor License Agreement
Contribution License Agreement
This Contribution License Agreement ( “Agreement” ) is agreed to by the party signing below ( “You” ),
and conveys certain license rights to the .NET Foundation ( “.NET Foundation” ) for Your contributions to
.NET Foundation open source projects. This Agreement is effective as of the latest signature date below.

Definitions.

“Code” means the computer software code, whether in human-readable or machine-executable form,
that is delivered by You to .NET Foundation under this Agreement.
“Project” means any of the projects owned or managed by .NET Foundation and offered under a license
approved by the Open Source Initiative (www.opensource.org http://www.opensource.org ).
“Submit” is the act of uploading, submitting, transmitting, or distributing code or other content to any
Project, including but not limited to communication on electronic mailing lists, source code control
systems, and issue tracking systems that are managed by, or on behalf of, the Project for the purpose of
discussing and improving that Project, but excluding communication that is conspicuously marked or
otherwise designated in writing by You as “Not a Submission.”
“Submission” means the Code and any other copyrightable material Submitted by You, including any
associated comments and documentation.

Your Submission. You must agree to the terms of this Agreement before making a Submission to any
Project. This Agreement covers any and all Submissions that You, now or in the future (except as
described in Section 4 below), Submit to any Project.

Originality of Work. You represent that each of Your Submissions is entirely Your original work.

Should You wish to Submit materials that are not Your original work, You may Submit them separately
to the Project if You (a) retain all copyright and license information that was in the materials as You
received them, (b) in the description accompanying Your Submission, include the phrase “Submission
containing materials of a third party:” followed by the names of the third party and any licenses or other
restrictions of which You are aware, and (c) follow any other instructions in the Project’s written

Your Employer. References to “employer” in this Agreement include Your employer or anyone else
for whom You are acting in making Your Submission, e.g. as a contractor, vendor, or agent. If Your
Submission is made in the course of Your work for an employer or Your employer has intellectual
property rights in Your Submission by contract or applicable law, You must secure permission from Your
employer to make the Submission before signing this Agreement. In that case, the term “You” in this
Agreement will refer to You and the employer collectively. If You change employers in the future and
desire to Submit additional Submissions for the new employer, then You agree to sign a new Agreement
and secure permission from the new employer before Submitting those Submissions.

Licenses.

a. Copyright License. You grant .NET Foundation, and those who receive the Submission directly
or indirectly from .NET Foundation, a perpetual, worldwide, non-exclusive, royalty-free, irrevocable
license in the Submission to reproduce, prepare derivative works of, publicly display, publicly perform,
and distribute the Submission and such derivative works, and to sublicense any or all of the foregoing
rights to third parties.
b. Patent License. You grant .NET Foundation, and those who receive the Submission directly or
indirectly from .NET Foundation, a perpetual, worldwide, non-exclusive, royalty-free, irrevocable license
under Your patent claims that are necessarily infringed by the Submission or the combination of the
Submission with the Project to which it was Submitted to make, have made, use, offer to sell, sell and
import or otherwise dispose of the Submission alone or with the Project.
c. Other Rights Reserved. Each party reserves all rights not expressly granted in this Agreement.
No additional licenses or rights whatsoever (including, without limitation, any implied licenses) are
granted by implication, exhaustion, estoppel or otherwise.

Representations and Warranties. You represent that You are legally entitled to grant the above
licenses. You represent that each of Your Submissions is entirely Your original work (except as You may
have disclosed under Section 3 ). You represent that You have secured permission from Your employer to
make the Submission in cases where Your Submission is made in the course of Your work for Your
employer or Your employer has intellectual property rights in Your Submission by contract or applicable
law. If You are signing this Agreement on behalf of Your employer, You represent and warrant that You
have the necessary authority to bind the listed employer to the obligations contained in this Agreement.
You are not expected to provide support for Your Submission, unless You choose to do so. UNLESS
REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING, AND EXCEPT FOR THE WARRANTIES
EXPRESSLY STATED IN SECTIONS 3, 4, AND 6 , THE SUBMISSION PROVIDED UNDER THIS AGREEMENT IS
PROVIDED WITHOUT WARRANTY OF ANY KIND, INCLUDING, BUT NOT LIMITED TO, ANY WARRANTY OF
NONINFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.

Notice to .NET Foundation. You agree to notify .NET Foundation in writing of any facts or
circumstances of which You later become aware that would make Your representations in this
Agreement inaccurate in any respect.

Information about Submissions. You agree that contributions to Projects and information about
contributions may be maintained indefinitely and disclosed publicly, including Your name and other
information that You submit with Your Submission.

Governing Law/Jurisdiction. This Agreement is governed by the laws of the State of Washington, and
the parties consent to exclusive jurisdiction and venue in the federal courts sitting in King County,
Washington, unless no federal subject matter jurisdiction exists, in which case the parties consent to
exclusive jurisdiction and venue in the Superior Court of King County, Washington. The parties waive all
defenses of lack of personal jurisdiction and forum non-conveniens.

Entire Agreement/Assignment. This Agreement is the entire agreement between the parties, and
supersedes any and all prior agreements, understandings or communications, written or oral, between
the parties relating to the subject matter hereof. This Agreement may be assigned by .NET Foundation.

.NET Foundation dedicates this Contribution License Agreement to the public domain according to the Creative Commons CC0 1.
—
Reply to this email directly, view it on GitHub https://github.com/dotnet-presentations/blazor-workshop/pull/362#issuecomment-1522672423 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDA2RFLIRIM4P6RW6KSAC3XDCBYNANCNFSM6AAAAAAXLZHDBI .
You are receiving this because you were mentioned.  https://github.com/notifications/beacon/ABDA2RFHOS2QLP76SLOKP4LXDCBYNA5CNFSM6AAAAAAXLZHDBKWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTS2YIRSO.gif Message ID: @.*** @.***> >
--
Este correo electrónico ha sido analizado en busca de virus por el software antivirus de Avast.
www.avast.com


Add QI collection and reader

requires changes in ismn package to be released as well

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.09%) to 95.273%

Totals

Change from base Build<PHONE_NUMBER>:
-0.09%

Covered Lines:
998

Relevant Lines:
1038

💛  - Coveralls


Update pull request and sonar workflow

This PR updates workflows files to align with the latest template in bootstack-charms-spec. Additionally, we are now calling sonar reusable workflow in bootstack-actions.

The functional test error is not related to this PR because it also failed when I synced upstream master into my forked repo (run log). I will look into the issue and propose a fix if needed.


New version, changed spin to/from binary conversion convention

Codecov Report

Merging #10 into master will increase coverage by 0.14%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #10      +/-   ##
==========================================
+ Coverage   97.33%   97.48%   +0.14%
==========================================
  Files          78       79       +1
  Lines        4961     5001      +40
  Branches      529      528       -1
==========================================
+ Hits         4829     4875      +46
+ Misses         81       77       -4
+ Partials       51       49       -2

Impacted Files
Coverage Δ

qubovert/problems/_problem_parentclass.py
85.71% <ø> (ø)
:arrow_up:

tests/utils/test_hisingmatrix.py
100% <ø> (ø)
:arrow_up:

tests/utils/test_solve_bruteforce.py
100% <ø> (ø)
:arrow_up:

qubovert/_qubo.py
100% <100%> (ø)
:arrow_up:

qubovert/_ising.py
100% <100%> (ø)
:arrow_up:

qubovert/problems/np/coloring/_job_sequencing.py
97.32% <100%> (+0.07%)
:arrow_up:

tests/test_qubo.py
99.47% <100%> (+0.01%)
:arrow_up:

qubovert/problems/np/bilp/_bilp.py
97.72% <100%> (+0.16%)
:arrow_up:

qubovert/_pubo.py
90% <100%> (ø)
:arrow_up:

qubovert/_version.py
100% <100%> (ø)
:arrow_up:

... and 21 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6a2a7a8...d8c637b. Read the comment docs.


[IGNORE] Set up end-to-end tests

The purpose of this PR is to set up Playwright for UI end-to-end tests and document some basic patterns. To start, it includes some basic tests running against Chromium.
TODOs

[x] Clean up a few things
[x] Write up README explaining how the e2e tests and playwright work
[ ] Write more tests. I'm going to do this in follow-ups because this PR was already getting waaaaay tooo biiiig.
[ ] Extract the CI and local dev configs into separate configurations that extend from a base config. Going to also do this in a follow-up.

One of the selectors was being a little flaky, so adjusting a few minor bits for that test to hopefully fix it.

One of the selectors was being a little flaky, so adjusting a few minor bits for that test to hopefully fix it.

That wasn't the issue. It was that I'd merged in upstream changes that made some of the selectors invalid. Fixed now and the tests are passing.


fix: overlay position dislocation when resize the window

I tried to using the entire document directly to the targetNode, But the effect is not ideal.
close #1

I think it is enough to reset the overlay when the size changes and the scroll bar, there is no need to consider too complicated

Yep. I think so.


ignore private methods and non-settable properties

Before this PR

If you had an abstract class with private methods or public methods with parameters, it would try to start assigning those fields even though it doesn't make any sense.
After this PR

==COMMIT_MSG==
Stricter filtering on what is and isn't settable.
==COMMIT_MSG==
Now we ensure that for a method to be assigned it:

is not private
is a zero arg method
it's either nested
or it is settable i.e. derives from some sort of Property, had to include ConfigurableFilesCollection as I had missed that last time round.

Possible downsides?
It feels very whack-a-mole. It gets us to our goal of more parallelisable things, but I wonder whether we can do a two-pronged approach with trying to attack the configuration cache (which supposedly gets you parallel runs for free as well).

Released 1.4.0


fix(jvm): Bind all interfaces to restore remote debugging with newer JDK versions

See https://www.oracle.com/technetwork/java/javase/9-notes-3745703.html#JDK-8041435.
Release Note
fix(jvm): Bind all interfaces to restore remote debugging with newer JDK versions

Travis is not reporting status back..


Replace ModuleNotFoundError by ImportError

Fixed: ModuleNotFoundError is only available in python>=3.6.

I want a comment somewhere saying that ModuleNotFoundError is to be used from Python 3.6 on.


Malformed log entry from golang http library

log messages must have 'L!' prefix where L is one of 'D', 'I', 'W', 'E'[log] 2017/05/23 13:37:30 http: Accept error: accept tcp [::]:9092: accept: too many open files; retrying in 1s
log messages must have 'L!' prefix where L is one of 'D', 'I', 'W', 'E'[log] 2017/05/23 13:37:32 http: Accept error: accept tcp [::]:9092: accept: too many open files; retrying in 1s
log messages must have 'L!' prefix where L is one of 'D', 'I', 'W', 'E'[log] 2017/05/23 13:37:37 http: Accept error: accept tcp [::]:9092: accept: too many open files; retrying in 1s
log messages must have 'L!' prefix where L is one of 'D', 'I', 'W', 'E'[log] 2017/05/23 13:37:40 http: Accept error: accept tcp [::]:9092: accept: too many open files; retrying in 1s
log messages must have 'L!' prefix where L is one of 'D', 'I', 'W', 'E'[log] 2017/05/23 13:37:42 http: Accept error: accept tcp [::]:9092: accept: too many open files; retrying in 1s
log messages must have 'L!' prefix where L is one of 'D', 'I', 'W', 'E'[log] 2017/05/23 13:37:47 http: Accept error: accept tcp [::]:9092: accept: too many open files; retrying in 1s
log messages must have 'L!' prefix where L is one of 'D', 'I', 'W', 'E'[log] 2017/05/23 13:37:50 http: Accept error: accept tcp [::]:9092: accept: too many open files; retrying in 1s
log messages must have 'L!' prefix where L is one of 'D', 'I', 'W', 'E'[log] 2017/05/23 13:37:52 http: Accept error: accept tcp [::]:9092: accept: too many open files; retrying in 1s
log messages must have 'L!' prefix where L is one of 'D', 'I', 'W', 'E'[log] 2017/05/23 13:37:57 http: Accept error: accept tcp [::]:9092: accept: too many open files; retrying in 1s

See https://golang.org/pkg/net/http/#Server, we can provide a logger to the server so these message can correctly follow the wlog format.

Going  to close this as it was delivered a while ago.


Exit fzy, exit script.

Hi
If I'm using fzy in a script, and I CTRL-C to abort, I would like to exit the script. Currently, fzy aborts, but the script continues.
Example:
#!/bin/sh

file=$(echo 'foo' | fzy)

cp "$file" ~/

CTRL-C while fzy is displaying 'foo' passes 'foo' to the cd command.
I currently work around this using, for example:
#!/bin/sh

file=$(echo 'foo' | fzy)
case "$?" in
	0) : ;;
	1) exit 1 ;;
esac

cp "$file" ~/

In this case, CTRL-C while fzy is displaying 'foo' will abort the script.
(It occurs to me that CTRL-C is undocumented, and maybe you're not interested?)

I'm going to take a wild stab in the dark and say if you use bash instead of sh the problem will not happen as bash will abort your script if a command fails.
If if doesn't, set -eo pipefail will make sure your script exits when a command fails (tbf, all your scripts should start with set -euo pipefail).

Thanks. I'm using dash, but set -e (for errexit; there's no 'pipefail' in dash) does the trick nicely.


document how to turn on tracing in demo applications

https://github.com/rancher/opni/issues/899

this is a duplication of #899. close issue.


How to set padding between icon and text ?

padding was set based on android material design guidelines. you want extra padding, adjust the icon


feat(commandbus): async commands

TODO

[ ] Docs

Pull Request Test Coverage Report for Build<PHONE_NUMBER>1
Details

32 of 79 (40.51%)  changed or added relevant lines in 6 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.4%) to 82.123%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/Tempest/CommandBus/src/HandleAsyncCommand.php
12
15
80.0%

src/Tempest/CommandBus/src/AsyncCommandRepositories/FileRepository.php
0
14
0.0%

src/Tempest/CommandBus/src/MonitorAsyncCommands.php
0
30
0.0%

Totals

Change from base Build<PHONE_NUMBER>8:
-0.4%

Covered Lines:
7272

Relevant Lines:
8855

💛  - Coveralls

@aidan-casey @innocenzi can you give a review?

Code-wise: just curious about the usage of HasConsole. Since we inject Console in the constructor anyway, why not use it directly?

No big reason for this… just something I've gotten used to as I prefer to call $this-> instead of $this->console.


Code blocks missing margin-top when nested in definition lists

Describe the bug
Kramdown natively supports definition lists. Users can nest code blocks inside of definition lists with the following indentation syntax (not supported by GFM).

term
: definition

  ```sh
  echo code block
  ```

However, there is no margin between the definition text and the code block. The HTML is rendered (roughly) as follows.

term

    definition

          echo code block

To Reproduce
Create new Markdown document with the above contents. Serve the website and view the result in a browser.

Expected behavior
A margin should appear between the definition text and the code block.

Desktop:
Chrome OS 84.0.4147.136
Additional context
The simplest fix would be to add margin-top: 0.75rem; to highlighter-rouge, but this would also introduce a margin to definitions that only contain a single code block. It looks fine visually though, and is consistent with how ol and ul lists appear when nested inside definition lists.

An alternative that displays well but results in less meaningful HTML markup is to treat the nested code block as a separate definition. Note the colon before the code block.

term
: definition
: ```sh
  echo code block
  ```

@kevinlin1 Kramdown also allows a blank line between the term and its description:
term

: definition

  ```sh
  echo code block
  ```

That currently (v0.3.1) displays with a margin between the definition text and the code block:

The definition text is now a paragraph element, rather than just text, and the margin comes from the bottom of the paragraph. The paragraph also has a top margin, so it starts below the bottom of the term (which corresponds to the default layout produced by browsers) but at least the HTML is semantically correct. And to eliminate the top margin, the following selector seems to work:
dd > p:first-child {
  margin-top: 0;
}

giving:

The lack of margin between plain (non-paragraph) text and a code block is independent of definition list styling, but I don't think Kramdown generates that pattern in other contexts. For example, regarding ordinary lists Kramdown states:

The content of a list item is made up of either text or block-level elements. Simple list items only contain text like in the above examples. They are not even wrapped in a paragraph tag. If the first list text is followed by one or more blank lines, it will be wrapped in a paragraph tag:

It could be regarded as a Kramdown bug that it doesn't do the same in definition lists.

Thanks for the reference @pdmosses! gettalong/kramdown#482 proposes achieving parity between definition lists and other list types as you've suggested. In the meantime, we can still consider merging your CSS suggestion and recommend inserting a blank line between the term and the first definition.

@kevinlin1 good to know about that Kramdown PR. The following comment in issue https://github.com/gettalong/kramdown/issues/486 suggests that the difference between definition lists and other list types regarding paragraphs was a deliberate design choice:

According to the syntax documentation for (un)ordered lists, the first paragraph should only be a real paragraph if its text is followed by a blank line. Similar for definition lists although there the blank line must precede the definition text.

The rationale for the difference is unclear to me. But anyway, by adding blank lines, users can always arrange for all text in lists to be enclosed in paragraphs. That makes the spacing of the content of each list item consistent with the layout the same content would have outside lists.
However, paragraphs and most other block-level elements have a nonzero top margin. That margin generally needs to be removed to make block-level content in a dd align nicely with its associated dt element; similarly for dl elements in lis. It would be good to expand https://pmarsceill.github.io/just-the-docs/docs/index-test/ to check all block-level nesting combinations...

@kevinlin1 tests for alignment of block-level first children in lists are now available on a separate test site, together with my proposed CSS fixes.
@pmarsceill could these fixes be included in v0.3.2?


Developing the BC Developers’ Exchange website front-end (BCDX-MTIC-004)

Paid Opportunity
We are offering a fixed price of $3,000 to complete the work described in this issue, according to the terms of our experimental procurement model, Code-with-Us.
Here’s how payment works
Background
bcdevexchange.org is the public facing site for the BC Developers' Exchange - an experiment in tech innovation and collaboration.
You can find the code for the theme here. We want to apply our new theme and design templates to the code already built for the site and deliver the following functionality:
User Story
As a government program owner I need to be able to navigate through a series of self-service screens on bcdevexchange.org to:

Sign Up
Create a page that describes my government’s program
Create a page that describes my program’s project
Create a page that describes a work opportunity for my program’s project
View a page of Resources that are available to me

Acceptance Criteria
To be paid for this opportunity, you will:

Participate as a part of our team’s sprint planning workshop from 10:00 AM - 12:00 PM on December 16, 2016 at 1012 Douglas Street in Victoria, BC.
Spend a minimum of 2 hours per weekday (Monday - Friday, excluding holidays, between the hours of 8:30 AM and 5:00 PM) working in person with us at our office (1012 Douglas Street in Victoria, BC) for the duration of your work. We want you to work side-by-side with our lead developer and designer to apply the the new theme and deliver the desired functionality in our dev environment.
Deliver a functioning presentation layer for bcdevexchange.org that includes all of the following:

Homepage
Resources page
Program list page
Program detail page
Project list page
Project detail page
Opportunities page
Opportunity detail page

Informational Open House
On Monday, December 12, from 10:00-11:00 AM and 1:00-2:00 PM, we will be hosting an informational drop-in session for interested developers to drop in and talk with our team about the work. If you’d like to attend, please email<EMAIL_ADDRESS>Please also feel free to ask questions via the comments section below.
How to Contribute (How to Apply)
To apply for this work, please email a proposal to<EMAIL_ADDRESS>by 4:00 PM PDT on Wednesday, December 14, 2016.  Please reference the issue name "Developing the BC Developers’ Exchange website front-end (BCDX-MTIC-004)" in your email.
What to include in your proposal (evaluation criteria)
We will score your proposal by the following criteria:

Your confirmation of being able to commit the time to meet all of the Acceptance Criteria by January 13, 2016.
A short (around 200 words or less would be nice) summary of your experience working with agile projects (40 points).
References to your relevant experience and demonstrated ability to do the work (60 points). For example, a link to your GitHub projects.
With your proposal, you must attach a copy of the Code-with-Us Terms, with the required information asked for in the “Acceptance” section of the Terms inserted into the document (Mandatory).

If we are satisfied with the proposals, we will assign the work to the person with the highest scoring proposal by the end of the working day on Thursday, December 15, 2016.
Here’s more detail about how assignment works
Questions, Comments, Suggestions?
Please post your questions in the comments section below.

Would I be able to attend the first meeting and then talk to you guys via skype or phone there after?  Thanks

@melavo thanks for your enquiry. We haven't actually posted this opportunity yet but we're close to doing so. When we do I'll answer your question.

@melavo regarding your question... this specific issue has been designed to test onsite collaboration between the assigned developer and the BC DevEx team. While we're testing this idea it doesn't mean that all CWU opportunities will need onsite collaboration. Having said that we do actually need the work done so we do encourage you to submit a proposal however preference will be given to proponents who are able to work side-by-side with our team for a couple of hours a day.

Just a reminder of our information sessions today at 10:00 and 1:00 PM Pacific Time. If you would like to attend (virtually, or in person at 1012 Douglas Street in Victoria, BC), please send us an email<EMAIL_ADDRESS>We will post a summary of what's shared in the meetings here later today.

We did host one information session yesterday at 1:00. Here is a video of that session:
https://youtu.be/q7DjmgpshNE

Just another prompt/reminder to any who are watching this issue:  Proposals due by 4:00 PM Pacific today.

Proposal period is now closed. Thanks to the two folks who submitted! We'll be evaluating later today and will let you know end of day today, or early tomorrow if we are making an assignment.

We have scored the proposals and are assigning this work to @cywolf.

An interesting process, I like it.

Thanks for the great work, @cywolf!


Assumption that User Id queries are case-insensitive

I have a customer using Postgress for their DB with ASP.NET Identity and they're running into issues with the case sensitive queries against the user Id column. Is there anything that can be fixed/changed in the DbContext model builder that sets that column as case sensitive at the EF level in a DB provider agnostic way?

cc @roji to answer if there is anything Npgsql specific you can add to the model to make the queries case insensitive.
I am curious though why the Id column could cause this problem. Are the entity types or the model customized to use a different mechanism to generate the key values from the default? In the default implementation Guid.NewGuid().ToString() is assigned on construction of a new instance and as far as I know we the exact values (with the same casing) should be used from there on in any query or update. Is there anything in the application that is switching those strings to uppercase?

Yes, the Id is provided by a migration tool, IIRC.

Is it possible to get more details here? Is the problem with the column identifier (i.e. Id) or with its contents?
Regarding identifiers, in PostgreSQL, all unquoted identifiers are always folded to lowercase. The Npgsql EF Core emits SQL code that quotes all identifiers, making all matches case-sensitive - there's no way to change either PostgreSQL's behavior or Npgsql's. Did someone create the Id column with a different casing, no via EF Core?
Basically I'd need more info to be able to help here.

The contents itself. The data type used in .NET is a GUID, but then it's converted to a string before using in the DB. This is not a PG issue, IMO, it's an ASP.NET Identity in its string mapping for guids as the PK.

So if I understand correctly, the Id column was populated with uppercase GUID strings (by whom exactly?) but is being queried with lowercase? This seems like it would be a bug in ASP.NET identity which has to be fixed there (i.e. use same case when populating and querying) rather than trying to fix this at the database level...
I'm guessing there's no issue with SqlServer because a case-insensitive collation is set up at the database level? Unfortunately PostgreSQL doesn't support this. The only option for case-insensitive collation is by defining the column type as citext instead of text (see the citext extension). If you can customize the model, than doing .HasColumnType("citext") may provide a workaround (you'd also need to do .HasPostgresExtension("citext")).
But again, this seems like a problem that should be fixed in ASP.NET Identity itself...

ASP.NET Identity is being used to provide a new identity management system for an old, existing one. Data is being imported, so the Id is being set explicitly as part of a data migration/import from the old system. Then yes, during normal processing, queries are being done and somewhere, somehow the GUID Id being used is cased differently when a query is being done. Possibly due to ToString() on the GUID object emitting casing that differs form how their old system had it, and how it was imported into ASP.NET Identity. I can ping the customer to make any additional comments here.

OK, thanks for the added info.
To recapitulate, the correct way to fix this is to ensure the same case is used everywhere - if I understand correctly this is more of a client issue. If a workaround is absolutely necessary at the PostgreSQL/Npgsql/EF Core level instead, it may be possible to redefine the column holding the data as citext instead of text.

Yes, IMO, this was never a PG issue except for the fact that it's case sensitive whereas SqlServer is not (by default).

Sorry for being slow responding--we had some internal team changes. When value conversions are implemented in EF Core 2.1, then it should be possible to put a value converter on the property that will ensure all IDs are converted to upper or lower case before being sent to the database. Until then, I think the best fix would be to run a pass on the client data and put it in a consistent state.
Closing this discussion for now, but feel free to re-open if needed.


How to interpret eval results?

Hi,
im struggling with interpreting the results of the evaluation of the KITTI data.
Can somebody please explain how to read those results e.g.:
Car<EMAIL_ADDRESS>0.70, 0.70:
bbox AP:90.8339, 90.0913, 89.7052
bev  AP:90.3835, 88.7014, 88.1821
3d   AP:89.8144, 86.5609, 79.3242
aos  AP:90.81, 90.01, 89.55
Car<EMAIL_ADDRESS>0.70, 0.70:
bbox AP:96.9829, 95.9043, 93.4644
bev  AP:93.9380, 90.2077, 87.7245
3d   AP:93.0861, 86.3323, 81.8653
aos  AP:96.96, 95.77, 93.28
Car<EMAIL_ADDRESS>0.50, 0.50:
bbox AP:90.8339, 90.0913, 89.7052
bev  AP:98.5752, 90.0662, 89.7175
3d   AP:90.8197, 90.0441, 89.6911
aos  AP:90.81, 90.01, 89.55
Car<EMAIL_ADDRESS>0.50, 0.50:
bbox AP:96.9829, 95.9043, 93.4644
bev  AP:99.1163, 95.9381, 93.4898
3d   AP:96.9698, 95.8939, 93.4472
aos  AP:96.96, 95.77, 93.28

I already got the difference between AP and AP_R40.
Do the three columns stand for the three difficulty levels (easy, moderate, hard)?
What are e.g. 0.70, 0.50, 0.50 in the header of every block standing for?
As I am quite new to this topic, any help would be highly appreciated.

KITTI leadboard uses moderate Car 3D AP (R40) for ranking, which is 86.3323 in your results. Some literature also reports moderate Car 3D AP (R11), which is 86.5609 in your results.
Car<EMAIL_ADDRESS>0.70, 0.70:              # IoU threshold for metric calculation （IoU=0.7）
bbox AP:90.8339, 90.0913, 89.7052  # 2D AP(R11) on image plane for easy, moderate and hard split
bev  AP:90.3835, 88.7014, 88.1821    # Rotated 2D AP(R11) on BEV plane for easy, moderate and hard split
3d   AP:89.8144, 86.5609, 79.3242    # 3D AP(R11) in 3D space for easy, moderate and hard split
aos  AP:90.81, 90.01, 89.55               # orientation similarity for easy, moderate and hard split
Car<EMAIL_ADDRESS>0.70, 0.70:           # IoU threshold for metric calculation
bbox AP:96.9829, 95.9043, 93.4644  # 2D AP(R40) on image plane for easy, moderate and hard split
bev  AP:93.9380, 90.2077, 87.7245   # Rotated 2D AP(R40) on BEV plane for easy, moderate and hard split
3d   AP:93.0861,  86.3323, 81.8653    # 3D AP(R40) in 3D space for easy, moderate and hard split
aos  AP:96.96, 95.77, 93.28                # orientation similarity for easy, moderate and hard split
The metric below is the same as above, but the IoU threshold is different.
Car<EMAIL_ADDRESS>0.50, 0.50:               # IoU threshold for metric calculation （IoU=0.5）
bbox AP:90.8339, 90.0913, 89.7052
bev  AP:98.5752, 90.0662, 89.7175
3d   AP:90.8197, 90.0441, 89.6911
aos  AP:90.81, 90.01, 89.55
Car<EMAIL_ADDRESS>0.50, 0.50:
bbox AP:96.9829, 95.9043, 93.4644
bev  AP:99.1163, 95.9381, 93.4898
3d   AP:96.9698, 95.8939, 93.4472
aos  AP:96.96, 95.77, 93.28

Thanks for the quick response! It did help a lot.
Just one more question to make sure I got it:
Cyclist<EMAIL_ADDRESS>0.50, 0.50: --> IoU threshold = 0.5
Cyclist<EMAIL_ADDRESS>0.25, 0.25: --> IoU threshold = 0.25
Why is the header then 0.50 0.25 0.25 instead of just 0.25? Is there a specific meaning or is only the last number relevant?

I think maybe the first 0.5 means that the bbox metric is calculated by 0.5 threshold, because the bbox results are the same under both<EMAIL_ADDRESS>0.50, 0.50: and<EMAIL_ADDRESS>0.25, 0.25:

Thanks for your help and work in general!