Document YAML DSL on the website

There's no YAML page in the Languages sections.

This is documented in the runtime repo https://github.com/apache/camel-k-runtime/blob/master/docs/modules/languages/pages/yaml.adoc


python312Packages.boto3-stubs: 1.35.84 -> 1.35.86, python312Packages.botocore-stubs: 1.35.84.post1 -> 1.35.86

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 367203

x86_64-linux

:white_check_mark: 14 packages built:

aws-sam-cli
aws-sam-cli.dist
python311Packages.boto3-stubs
python311Packages.boto3-stubs.dist
python311Packages.botocore-stubs
python311Packages.botocore-stubs.dist
python311Packages.types-aiobotocore
python311Packages.types-aiobotocore.dist
python312Packages.boto3-stubs
python312Packages.boto3-stubs.dist
python312Packages.botocore-stubs
python312Packages.botocore-stubs.dist
python312Packages.types-aiobotocore
python312Packages.types-aiobotocore.dist


Not working Firefox mobile

I tried to use tryjshell on my phone's browser. I opened it in Firefox since it is my primary web browser. However, the keyboard doesn't show so I can't type in the shell. I'm just wondering because tryjshell is working in Chrome and Firefox Focus. Another thing that makes me wonder is that, the keywboard would appear in Firefox only after I open tryjshell in Chrome and Firefox Focus. And even if the keyboard has already appeared in Firefox, no characters appear in the shell after hitting them. I'm not sure if it is Firefox issue or tryjshell's.

Actually it should support major browsers, we use xterm, you can create a ticket on there https://github.com/xtermjs/xterm.js/ with your browser name and version.

Works for me if i open a new tab and close it

Weird. It's now working but the keyboard would only show up once I access tryjshell via Chrome first.


update extractTraits workflow

make optional for non-UKBB cohorts

Completed here: bb2dcb4

closing as tracked by milestone


ML 5.13 : MSE & MAE (D)

Description
📌  Issues for Week 5
Welcome to 'ML' Team, good to see you here
:arrow_forward: This issue will helps readers in acquiring all the knowledge that one needs to know about ----
MSE and MAE.
:red_circle: To get assigned to this issue, add your Batch Numbers mentioned in the spreadsheet of "Machine Learning", the approach one would follow and choice you prefer (Documentation, Audio, Video). You can go with all three or any number of options you're interested to work on.
✔ Domain : Machine Learning
:red_circle::yellow_circle: Points to Note :

The issues will be assigned on a first come first serve basis, 1 Issue == 1 PR.

"Issue Title" and "PR Title should be the same. Include issue number along with it.

Changes should be made inside the Machine Learning/Machine_Learning/Feature_Engineering_and_Performance_Metrices Branch.

Follow Contributing Guidelines & Code of Conduct before start Contributing.

This issue is only for 'GWOC' contributors of 'Machine Learning' domain.

:white_check_mark: To be Mentioned while taking the issue :

Full name
Batch Number
GitHub Profile Link
Which type of Contribution you want to make :

[ ] Documentation

Happy Contributing 🚀
All the best. Enjoy your open source journey ahead. 😎
Domain
Machine Learning
Type of Contribution
Documentation
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of this project.

Can you assign me this issue.
Jackwin Joseph
Batch : 4
GitHub link: https://github.com/JackwinJoseph3121
Documentation


Eliminate vendored sexagesimal library

Starting to remove the vendored libraries in js/lib and replace them with modules installed via npm..
Sexagesimal is a simple one to start with - how did I do?
cc @tmcw @kepta
re: #3180

LGTM! As iD starts using modules, we'll need to come to an approach for semver ranges - the new default is the ^ range, this PR uses ~. My gut feeling is shifting towards no semver specifiers at all since semver-automatic upgrading is rarely desired and incidental automatic upgrades are rarely bulletproof.

Good point about semver ranges.   I agree I'd rather not use them at all, because I don't like surprises.
@tmcw can I drop all the range specifiers and set up Greenkeeper?

👍 sounds like a plan. I haven't used Greenkeeper before, interested to hear how it goes.

 I haven't used Greenkeeper before, interested to
hear how it goes.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openstreetmap/iD/pull/3191#issuecomment-228505357,
or mute the thread
https://github.com/notifications/unsubscribe/AGpL7qdbbVaCiCT-WCoJSnTYMoJhG4MXks5qPJtcgaJpZM4I-HYp
.


Split up the notebook into multiple stages

The notebook is too large and often ends up causing memory issues + library import issues.
It's better if we instead split it up into N smaller sections. This will make it easier for students to traverse the exercises and check against the solutions.

Duplicate of #3


Log error messages when CLIPS is not found and a fatal error occurs

I had been troubleshooting my CLIPS path setting for a few hours when the path was visible and correct. It magically worked after a few restarts, and I never found out how or why it was failing to work. The logs were not available and all I could make up from the code of the extension is that it shows the 'Fatal error. Check if CLIPS is installed.' message upon encountering any error that remains unlogged.
It's generally good practice to log all errors that occur, especially fatal ones that have no clear indication as to why they occur in the first place. Also, including the path in the logs might be helpful to detect potential sync issues with the settings.

Hi, I just checked this issue. It turns out that VSCode does not output all logs to the console to an output window automatically (as I thought back when I wrote that code), but you have to do it manually.
So I have updated the code so now it logs both to the console and a dedicated "CLIPS" output window.
For context, this was a project I made for fun 2 years ago, and it was my first VSCode extension. So I was learning as I went.
Now I don't have time to maintain this project anymore, so I have added a notice in the README to reflect that the project is no longer being maintained. I will also be both closing this issue and archiving the project. Feel free to fork this project and make it your own if you wish to continue it from here.
Thanks for checking out the project and reporting this issue, I hope the extension was useful to you.


2.7.2 dev

What is the purpose of the change
XXXXX
Brief changelog
XXXXX
Verifying this change
XXXXX
Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Make sure there is a GITHUB_issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a GITHUB issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[ ] Format the pull request title like [Dubbo-XXX] Fix UnknownException when host config not exist #XXX. Each commit in the pull request should have a meaningful subject line and body.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
[ ] Run mvn clean install -DskipTests & mvn clean test-compile failsafe:integration-test to make sure unit-test and integration-test pass.
[ ] If this contribution is large, please follow the Software Donation Guide.

@arzyc What's the purpose of this PR? Why remove so many files?

@arzyc, we cannot accept such a huge PR in one single change. Pls. split your changes into multiple, within which one has single purpose, and single purpose only.
Thanks,
-Ian.


Add copy_bit functionality

This PR adds simple copy_bit functionality. It's use case is tracking boolean values for EdgeIndices in petgraph's Graph, while removing some edges - last edge is put into the place of the removed edge, and so boolean variable for the last edge should be copied according to it's new index.
I'm not sure if this is the most optimal implementation, so i'd love some comments on it.

Interesting function. I understand the need, having a variable just to handle the borrowing in the right order is a bit sad.
The issue is simply that methods have their arguments in the wrong order! :D
such a function works too:
fn set_bit(index: usize, value: bool, set: &mut FixedBitSet) {
    set.set(index, value)
}

Call using set_bit(to, fb[from], &mut fb);

Just musing, we'll merge this as it is. Good choice of method name.

Doesn't your solution introduce a small overhead (call to Vec's get followed by match on the Option as opposed to get_unchecked with if on bool)? I think that it's hard to tell without explicitly benching both solutions, I will implement them and compare results once i get some free time.

Both solutions are bounds checking, I don't think it's a difference. My gut feeling is that it's no difference.

I've just measured both implementations and it looks like there's no significant difference between the two. I'm not sure if you want that PR merged or should we leave API as it is?

I assumed you wanted to add this for ergonomics (due to the method + argument evaluation order / borrowck annoyance)

IMO this PR introduces nice interface for this operation and i would like it to be merged, I just wasn't sure what was your take on the matter :)

Don't have time to make releases now

Sorry, I forgot about this. Now 0.1.2 is released. I did tweak your function (see master branch).


Lower case for nucleotides results in error

Hi
Appreciating def. the effort of writing these tools in rust :)
Small issue which I encountered today while testing paf2maf is the fact that lower nucleotide cases in query or target will actually lead to an error:
wgatools paf2maf --query FF_SIM_1_query.fa --target  FF_SIM_1_target.fa test.paf -o /tmp/test.maf -r
2023-12-29T10:07:54.434994554+01:00 WARN file /tmp/test.maf exist, will rewrite it
2023-12-29T10:07:54.456604913+01:00 INFO start write file: `/tmp/test.maf`
2023-12-29T10:08:05.112005605+01:00 ERROR Invalid Base: `c`

Switching all my nucleotides to upper case fixed it. But I think that is something which you might encounter quite frequently and what should be fixed.

Hi @ebioman
Thanks for catching this!
I have fixed this error, please clone newest repo or install again :)
Please let me know if this can help you.


[CI:DOCS] Update basic_networking.md to include a 2nd Example for Rootless Networking

If creating a network is going to have no "network_dns_servers" by default, you guys should describe how to make this work in detail so noobs like me don't start thinking we ought to go back to Docker.  By all means have a real Tech Writer and a real Network Engineer review my proposed change. #18530
Richard Oberuc
Does this PR introduce a user-facing change?
None

I'm not sure I understand this example. Looking at Fedora running rootless containers this is what I built to model what you are doing:
#! /usr/bin/bash

LOOKUP="nslookup -type=aaaa quad9.net"

CREATED_NETWORK=0
if ! podman network ls --filter name=testing --noheading | grep testing > /dev/null
then
    CREATED_NETWORK=1
    podman network create testing > /dev/null
fi

for TYPE in default podman testing
do
    NETWORK=""
    if [ "$TYPE" != "default" ]
    then
        NETWORK="--network $TYPE"
    fi

    echo "-*-*-*- Testing: $TYPE -*-*-*-"
    ( set -x ; podman run --detach --name dns-rootless-net-$TYPE $NETWORK --quiet --rm registry.access.redhat.com/ubi9/ubi-init > /dev/null)
    podman exec dns-rootless-net-$TYPE dnf --assumeyes install bind-utils ncurses > /dev/null
    echo $LOOKUP
    ( set -x ; podman exec --tty dns-rootless-net-$TYPE $LOOKUP | \
        python3 -c "import sys; sys.stdout.write(sys.stdin.read().rstrip() + '\n')")
    ( set -x ; podman inspect dns-rootless-net-$TYPE | jq .[].HostConfig.NetworkMode)
    podman stop dns-rootless-net-$TYPE > /dev/null

    if [ "$TYPE" != "testing" ]
    then
        echo
        echo
    fi
done

if [ $CREATED_NETWORK != 0 ]
then
    podman network rm testing > /dev/null
fi

It results in:
$ bash blah.sh
-*-*-*- Testing: default -*-*-*-
+ podman run --detach --name dns-rootless-net-default --quiet --rm registry.access.redhat.com/ubi9/ubi-init
nslookup -type=aaaa quad9.net
+ podman exec --tty dns-rootless-net-default nslookup -type=aaaa quad9.net
+ python3 -c 'import sys; sys.stdout.write(sys.stdin.read().rstrip() + '\''\n'\'')'
Server:         <IP_ADDRESS>
Address:        <IP_ADDRESS>#53

Non-authoritative answer:
Name:   quad9.net
Address: 2620:0:871:9000::77
+ podman inspect dns-rootless-net-default
+ jq '.[].HostConfig.NetworkMode'
"slirp4netns"

-*-*-*- Testing: podman -*-*-*-
+ podman run --detach --name dns-rootless-net-podman --network podman --quiet --rm registry.access.redhat.com/ubi9/ubi-init
nslookup -type=aaaa quad9.net
+ podman exec --tty dns-rootless-net-podman nslookup -type=aaaa quad9.net
+ python3 -c 'import sys; sys.stdout.write(sys.stdin.read().rstrip() + '\''\n'\'')'
Server:         <IP_ADDRESS>
Address:        <IP_ADDRESS>#53

Non-authoritative answer:
Name:   quad9.net
Address: 2620:0:871:9000::77
+ podman inspect dns-rootless-net-podman
+ jq '.[].HostConfig.NetworkMode'
"bridge"

-*-*-*- Testing: testing -*-*-*-
+ podman run --detach --name dns-rootless-net-testing --network testing --quiet --rm registry.access.redhat.com/ubi9/ubi-init
nslookup -type=aaaa quad9.net
+ podman exec --tty dns-rootless-net-testing nslookup -type=aaaa quad9.net
+ python3 -c 'import sys; sys.stdout.write(sys.stdin.read().rstrip() + '\''\n'\'')'
Server:         <IP_ADDRESS>
Address:        <IP_ADDRESS>#53

Non-authoritative answer:
Name:   quad9.net
Address: 2620:0:871:9000::77
+ podman inspect dns-rootless-net-testing
+ jq '.[].HostConfig.NetworkMode'
"bridge"

TLDR; I don't observe the broken DNS stack in any case. Is this just an issue in Podman machines or possibly in WSL2? Limited to VPN or similar conditions?
Also this PR is against your fork not the main repo like I think you intended to do.

Seems like this PR is targeting your own main branch. Shouldn't it be the upstream instead?

I'm not sure about where the "pull" or "merge" goes. My "Git" skills are not very good.
The problem with setting the DNS on a "network create" is very specific to just happening on WSL2 on Windows.
Maybe this should go into a "TroubleShooting" for that?

@reo-1963 It's fine, you just need to edit the PR and change the base to containers/podman so your proposal can be discussed and processed by the Podman maintainers.


Fixed return type for methods when MULTI/EXEC is used.

As specified in official docs , if MULTI is used:

... Once in multi-mode, all subsequent method calls return the same object until exec() is called.

@michnovka Thank you for the PR! Could you please also update phpdocs for the return values to reflect that Redis can be returned in multi-mode. Something like @return bool|Redis TRUE if the command is successful or Redis in multi-mode

@isfedorov I ran it through regexp replace, but should be fine

I manually reviewed the changes, I believe the file is now ready for merge.


Prover integration

Integrating the zkLLVM's prover into SuperGenius.
The input will be a bytecode.ll file and the output will be a proof.bin file or just a binary to be recorded on CRDT/ipfs
Initially need to port the code that turns the bytecode into the circuit and assignment table (assigner and circifier) and feed those into the placeholder prover part of transpiler/crypto3

Code has been ported and now last integration step into SuperGenius.
Then testing for consensus (i.e. sending to all full ipfs nodes to store the transaction)

The proof for transfer transactions has been created and serialized into CRDT, as well as deserialized by other nodes to process locally their variables.


Open Folder (Ctrl + O on Windows) does not respect terminal.integrated.cwd setting

VSCode Version: 1.52.1
OS Version: Ubuntu 20.04.1 LTS

Steps to Reproduce:

Set "terminal.integrated.cwd" setting in .config/Code/User/settings.json to a valid path
Open a new terminal and the current directory will be set as per the setting above
Open a folder or workspace from the menu and the current directory will be the user's home directory

Does this issue occur when all extensions are disabled?: Yes/No
Yes

Should have made this a suggestion as judging by the name, the setting mentioned is only supposed to control the terminal functionality.

Should have made this a suggestion as judging by the name, the setting mentioned is only supposed to control the terminal functionality.


Test or pre-submit check to make sure types.go is consistent with output from generate_config_crd_types.go

This is to make sure:

No manual inputs are allowed.
Any change in tools or the model config would be reflect correctly (e.g https://github.com/istio/istio/pull/4246 was missing the change in https://github.com/istio/istio/pull/4305 for the script)

@diemtvu Can we close this? Is this still applicable?

I think there is already a linter for this
bin/check_pilot_codegen.sh
If you think this is still applicable, please reopen.


Book - layout of related books is broken

@xinyil thanks for catching this! the layout for the references tab was also broken for me, so I realigned both using css grid. references tab should match spec, and bibliography tab is as close as I could get for now.


⚠ Fortnite has degraded performance

In b2d7528, Fortnite (https://fortnite.com) experienced degraded performance:

HTTP code: 200
Response time: 1618 ms

Resolved: Fortnite performance has improved in 8628715.


Feature/7.x/remove upgrade assistant

This was a one off release.
We need to decide if we want this removed with 7.2 or 7.3 though.

Test failures:
7.0.0
build.bat seed:9368 integrate 7.0.0 "writable" "rolloverindex,putpipeline"

Rolloverindex test fails due to a mapping problem with the random data on this seed that needs looking into.
PutPipeline failed due to cluster ProcessClusterEventTimeoutException which I've seen happen a couple of times before with this test.
7.0.0-SNAPSHOT
Failed   Tests.Indices.IndexManagement.SplitIndex.SplitIndexApiTests.ReturnsExpectedResponse
Error Message:
 Tests.Framework.EndpointTests.ResponseAssertionException : Expected response.ShardsAcknowledged to be true, but found False.
Response Under Test:
Valid NEST response built from a successful (200) low level call on PUT: /nest-fluentasync-77a092ae/_split/nest-fluentasync-77a092ae-target?pretty=true&error_trace=true
# Audit trail of this API call:
 - [1] HealthyResponse: Node: http://localhost:9200/ Took: 00:00:30.5230768
# Request:
{"settings":{"index.number_of_shards":8}}
# Response:
{
  "acknowledged" : true,
  "shards_acknowledged" : false,
  "index" : "nest-fluentasync-77a092ae-target"
}

shards_acknowledged: false is not common here, need to understand when acknowledged and shards_acknowledged differ and what this means.
Will create new tickets for these.

Merging this in on my own accord @codebrain @russcam

This is now also cherry picked to master


Problem with struct params

#[deno_bindgen]
pub struct Inp {
x: i8,
y: i8
}
#[deno_bindgen]
pub fn xd(v: Inp)-> i8 {
v.x+v.y
}
/*compiler says...
custom attribute panicked
message: Type defination not found for Inp identifier
.......and same goes for vectors
*/

@kakashi-69-xd Hey, I recently ran into this issue as well. What operating system are you on? This is only a brief investigation, but it seems like it doesn't work on my OSX x64 machine, but does work on my GNU+Linux x64 machine. It could be tied to the operating system in some way.

it was on windows.. Linux is fine..xd


how to read PARAMS file and JSON file?

I use the command  " epoch.end.callback = mx.callback.save.checkpoint(path)", and get the following files ,
studyfolder-0001      (file type: PARAMS file )
studyfolder-symbol.json     (file type:JSON file )
How to read these files?
thanks.

I use the mxnet R packages.

there should be a mx.model.load_checkpoint

@luoruisichuan I think you can find an example of loading model from https://github.com/thirdwing/mxnet_shiny/blob/master/server.R#L13

but I use the command in R console
model <<- mx.model.load(file.choose(), iteration = 3)
and then select file 'studyfolder-symbol.json'  or  'studyfolder-0003' and all have the following problem

model <<- mx.model.load(file.choose(), iteration = 3)
错误: [22:10:03] D:\chhong\mxnet\dmlc-core\src\io\local_filesys.cc:149: Check failed: allow_null  LocalFileSystem: fail to open "E:\studyfolder\studyfolder-symbol.json-symbol.json"
model <<- mx.model.load(file.choose(), iteration = 3)
错误: [22:10:28] D:\chhong\mxnet\dmlc-core\src\io\local_filesys.cc:149: Check failed: allow_null  LocalFileSystem: fail to open "E:\studyfolder\studyfolder-0003.params-symbol.json"
model <<- mx.model.load(file.choose(), iteration = 1)
错误: [22:14:42] D:\chhong\mxnet\dmlc-core\src\io\local_filesys.cc:149: Check failed: allow_null  LocalFileSystem: fail to open "E:\studyfolder\studyfolder-0003.params-symbol.json"
model <<- mx.model.load(file.choose(), iteration = 1)
错误: [22:16:47] D:\chhong\mxnet\dmlc-core\src\io\local_filesys.cc:149: Check failed: allow_null  LocalFileSystem: fail to open "E:\studyfolder\studyfolder-symbol.json-symbol.json"

what is the problem?
thank you

The first parameter is the prefix, not the file name. Your prefix should be studyfolder.


RAT-323, RAT-378 : Harmonize  - Part 3 -- DO NOT MERGE

** DO NOT MERGE **
Requires harmonize pt 2
Fixes for RAT 378

@Claudenw pls merge in master and rebase so that we get all the latest changes from master and part2 properly. Thanks.


R8 And/Or Progaurd

Investigate the use of R8 or Progaurd as a code optimisation, obsfucation and or minimisation tool.
Additional context
See more information here: https://jakewharton.com/shrinking-a-kotlin-binary/
Apart of the https://github.com/Flank/flank/issues/774 issue chain.

The status of this is blocked until #1072 is complete


fix: utils path and change the typography hook and util

A small fix to avoid react package issues
Type of change

[x] Bug fix (non-breaking change which fixes an issue)

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my code
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

Folders always have plural names because could store multiple files. Change hook to hooks


ninja -C [path to .build] .check endlessly "Running tests" on FreeBSD

Version: <IP_ADDRESS>
OS: FreeBSD 13.0-RELEASE
I'm trying to compile a port using make with ninja but Ninja appears to be stuck checking the .build directory. It endlessly displays "[29/30] Running tests..."
If I end the process and run make install again (which activates Ninja), it shows that it enters the .build directory (in this case, .../work/tdepim-trinity_14.0.10/.build) and if I look at htop it's running ninja -C [the path..] .check, but it's using none of the CPU and only .1% of memory. Is this a known issue, an issue with something else, and is there a solution?

I think this looks like an issue with your port - the test process is stuck.


[WIP] Initial RBAC implementation

This pull request includes initial work in progress changes needed for RBAC.
I plan to finish and polish it up and get those changed merged into master soon. Permission loading from files is not there yet, but I plan to implement it soon (I will gist some examples in the upcoming days).
Keep in mind that those changes are behind a feature flag (config option, disabled by default) so they can be merged into master without any user impact.
P.S. Sorry for all the (merge) commits. I tried to be a good citizen and make general cleanup, refactoring and improvements changes in a branch based off master, get it merged into master first and then merge it back into this branch.
TODO

[ ] More tests
[ ] Finish and polish up the code
[ ] More and better docs
[ ] Loading roles and permission grants from YAML files (incremental change, can be done in a separate PR as well)
[ ] Lint script which checks that all the controllers perform permission checking
[ ] Performance optimizations
[ ] Log "access denied" errors in a special audit log
[ ] Make sure all the operations currently carry over the user who performed the operation in the context

Open questions / To discuss
What to do with other controllers where we currently don't perform any permission checking?
Things such as runner types, actions executions, history, stream, etc. We could add a new set of permission for each of those resources, but this might be an over kill right now. We could also add a single "SYSTEM_RESOURCE" permission type which applies to all the other (non content pack) resources.
How to handle action execution permissions
Should we introduce new permissions for "action executions" endpoint? We could also inherit actions permissions, but I'm not too big of a fan of that.
How to handle temporary tokens which are passed to sensors and actions
I propose to tie those tokens to a "system" user (this user can't be manipulated) which has an "admin" role.
We could (and maybe should) also tie the token to the user which invoked the action, but this gets more tricky, especially for "event" actions which are not directly triggered by the API.
Maybe we could also perform do a mix:

If an action is triggered by the API tie token to the user who triggered the action
If it's a sensor or "event" triggered actions tie it to the system user

Blacklist for globally disabled runner type
Per @lakshmi-kannan that was a feature request. Imo that seems a reasonable feature. I think we should handle it via a special "blacklisted_runner_types" (or similar) configuration option since it doesn't really fit that much into the RBAC model and the config option is imo simpler and better.
Permission which allows user to disable / enable a resource
@dzimine said someone asked for that. It would be great to get some more context into it since I don't really understand how it's useful.
In any case, this should be easily to implement via new "_modify_state" (or similar) permission type.

Alright, lets discuss RBAC definition files format.
A while ago @lakshmi-kannan and I had a short discussion about the potential RBAC definition file(s) format.
We both agreed that spliting it into multiple files might be better, since it makes it easier to write.
On the other hand, splitting it into multiple files makes it a bit harder to see / visualize the end result.
RBAC definition files format
Location: /opt/stackstorm/rbac/
Files:

``/opt/stackstorm/rbac/<role_name>_role.yaml - role definition
``/opt/stackstorm/rbac/_roles.yaml - user role assignments

Definition type (role definition vs role assignment) is inferred from the file suffix.
As an alternative, we could put files into subdirectories. For example:

/opt/stackstorm/rbac/roles/<role_name1>.yaml
/opt/stackstorm/rbac/roles/<role_name2>.yaml
/opt/stackstorm/rbac/role_assignments/<username1>.yaml
/opt/stackstorm/rbac/role_assignments/<username2>.yaml

Example role definition format
my_super_role_role.yaml
---
    name: "my_super_role"
    description: "Super cool role"
    permission_grants:
        -
            resource_ref: "mypack.myaction"
            permission_types:
               - "action_delete"
               - "action_modify"
               - "action_execute"
        -
            resource_ref: "mypack.myrule"
            permission_types:
               - "rule_modify"
               - "rule_delete"
        -
            resource_ref: "mypack.mywfaction"
            permission_types:
               - "action_execute"
        -
            resource_ref: "mypack"
            permission_types:
               - "action_execute"

Note 1: permission_grants could also be a dict where the key is resource_ref.
Note: 2: We don't need resource_type or resource_uid attribute since we can (mostly) infer resource type from the permission type. I would personally still prefer to use resource_uid which includes the type since user could potentially try to grant invalid permission type for the resource and the loading of the permissions would fail (and explicit is usually also better than implicit).
Example user role assignments format
Note: Username is parsed from the file name.
user_1_roles.yaml
---
    roles:
        - my_custom_role_1
        - my_custom_role_1

Implementation
For v1, the implementation could load role definitions and assignments when the register content script is called.
A better solution would probably be to add a separate script for loading rbac related info (e.g. ``st2 load-rbac-info` or similar).
The difference between loading those definitions and pack content from disk is that this script would perform a full sync between disk and DB content - new roles would get added, existing ones updated, and removed one would be removed. That's the same model I believe we should also eventually move to for the content pack resources.
Performing only add + merge is not a good idea in this case.
Definitions would be loaded in the following order:

Role definitions in alphabetic order
User role assignments

Instead of separate files, can it be the same file with different top-level keys or something?
Splitting up the roles feels weird to me. Conceptually, I get the reasoning - they are different concerns. But, to your point on visualization, I can see cases where something like default permissions for ChatOps would want to be shipped as a single unit.
Are groups covered here? 1:1 Role/User seems tedious, and also makes it difficult to bundle ACLs

@jfryman

Instead of separate files, can it be the same file with different top-level keys or something?
Splitting up the roles feels weird to me. Conceptually, I get the reasoning - they are different concerns. But, to your point on visualization, I can see cases where something like default permissions for ChatOps would want to be shipped as a single unit.

It can, that was the initial idea, but @lakshmi-kannan and I had some concerns. In a typical scenario, this file will also be very big.

Are groups covered here? 1:1 Role/User seems tedious, and also makes it difficult to bundle ACLs

Not sure I understand what you mean with groups. Role == group? Or you mean something else - e.g. grouping multiple roles in a group?

Example user role assignments format

Note: Username is parsed from the file name.

user_1_roles.yaml

How do I add a user to a group, and then apply groups to the role?

@jfryman There is no concept of a group right now.
It's flat so there are just roles. I assume that with groups user could group together related permissions and then apply groups to the roles.
If so, the same can be achieved with roles - user can have multiple roles assigned with it and the idea is to group related permission grants in a role (there are some w.i.p. docs available at https://github.com/StackStorm/st2/pull/1765/files#diff-0116a6a629a2ce95463ea20aace3c760R24).
I do see some value in the groups concept (if I understand what you means with groups correctly), but it makes the whole think a bit more complicated so not fully sold on it yet.

Groups are necessary when I'm shipping features, but I don't know what the end-state of my target StackStorm system is going to be. I can ship ACLs all pinned to a group, and then apply local users to those groups. Otherwise, we'd have to ship something that would allow me as an author to quickly and easily drop users into roles.
Seems like not so big a deal? Maybe with <50 users, but it will get nuts quickly. Groups should be part of this construct, imo.

Things such as runner types, actions executions, history...
Protecting history is quite important. A lot of info I would rather not others see go into input and output params of my actions. Including the very fact that by looking into "everyone's" history, I can see what actions are in the system even if I don't have permissions to view the actions. Let's think it over.

If it's a sensor or "event" triggered actions tie it to the system user.
We should inherit the user from the rule that resulted in action. Trigger can trip multiple rules. Rules may be created by different users. Those who create a rule are MORE responsible for this action than those who created the triggers/launched sensors.

How do I add a user to a group, and then apply groups to the role?
I assume user groups belong to identity provider (e.g., linux group or LDAP group), and we can map to roles both individual users and roles. Yes?

Should we introduce new permissions for "action executions" endpoint? We could also inherit actions permissions, but I'm not too big of a fan of that.
First, executions don't have good way to reference them up, other but than by properties.
For POST(create), it's reasonable that it interits action.execute.
For GET, I don't know; the most typical use cases are to see all, to see "all actions run by user=ME", or by tags, once we begin to use them more, and inherit tags by executions from actions.
Good question.

PermissionTypes
PermissionType - should they really have to be static tuples? like ACTION_VIEW? Isn't it permission assigments that mix permission type with entities? In design, permission types were VIEW/MODIFY/EXECUTE/ALL. Why combining them statically with resource types? And how to use them? I can't use ACTION_VIEW and RULE_VIEW interchably in code, or configuration.
@request_user_has_permission(permission_type=PermissionType.ACTION_VIEW)

This needs clarification.

PermissionType

I actually like the separate permissions. It allows the precise permission to be applied on groups. e.g.
---
    name: "aws_operator"
    description: "AWS operator role"
    permission_grants:
        -
            resource_ref: "aws"   # I assume this will imply the pack
            permission_types:
               - "action_view"
               - "action_execute"
               - "execution_view"

This way there is no ambiguity on what permissions the roles provide. In this case I rather have a tedious approach to permission definition as that provides control - also since role definition and permission assignment should be admin functionality I dare say that it should be ok to expose such granularity.

@dzimine

Protecting history is quite important. A lot of info I would rather not others see go into input and output params of my actions.

Alright, I will introduce new permission types for each of those endpoints and wrap them with permission checks.
I believe the existing "mask_secrets" approach already solves that. After this change is in, this will still work as before, just the admin role will be assigned differently (previously admin user was specific in the config, now you will assign admin role to the user).
We could also build on that and also show those values to every user which has "view" permission on a particular action, but this is starting to get quite complicated and expensive (it would include a lot of late filtering and I'm quite worried about the overhead).

PermissionType - should they really have to be static tuples? like ACTION_VIEW? Isn't it permission assigments that mix permission type with entities? In design, permission types were VIEW/MODIFY/EXECUTE/ALL. Why combining them statically with resource types? And how to use them? I can't use ACTION_VIEW and RULE_VIEW interchably in code, or configuration.

The change came after a discussion with @lakshmi-kannan and @manasdk a while back. They wanted a set of permissions for each resource type.
This does make some sense since not all the permission types apply to all the resource types, but we could also do this differently - store a resource_type to permission types list internally, drop resource type prefix from an external PermissionType and add a new meta endpoint which returns a list of available permission types for a particular resource type.
One of the problems we wanted to address is determining resource type without the user needing to specify an UID which encodes the resource type or additional resource_type attribute.
I personally prefer resource_uid + permission_type without the prefix approach. Something like this:
---
    name: "aws_operator"
    description: "AWS operator role"
    permission_grants:
        -
            resource_uid: "action:aws:create_instance"
            permission_types:
               - "view"
              - "execute"

Anyways, @manasdk also had a good point above (aka making it explicit).

@request_user_has_permission(permission_type=PermissionType.ACTION_VIEW)
This needs clarification.

Which part? How the decorator works, how to use it or both?
There are some docstrings in the decorators.py module right now, but I also plan to add more / improve it.

Which part? How the decorator works, how to use it or both?
the whole thing. Not this string. Never mind. We already discussing :)

I got the problem - applying permissions to a mixed bags of resources. I still don't like the proposed solution (define a bag, than apply permission-assigments (aka tuples of (resource_type, permission_type)) to the bag. Like in @manaask case, bag is a "pack". Let's think generically, how to express defining a bunch of mixed resources, and applying a set of permissions to it? Including selecting multiple resources, beyond pack/across packs, based on properties.
To throw some ideas:

Define a bag of resources with a mixture of

---
name:
permission_grants:
    -
    		# Some flexible way to define a bag of mixed resources
        resources:
           # Invent defining a group of resources
           # Specify by uid, by ref, by widlchart, by properties, some can co-exist
           ref: core.* #
           parameters:
              enabled: True
              tags:
           # Filter by resource type
           resource_type: [action, rules] # default "any"
        # Apply permissions
        permissions:
           - execute
           - view
    -
        resources:
        	uid:<PHONE_NUMBER>029
        	...

specify the bag and give hint to the system on which actions apply to which types of resources in the bag:

---
name:
permission_grants:
    -
    		# Some flexible way to define a bag of mixed resources
        resources:
           # HERE: invent syntax here which defines a group of resources
           ref: core.* #
           uid:<PHONE_NUMBER>029
           parameters:
              enabled: True
        # Apply permissions, specifying to which resource types they apply.
        permissions:
           - execute: [actions, rules]
           - view: all

For one thing, before we commit to any model and correspondent syntax, lets pick 3-4 specific use cases and see how they solve the problem.
This may surface if we are not on the same page re which cases we shall support, which is good to surface now.

Can we use users and groups interchangably here, so I can assign a role to LDAP user OR group.
What are the benefits of splitting it to file/per/user-group, as opposed to a single assigment file?
permission_grants: shall we offer blacklist + whitelist combination? call them assigments, and each array element will either "allow" or "deny" described permissions. In cases where I want to allow everything except just one action, will be easier to define.

Use cases (my quick crack, please @jfryman help refine and add more):

Assign actions, rules and sensors/triggers by pack.
Allow individual actions or rules (VERY unlikely!)
Deny individual actions or rules (QUITE likely)
Allow only executions initiated by user ME (admin sees all executions, users only see THEIR executions)
Group all relevant resources across packs by a tag, and assign them all
Allow/restrict access to specific KV-store sections
Allow only disable-enable, and deny other CREATE/CHANGE, for rules and actions (Netflix/@phool ask).

Couple of syntax pointers:

AWS IAM Policy structure http://docs.aws.amazon.com/IAM/latest/UserGuide/policies_overview.html#policies-introduction
I want to place OpenStack as an example but it’s almost serves as an anti-example: http://docs.openstack.org/kilo/config-reference/content/section_keystone-policy.json.html

I do think action_view, action_modify kinda permissions are more clear. These can be applied to groups of resources or individual resources without any problems.

I definitely do think we need grouping constructs for specifying permission grants to resources. We can start with a list of resources called out by names and allow glob expressions. So +1 to dzimine's suggestion.

permission grants:
  -
  resources: [list of resources by explicit names or glob]
  permissions: [list of permissions]

What are the benefits of splitting it to file/per/user-group, as opposed to a single assigment file?

IMO manageability. We'll start hating the one file structure as soon as we have serious number of content/users. I like Tomaz's current approach of convention based.

I do agree with jfryman on group level for users. I don't think this is needed for initial prototype though. If we make our models flexible, it seems we can add this construct later.

permission_grants: shall we offer blacklist + whitelist combination? call them assigments, and each array element will either "allow" or "deny" described permissions

Usually, it's a best practice to deny everything and start allowing things. I don't know if that's the right approach for RBAC though. Maybe we can have a global config for deny all or allow all by default. If there is a RBAC schema, then it trumps everything.

@dzimine

Couple of syntax pointers:

AWS IAM Policy structure http://docs.aws.amazon.com/IAM/latest/UserGuide/policies_overview.html#policies-introduction
I want to place OpenStack as an example but it’s almost serves as an anti-example: http://docs.openstack.org/kilo/config-reference/content/section_keystone-policy.json.html
It is generally good to base on something people are generally familiar, like AWS (all other things being equal).

I personally think they are both anti-patterns.
AWS one is powerful, but it's anything but simple and easy to understand (writing that JSON is also not fun).
I'm trying to find a middle ground between "enterprise" (1000 switches and options, hard to understand write and manage, everyone hates it including myself) and "startup" (in some cases "maybe" not flexible enough) solution.

Use cases (my quick crack, please @jfryman help refine and add more):

Assign actions, rules and sensors/triggers by pack.
Allow individual actions or rules (VERY unlikely!)
Deny individual actions or rules (QUITE likely)
Allow only executions initiated by user ME (admin sees all executions, users only see THEIR executions)
Group all relevant resources across packs by a tag, and assign them all
Allow/restrict access to specific KV-store sections
Allow only disable-enable, and deny other CREATE/CHANGE, for rules and actions (Netflix/@phool ask).

Most of those things are possible with the existing implementation (other things which are not possible right now can be implemented later as add-on features). Only thing which is not possible is "Deny individual actions or rules (QUITE likely)". To keep things simple and easy to understand (no possibility of an overlap and conflicting grants), we default to "Deny all" and allow users to grant "allow" on particular resource or set of resources.
If more people think we definitely need "deny" I can implement it, but do keep in mind (as noted above) that this will increase complexity and will make it harder to understand. In any case, it would be great if we can get on the same page on this ASAP.
Another thing which is not possible right now is granting permissions on tags. That's by choice to reduce the scope. I consider it as an add-on feature (easy to add later on) and existing design accounts for that.

@lakshmi-kannan

I definitely do think we need grouping constructs for specifying permission grants to resources. We can start with a list of resources called out by names and allow glob expressions. So +1 to dzimine's suggestion.

We already have one grouping concept right now and it's called a pack. In the future we will have another one - "tags".
Glob expressions make some sense (for things such datastore item keys), but I don't think we need another concept in addition to those. It doesn't add anything, just complexity (everything can already be achieved with the existing constructors / concepts).


Compatibility of OpenCV 2 and 3

Would you mind keeping the fourcc construction for OpenCV 2 as follows:
                fourcc = cv2.cv.CV_FOURCC(
                    chr(ord('T')),
                    chr(ord('H')),
                    chr(ord('E')),
                    chr(ord('O'))
                )

For some reason, the fourcc = cv2.cv.CV_FOURCC(*'THEO') syntax doesn't work inside Facebook's codebase.

@lvdmaaten I've made the change, please see if this is what you mean.

Great, thanks @zuoxingdong!


Deploying a HPA using the kubeless cli when one already exists causes an error loop in the function controller.

Is this a BUG REPORT or FEATURE REQUEST?:
BUG
What happened:
If a horizontal pod autoscaler definition already exists and you deploy it again using the kubeless cli like so:
$ kubeless autoscale create --max 6 --min 2 --value 100 "$function"

It will bubble up an error from the kube api with logs like this:
time="2020-02-28T05:06:23Z" level=info msg="Processing change to Function alchemy-faas/cisco-umbrella-get-policy-settings" pkg=function-controller
time="2020-02-28T05:06:24Z" level=error msg="Function can not be created/updated: horizontalpodautoscalers.autoscaling \"cisco-umbrella-get-policy-settings\" already exists" pkg=function-controller
time="2020-02-28T05:06:24Z" level=error msg="Error processing alchemy-faas/cisco-umbrella-get-policy-settings (giving up): horizontalpodautoscalers.autoscaling \"cisco-umbrella-get-policy-settings\" already exists" pkg=function-controller
ERROR: logging before flag.Parse: E0228 05:06:24.897570       1 function_controller.go:185] horizontalpodautoscalers.autoscaling "cisco-umbrella-get-policy-settings" already exists

Which repeats until the max retry limit is reached.
Due to the use of a rate limiter when processing the queue, this causes the function controller to start slowing down the queue consumption which had follow on issues in my case (actually the cause of serverless/serverless-kubeless#199).
What you expected to happen:
If the HPA definition already exists, then do a patch instead of create here:
https://github.com/kubeless/kubeless/blob/3eff84c61ac407eb471d060cf2c48637ca73c2ce/pkg/controller/function_controller.go#L369
This can be done the same way it's currently handled for deployments here:
https://github.com/kubeless/kubeless/blob/3eff84c61ac407eb471d060cf2c48637ca73c2ce/pkg/utils/kubelessutil.go#L710
How to reproduce it (as minimally and precisely as possible):
Use kubeless cli to deploy a HPA twice and watch the function-controller logs.
Anything else we need to know?:
I'd be happy to provide a pull request if you like.
Environment:

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-11T18:14:22Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.10-gke.17", GitCommit:"bdceba0734835c6cb1acbd1c447caf17d8613b44", GitTreeState:"clean", BuildDate:"2020-01-17T23:10:13Z", GoVersion:"go1.12.12b4", Compiler:"gc", Platform:"linux/amd64"}

Kubeless version (use kubeless version): v1.0.5-8-g59da64f8
Cloud provider or physical cluster: gke

this can be closed then, right @d0x2f ?

Yep, I'll close it.


Push to GItHub fails

Hello,
Thank you for the great software! I'm having a little problem, though. I receive the message "GitHub: Invalid request. "sha" wasn't supplied." When I attempt to Push my system to GitHub from my laptop. Pull works fine on the device. Both Push and Pull work fine on my iPad using the same GitHub token.
System Designer version number: 3.3.0
System Designer platform: Windows
Your OS version: 10
Steps to reproduce the problem:

Install software from MS Store.
Setup GitHub token as described in the docs
Pull system (succeeds)
Push system (fails with error)

Thank you for reporting the issue! I am able to reproduce it. I will check where it comes from and will fix it.

I found the issue (related to the way browsers cache requests). A fix is coming soon.

I plan to publish a new version of System Designer on the stores this weekend. Depending on the validation process, it will be available at the end of next week I think.


Update mstest monorepo to v3 (major)

This PR contains the following updates:

Package
Type
Update
Change

MSTest.TestAdapter
nuget
major
2.2.10 -> 3.1.1

MSTest.TestFramework
nuget
major
2.2.10 -> 3.1.1

Release Notes

microsoft/testfx (MSTest.TestAdapter)
v3.1.1
See the release notes here.
v3.0.4
See the release notes here.
v3.0.3
See the release notes here.
v3.0.2
See the release notes here.
v3.0.1
See the release notes here.
v3.0.0
See the release notes here.

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about these updates again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

Renovate Ignore Notification
Because you closed this PR without merging, Renovate will ignore this update. You will not get PRs for any future 3.x releases. But if you manually upgrade to 3.x then Renovate will re-enable minor and patch updates automatically.
If you accidentally closed this PR, or if you changed your mind: rename this PR to get a fresh replacement PR.


How to get and set the fx and fy of the camera from the test-rendering？

Is there any way to get or set the parameter of the camera in test-rendering (python file). And the which kind of device in parser.add_argument('--device', type=int, default=0) can be set?

https://github.com/facebookresearch/House3D/issues/42

device is an integer. It means which GPU to use.

Thanks a lot


Is there anyway to disable the MSSQL sink in appsettings.json without replacing all Serilog configuration?

We have some deployment scenarios where we do not want to have any connections to the database. Does that mean we need to move the entire Serilog:WriteTo:Mssql section to the production config just to prevent connecting to the database? Is there an easy way to disable the sink other than removing it entirely?

Adding the following to appsettings.Production.json file will do for now:
"Serilog:WriteTo:Mssql:Args:restrictedToMinimumLevel": "Fatal"
Is there any other option?

Adding the following to appsettings.Production.json file will do for now:
"Serilog:WriteTo:Mssql:Args:restrictedToMinimumLevel": "Fatal"
Is there any other option?

Hi and thanks for your question!
You could also do the sink initialization programmatically (as shown in the CombinedConfig sample) an then add or don't add the MSSQL sink based on a config value of your choice. If you initialize in code you can still have the sink read most of its values directly from the config. The sample demonstrates this.

Hi and thanks for your question!
You could also do the sink initialization programmatically (as shown in the CombinedConfig sample) an then add or don't add the MSSQL sink based on a config value of your choice. If you initialize in code you can still have the sink read most of its values directly from the config. The sample demonstrates this.


How to assemble an 80960KB program from a disassembly?

I am assuming Sega originally used a tool provided by Intel called CTOOLS. However, a disassembly might be able to be built with GNU960. The last supported GCC for the i960 architecture was 3.4.6 (See "Obsolete Systems"). However, this is a disassembly, not source code, so we would need binutils, and the last build that supported the i960 architecture that I could successfully compile using cygwin was binutils 2.16.1a, both ELF and COFF formats.
So what would be required to build a disassembly of an i960 program? If deemed too difficult, however, is this repository purely going to be for research purposes?

To be honest, I didn't think about that yet. For the time being I'm just document that assembly.

Alright, but for future reference, I unearthed an 80960 Windows-compatible assembler from CTOOLS 6.0 here with thanks to a professor at the Oregon Institue of Technology. Also if you're interested in turning a disassembly into C code, might I suggest the ghidra_i960 project. It's not 100% yet, but I am working with the author to help assist in my own Model2 disassembly of a different game.

Linux compatible GCC for both coff and elf formats were compiled by @DrItanium. 80690 assembly files can be built from this, however getting a disassembly to a point where it can be built with the tools is a different issue itself. Closing.


Accept async functions

https://github.com/github/github/pull/111598#discussion_r272512629 & https://github.com/facebook/flow/issues/5882.
As far as I can see TypeScript does not seem to care? 🤔
cc @josh

My original thinking here is that I considered these async callbacks a footgun.
Accepting a promise return value may indicate that caller will wait on that return--which selector-observer can not implement correctly.
Internally, selector-observer preserves the invariant that a remove always follows add. You'll always get add, remove, add, remove. But never see any behavior like add, add, remove, remove on a single element. This is easier promise given a single event loop tick. Things seem a little harder to predicate once async code is introduced.
Consider:
// using a singleton element to simplify the example
// invariant: count will always been 0 or 1
let count = 0

observe("#singleton", {
  add() {
    count++
  },
  remove() {
    count--
  }
})

When adding async I'd expect the behavior to be preserved. And it kinda does because of the nature of async functions being implemented called.
let count = 0
observe("#singleton", {
  async add() {
    count++
  },
  async remove() {
    count--
  }
})

But once you though an await into the mix (which you probably are if you want to use async), then the invariant starts to break.
let count = 0
observe("#singleton", {
  async add() {
    await madness()
    count++
  },
  async remove() {
    count--
  }
})

In these cases, I think it's better that the user introduce their own async IIFE.

That makes sense. Would you be open to only accepting async on InitializerCallback then? That is what the linked use case needs.

Would you be open to only accepting async function on InitializerCallback then?

I think that's another confusing case. The initializer hook looks for a truthy value returned from the initialize. In the current case any null or void would be correctly ignored. But a Promise<void> would be considered truthly. And there's no runtime way to know without awaiting the promise.
https://github.com/josh/selector-observer/blob/cd822ea2e7eb9c556cefe2de26d8a6314df4c50f/lib/apply.js#L49-L51
I kinda extended initialize in a really sloppy way to support this return value. So I think we're boxed in.
Without adding additional hooks/features to support async hooks, I don't think this is going to be easy to do in a safe way.
GitHub's use case is mainly using the async function syntax out of convenience. I don't think we actually care about the async initialization semantics. So it's probably not worth the effort to extend.

Ah I forgot about InitializerCallbacks.

GitHub's use case is mainly using the async function syntax out of convenience. I don't think we actually care about the async initialization semantics. So it's probably not worth the effort to extend.

Fair. 🙆


Build pipeline

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: pinalab sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Added etcd-druid documentation

Proposed Changes
Added etcd-druid installation instructions.

Thanks a lot for your feedback. I just made the adjustments.


Vue2 and Multselect fail after 2.7 backport of composition API

Version

Vue version: 2.7.3

Description
Multiselect uses the Composition API and in order to make Vue2 work, you need to use the @vue/composition-api dependency. I've been using Multiselect, Vue2 and the Composition plugin with no issues until today when I upgraded to the latest version of vue...which backports the composition API.
I tried to just use the vue3 Multiselect and that doesn't work. The errors you get are just nonsensical things like, "classList doesn't exist.
Demo
I can't reproduce in fiddle because it's not running the latest version of Vue 2.

Also, to confirm, I was able to downgrade Vue to 2.6.14 and Multiselect does start working again.

I tried explicitly adding Vue 2.6.14 as a dependency, but it doesn't work for me using Nuxt 2.

Is it possible to just change
"vue-prev"<EMAIL_ADDRESS>"vue-prev-composition-api": "npm<EMAIL_ADDRESS>to
"vue"<EMAIL_ADDRESS>?

Vue.js 2.7 includes Composition API so there are two changes you need to do in order to use the Multiselect:

do not import & use @vue/composition-api
alias @vue/composition-api to the Vue build you're using (eg. node_modules/vue/dist/vue.runtime.esm.js)

So during the installation this step can be left out:
import Vue from 'vue'
import VueCompositionAPI from '@vue/composition-api'

Vue.use(VueCompositionAPI)

And this needs to be added, eg. when using Vite:
// vite.config.js

import { defineConfig } from 'vite'
import vue from '@vitejs/plugin-vue2'
import path from 'path'

export default defineConfig({
  plugins: [vue()],
  resolve: {
    alias: {
      '@vue/composition-api': path.resolve(__dirname, 'node_modules/vue/dist/vue.runtime.esm.js'),
    }
  },
})

In Nuxt 2 aliasing works this way:
// nuxt.config.js

import path from 'path'

export default {
  alias: {
    '@vue/composition-api': path.resolve(__dirname, 'node_modules/vue/dist/vue.runtime.esm.js'),
  }
}


[Document] Add new allowRemove option in the documentation

Is your feature request related to a problem? Please describe.
In the latest version v4.20.0 there is a new option called allowRemove. However, this is not covered in the docs.
Describe the solution you'd like
Add this new option in the docs.
Describe alternatives you've considered
NA

Done ✅


Explicitly define sidebar instead of autogenerating

This is just generally nicer, but also will let us add some structure/hierarchy in the sidebar as we add content.
See: https://docusaurus.io/docs/sidebar/items

Oh, and I should also mention that I reordered things according to my proposal on discord.  Feel free to shuffle around further of course


Change in font of Heading

@yeshijinsel  would be please add those fonts and import it.

ok

but which font
is it( https://fonts.google.com/specimen/Varela+Round?selection.family=Varela+Round ) this one

Yes you can use any font you like but let it be cool and from google font.

@yeshijinsel  this is not html only bro. It is a mikrotik router template. So we cannot include using link tag. Hope you understand. And thanks for your interest. There a couple of problems in the codebase which you can clearly see. Thanks


List of new features

@Joshua106728 @bentneb9 @SlpyBkWrm @Masonnunya
What features are missing from the map? Should the user be able to add multiple layers to the map? Add to the list!

[ ] Check incoming Overpass queries to make sure they are valid
[ ] Make sure coordinates are valid
[ ] Allow users to search for a place to bring them there (i.e., "Bring me to Washington, DC")
[ ] Custom colors for layers

[ ] - Suggestions from Ellie, Mason, and Caden. Add Identifiers to the popups. ex: "you've clicked on 'The Leaky Cauldron'" like so? And when you click on a road it tells you which road and so on.
[ ] - A popup on the screen that tells you when the query is done running.
[ ] - Another thing I noticed is that in order to see the queries you have to manually go into the layers section to select them before they show, perhaps there is a way to make that process automatic instead?
[ ] - Maybe something to prevent overlapping queries? ie: no overlapping colors
[ ] - Also in the layers if there was an indicator of what color everything is? like a red square or dot next to an oways layer for example.
[ ] - Is there a way to add labels to the ersi layer map? It would be dead useful to be able to navigate a bit smoother with road labels.
more to be added

petition to change default highways style to cornflower blue #6495ED


Addresses misc feedback

Addresses feedback for several recent PRs, not including the property wrapper one. There's also a change in here to make XPCServer be consistent with XPCClient in making it clear what type of requirement is needed (now that are two different kinds).

@amomchilov For your reviewing pleasure.
FYI I'm about to take a long weekend to go skiing, TBD whether I'm going to take my laptop with me - so might be non-responsive until next week. At which point I'll go reengage with property wrapper PR.

@jakaplan  Enjoy your trip, and thanks for patching all this feedback in!


Delete all

also contains changes to change password:

move to own component since it didnt belong in toptoolbar
now uses server feedback & displays it to user
better rules, local checking before submitting to server
fix the layout the toggle eyes were out of the input field

closes #448

TODO: clean up & fix build errors

there are some ts-ignores cuz otherwise I cant access the field and I have no idea how to fix it

Could you change the color of the 'advanced options' to black?
Maybe redirect projects or students when having deleted something using href (not router) so the storage will be deleted?
Instead of showing an error when 'DELETE COACHES' is not typed in, you could disable the 'DELETE' button.
Could you fix the Vue build errors?

Codecov Report

Merging #451 (89ce168) into main (56c6f7b) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #451   +/-   ##
=======================================
  Coverage   88.48%   88.48%
=======================================
  Files          12       12
  Lines         990      990
=======================================
  Hits          876      876
  Misses        114      114

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@lisadejonghe

Could you change the color of the 'advanced options' to black?
Do you mean when the tab is selected or the title of the page or ?
Maybe redirect projects or students when having deleted something using href (not router) so the storage will be deleted?
Maybe you want to delete more so I'd just stay on the page
Instead of showing an error when 'DELETE COACHES' is not typed in, you could disable the 'DELETE' button.
Tried it doesn't work
The 'change password' button is broken? It works on main.
You broke it when merging main into the branch, you forgot to import the ChangePasswordDialog
Could you fix the Vue build errors?
They were fixed not sure what went wrong

@lisadejonghe

Could you change the color of the 'advanced options' to black?

Do you mean when the tab is selected or the title of the page or ?

I mean the selected tab.

Maybe redirect projects or students when having deleted something using href (not router) so the storage will be deleted?

Maybe you want to delete more so I'd just stay on the page

Why would you want this, that's the reason for the checkboxes, no?

Instead of showing an error when 'DELETE COACHES' is not typed in, you could disable the 'DELETE' button.

Tried it doesn't work

There must be a way to make it work no?

The 'change password' button is broken? It works on main.

You broke it when merging main into the branch, you forgot to import the ChangePasswordDialog HERE

Woops, my bad.

Could you fix the Vue build errors?

They were fixed not sure what went wrong

Using // @ts-ignore is not really a solution if that's what you mean.

Maybe redirect projects or students when having deleted something using href (not router) so the storage will be deleted?

Maybe you want to delete more so I'd just stay on the page

I also think it's better to route to /projects, I would suspect the user would check all the things it wants to delete, and he/she does not need to be on that page anymore. Also, is it possible to reset the stores after the data has been deleted?

Instead of showing an error when 'DELETE COACHES' is not typed in, you could disable the 'DELETE' button.

Tried it doesn't work

I also think this is a cleaner, more intuitive way of doing this. I have done something similar in CreateProject.vue, maybe you could have a look at it?

@lisadejonghe

Could you change the color of the 'advanced options' to black?

Do you mean when the tab is selected or the title of the page or ?

I mean the selected tab.

this is the same as the toptoolbar the selected part is highlighted

Maybe redirect projects or students when having deleted something using href (not router) so the storage will be deleted?

Maybe you want to delete more so I'd just stay on the page

Why would you want this, that's the reason for the checkboxes, no?

I would just stay on the page to make it clear that it worked & so u can delete other things as well if you want to do one by one. But if you really want a redirect then so be it.

Instead of showing an error when 'DELETE COACHES' is not typed in, you could disable the 'DELETE' button.

Tried it doesn't work

There must be a way to make it work no?

Yeah if you check after every typed character if the button can be turned on, but it's kinda overkill and useless since pressing it wont do anything anyways.

The 'change password' button is broken? It works on main.

You broke it when merging main into the branch, you forgot to import the ChangePasswordDialog HERE

Woops, my bad.

No problem

Could you fix the Vue build errors?

They were fixed not sure what went wrong

Using // @ts-ignore is not really a solution if that's what you mean.

If you scroll up a bit I said I tried to fix it but I couldn't but the build errors now are different ones and showed up when u merged main into this branch.

The redirect is indeed not that good since the download buttons are also on that page.

Instead of showing an error when 'DELETE COACHES' is not typed in, you could disable the 'DELETE' button.

Tried it doesn't work

There must be a way to make it work no?

Yeah if you check after every typed character if the button can be turned on, but it's kinda overkill and useless since pressing it wont do anything anyways.

No, you do something like :disabled="var === 'DELETE STUDENTS'". Have you had a look at my example code?

Could you fix the Vue build errors?

They were fixed not sure what went wrong

Using // @ts-ignore is not really a solution if that's what you mean.

If you scroll up a bit I said I tried to fix it but I couldn't but the build errors now are different ones and showed up when u merged main into this branch.

This is because you did not use defineComponent in the new Vue file. If you add that, you'll be able to fix your ts issues.

Disabled is fixed.

Instead of showing an error when 'DELETE COACHES' is not typed in, you could disable the 'DELETE' button.

Tried it doesn't work

There must be a way to make it work no?

Yeah if you check after every typed character if the button can be turned on, but it's kinda overkill and useless since pressing it wont do anything anyways.

No, you do something like :disabled="var === 'DELETE STUDENTS'". Have you had a look at my example code?

if only it was that simple, you don't know the amount of variables, the only way to do this is by calling a function every time any of the fields is changed and then setting the variable in the :disabled but I felt like this is overkill for a feature that's only used 1 time but feel free to add it

Could you fix the Vue build errors?

They were fixed not sure what went wrong

Using // @ts-ignore is not really a solution if that's what you mean.

If you scroll up a bit I said I tried to fix it but I couldn't but the build errors now are different ones and showed up when u merged main into this branch.

This is because you did not use defineComponent in the new Vue file. If you add that, you'll be able to fix your ts issues.

oh I'll look into that

fixed build issues, delete all is finished


Restructure based on deliverables

Detail
To sync up the repos and project board with the website, we are converting the DARE funding proposal into an easier to read MarkDown file with labeled deliverables. This links into a slight restructuring of the website to make it more accessible to others.
Atm we don't have a new design, but want to start adding content - once we have the content we want we can start to how a website will shape up!
Actions

[x] Start the restructure
[x] Add the deliverables one pager as referenced here: https://github.com/uk-tre/community-management/issues/21

Who can help
No response

This is being covered in #21, needs review

Where do we stand with the associated PR, it is approved but I am unsure wheter or not we addressed redirects.
Also slightly confused what the difference is between this issue and https://github.com/uk-tre/community-management/issues/21
https://github.com/orgs/uk-tre/projects/1?pane=issue&itemId=44289577 depends on finalising this one as well to create the right categories for the board view

This has been on hold waiting for the new website, the associated PR is likely to be closed and a new one wiht updated content created

Not relevant anymore, see issue 16 in website repo for latest details


Randomize modparameter

This allows you to create randomized parameters for the synth. I'd like this for unit-testing numpy vs torch.
Depends upon #33
What's weird is that the note_on_duration is sometimes disregarded and you get like 16 second samples. Is this because of the one-hit thing @maxsolomonhenry ? That seems like a problem to me. You should be able to hillclimb the synth by setting random params to try to get a similar audio of the same duration :\

As per this issue we need to reconsider our parameter min-maxs. A 20 second maximum attack will get you into this mess.

As per this issue we need to reconsider our parameter min-maxs. A 20 second maximum attack will get you into this mess.

@maxsolomonhenry The issue is orthogonal to the one you're talking about. It's more than if you have a SynthModule (or Drum) and call "randomize_parameters", you don't expect the sound length to change. One you have a synth set up and start flipping around the parameters to find a particular sound, you don't want the sound length to vary.

@maxsolomonhenry The issue is orthogonal to the one you're talking about. It's more than if you have a SynthModule (or Drum) and call "randomize_parameters", you don't expect the sound length to change. One you have a synth set up and start flipping around the parameters to find a particular sound, you don't want the sound length to vary.

Again we're not hillclimbing note_on_duration. That's a hyperparameter.
If you're randomizing, the audio could have a max time value, based on the Parameter max values.

Again we're not hillclimbing note_on_duration. That's a hyperparameter.
If you're randomizing, the audio could have a max time value, based on the Parameter max values.

Randomizing the release time parameter would lead to samples with different lengths, right? Max length of the sample would be note_on_duration + maximum release time. One thing we could do to fix the length of the sound is to introduce another hyperparameter, something like render_length -- this could set the length of the output. For example, if the note_on_duration was 1s and that render_length was 2s, then the release time portion would just be cut off at 1s regardless of the actual release time.

Randomizing the release time parameter would lead to samples with different lengths, right? Max length of the sample would be note_on_duration + maximum release time. One thing we could do to fix the length of the sound is to introduce another hyperparameter, something like render_length -- this could set the length of the output. For example, if the note_on_duration was 1s and that render_length was 2s, then the release time portion would just be cut off at 1s regardless of the actual release time.

@jorshi yes I like the idea of a render_length hyperparameter.

@jorshi yes I like the idea of a render_length hyperparameter.


the parameters is what ?

first parameters is .p7m file data->encodingwith utf8?
second parameters is pri_key.pem file data encoding with utf8?
third is .cer file data encoding with utf8 ?

Here is how i decrypt
class DecryptHelper {
    public class func decrypt(encrypted: String) throws -> String {
        guard let pkData = CSRGenerator.getPrivateKeyBits(CSR.Tag) else {
            throw "invalid private key"
        }
        let pKey = "-----BEGIN RSA PRIVATE KEY-----\n" + pkData.base64EncodedString() + "\n-----END RSA PRIVATE KEY-----"
        guard let certificate = SettingsHelper.get(key: "certificate") else {
            throw "invalid certificate"
        }

        let clean = encrypted.replacingOccurrences(of: "\n", with: "")
        guard let data = Data(base64Encoded: clean) else {
            throw "invalid base64 encoded script"
        }
        let enc = String(data: data, encoding: .utf8)

        guard let decrypted = ScriptDecrypt.decrypt(enc, privateKey: pKey, certificate: certificate) else {
            throw "invalid encrypted script"
        }
        return decrypted
    }
}

+ (NSString *)decrypt:(NSString *)encrypted privateKey:(NSString *)privateKey certificate:(NSString *)certificate {
    const char *enc = [encrypted UTF8String];
    const char *pk = [privateKey UTF8String];
    const char *cert = [certificate UTF8String];
    const char *decrypted = decrypt_smime(enc, pk, cert);
    if (decrypted) {
        return [NSString stringWithUTF8String:decrypted];
    } else {
        return nil;
    }
}

I get privatekey : generate private.pem file with .pfx file
cmd: "openssl rsa -in 1.key -out private.pem "
then get the private.pem data to utf8.string
it's
-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCA....
-----END RSA PRIVATE KEY-----
the cer : generate .cer with .pfx file
cmd: openssl pkcs12 -nodes -nokeys -in 1.pfx -passin pass:my pwd -out xxx.cer
then get the .cer file data to utf8.string
it's
Bag Attributes
localKeyID: 01 00 00 00
friendlyName: le-41f02d05-e0fc-4dea-8c9e-5d256588e8cd
Microsoft CSP Name: Microsoft Enhanced Cryptographic Provider v1.0
Key Attributes
X509v3 Key Usage: 10
-----BEGIN PRIVATE KEY-----
MIIEvAIBADANBg...
-----END PRIVATE KEY-----
Bag Attributes
localKeyID: 01 00 00 00
<EMAIL_ADDRESS>issuer=/DC=com/DC=langchao/DC=home/CN=INSPUR-CA
-----BEGIN CERTIFICATE-----
MIIGkjCCBX...
-----END CERTIFICATE-----
Bag Attributes:
subject=/DC=com/DC=langchao/DC=home/CN=INSPUR-CA
issuer=/DC=com/DC=langchao/DC=home/CN=INSPUR-CA
-----BEGIN CERTIFICATE-----
MIIDojC...
-----END CERTIFICATE-----
enc : got the enc data from x.txt file
it's
MIAGC....AAAAAA=
but
PKCS7 *pkcs7 = getContainer(encrypted);
the pkcs7 is null

How can I put the encrypt data to a PKCS7 container?
Your API just input enc..
is that enc have a special format?
now the encrypted data is "MIAGC....AAAAAA="
should I format it?

This utility is specifically for decrypting PKCS7 containers. It is not used to create PKCS7 encrypted data. Typically the use case for PKCS7 encryption is when the data you need to encrypt is > 2048 bytes. IOW if you have a large payload you need encrypted, then you would encrypt using openssl or similar. Once you have that encrypted payload, you can decrypt it using this utility.

Ruby:
   def encrypt_and_sign plaintext
     key =<EMAIL_ADDRESS>
     cipher = OpenSSL::Cipher.new('AES-256-CBC')
     p7=OpenSSL::PKCS7::encrypt([key], plaintext, cipher, OpenSSL::PKCS7::BINARY)
     smime_text = OpenSSL::PKCS7::write_smime(p7)

     return Base64.encode64(smime_text)
   end

OpenSSL command-line is easier:
openssl smime -encrypt -aes256 -to bob -from mike -subject 'wut' -text -in file-to-encrypt.txt bob.crt > encrypted-file.smime

let data = try! Data(contentsOf: URL(string: path)!)
let receiptBIO = BIO_new(BIO_s_mem())
BIO_write(receiptBIO, (data as! NSData).bytes, Int32(data.count))
let pkcs7 = d2i_PKCS7_bio(receiptBIO, nil)
if pkcs7 == nil  {
print("??")
}
return pkcs7!
}
parameter is .P7M file path
done.


Publish main tests as test jar

Fixes #104
/cc @aloubyansky
I also remove the skip release flag as I assume I will need to release the test jar.

Looks good @loicmathieu. If you want to try integration into the platform before you release, you could add your config similar to e.g. https://github.com/quarkusio/quarkus-platform/blob/main/pom.xml#L190 (make sure this value is set to true https://github.com/quarkusio/quarkus-platform/blob/main/pom.xml#L193)
The first build will take awhile but subsequent builds should be ~15 sec.

@aloubyansky thanks, I didn't thought of trying locally the platform. I'll do it before pushing a PR to it to include the library as a member.
Anyway, I'll merge this one as I'll need a release to open a PR to the platform. I'll publish a 2.0.0.RC1 soon.


CentOS 7.1 breaks params.rb

    $majdistrelease = regsubst($::operatingsystemrelease,'^(\d+)\.(\d+)','\1')

Does not work on CentOS 7.1.1503. That variable does not seem to exist.
lsbdistdescription => CentOS Linux release 7.1.1503 (Core)
lsbdistrelease => 7.1.1503
lsbmajdistrelease => 7
lsbminordistrelease => 1
Thus you get:
Mar 31 17:43:10 b0c008ash2001 puppet-user[42116]: comparison of String with 5 failed at /var/lib/puppet/repo/modules/snmp/manifests/params.pp:246 on node b0c008ash2001.mgmt.symcpe.net
TIA,

What value does facter operatingsystemmajrelease return?

Which version of the centos-release package do you have installed?  You should have release el7.centos.2.8, otherwise it is a CentOS bug. http://lists.centos.org/pipermail/centos-announce/2015-April/021010.html

Yep you can close this, CentOS 7.1 screwed up and fixed in latest :)
Thanks dude!


object fit property doesn't work on images

Describe the bug
object fit property doesn't work on images if added through nativewind classname, however if i add it through a style object like style={{ objectFit: "contain"}}, then it works
To Reproduce
Try to use object-contain or otherwise on an image

Any update on this?

We are closing all pre-v4.1 issues now that v4.1 is released. If the problem is still happening with the latest v4.1, please open a new issue and reference this one in the description.


RLOS - Using Mils for Horizontal FOV gives incorrect results

The results returned when using mils are incorrect (Update: 5 degrees off for ending angle), the following two examples should be 90 degrees for 1600 mils and 180 degrees for 3200 mils:

The conversion factor was incorrect for this so this one should be an easy fix:

verified fix with build 1615 in pro and arcmap


Bug: absURL to relURL

Description

Change all absURL to relURL
Motivation and Context

Fixes #80
How Has This Been Tested?

Hugo Version: v0.3
Browser(s): Chrome
Screenshots (if appropriate):
Types of changes

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[X] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation, including theme.toml, accordingly.
[X] I have read the Contributing Document.

@pacollins, maybe my description has not been accurate enough. This applies not only to the links in header.html and footer.html where actually absURL ist used but also to sitebar.html and other places where the absolute URL is used.

@VeitSanner - I actually had to revert because I noticed this, had to drive home and was going to address this in your thread.


Any plans to improve accuracy?

Colour pallet considerations aside, are there any plans on reproducing VIC-II quirks that rely on its video output circutry such as PAL colour blending and NMOS state transition times?
The first is a quirk that allows for the creation of more colours than the VIC-II can natively produce and is a by-product of the PAL signal generation.  More C64 pixelers are starting to take advantage of this.  Minor example:

(I haven't build your board.  The original image was borrowed from a poster over on the Lemon64 forums)
The white circle represents colour mixing that will happen with a 'old luma' VIC-II while the yellow circles are for both the new and old luma VIC-IIs.
This article (https://kodiak64.com/blog/luma-driven-graphics-on-c64) on the matter is well worth the read.
The second isn't as well known but is seen any time black is used on screen.  The NMOS process that the VIC-II is built on has a quirk that a gate will transition quickly from high to low and slowly from low to high.    Conversely, singular coloured/white pixels will be noticeably smaller on a black background.  Pixelers over the decades will have at least been subconsciously accounting for this effect in their work.

I am not completely sure how this color blending actually works on old PAL TVs, but I guess it somehow mixes the color from two lines to even out transmission errors. This effectively halves the vertical resolution of the color signal, but as humans tend to see variations in luminance better than variations in colors, this was some viable compromise.
But this whole scheme is completely restricted to the PAL system (meaning the color transmission mechanism that was used there). And exploiting this to create additional colors can obviously be done, but again makes sense only in the context of PAL.
As this modification does not try to generate a PAL signal at all I did not bother emulating this color merge effect. For the most times having a clear and pixel-perfect color output is better than emulating this strange color quirk.
Nevertheless, as all the computation is done in an FPGA and can be reprogrammed without too much difficulty (you need an USB-Blaster for that, but that can be obtained quite cheaply), I could think about implementing this. I am already working on a software-interface to fine-tune the color palette anyway, implementing such an optional color-blend feature would not be too difficult, I guess.
About the other idea with shorter pixels depending on color: Again this could be some image-filtering option. But I can not change the size of the output pixels, because they are generated in a strict pixel clock. Instead I could modify the luminance to make such white pixels look weaker. Again this would be some option, the user needs to turn on specifically.

@c0pperdragon - What a really nice in-depth technical answer and openness to look into perhaps catering to this great question. Kudos to both you and @Hojo-Norem on trying to make this cool solution EVEN better.

I've been thinking about this over the last couple of days.  Putting the black transition pixel size thing to the side I think that a decent simulation of PAL colour blending can be done if the FPGA has the resources to do it.  If it has the space to store the last raw scanline from the VIC-II side then all you would do is compare the current pixel being drawn with the one directly above and output based on the comparison.
I guess that you have some kind of look-up table to determine what colour to output for each pixel, right?  I'm also guessing that it's only 16 entries large, one for each of the VIC-II's native palette.  I'm no FPGA designer, but if I were doing this in pure software and wanted to do it the lazy way, I would just have a 16x16 table (or 14x14 if you skip black and white) representing every possible combination of colours.  If I'm interpreting your schematic properly, then the output colour resolution is 16bpp.  Working with a 16x16 table, that would mean that the lookup table would be 512 bytes.  If I'm looking at the correct information about the FPGA you are using then there may be embedded user FLASH memory that could be used for this.  But again I don't really know FPGAs... I could be barking up the wrong tree.

The FPGA has enough RAM to store several lines of output data. I am using this for the line-doubling already. So for every pixel I can easily access the pixel in the previous line.
Internally everything is computed in YPbPr values - there is just one mapping from the 16 C64 colors to these YPbPr coordinates.  This color space is convenient to compute the scanline darkening effect.
More or less the same thing can be done to combine colors without touching luma in this case. The resolution of each color channel is 5 bit, giving a total of 1024 possible color values.  A lookup table would not be convenient here, but computation is no big deal in an FPGA, especially if the operation clocks at a mere 16MHz. I guess by just averaging the Pb and Pr values we get the desired results already.
General issue on palette reprogramming: I will try to provide an API via writes into unused VIC registers. The downside of this is that it will be too powerful and could disqualify such machines from demo competitions. I imagine blanking the whole screen for 1 second or so after the palette was modified, so the feature can not be abused to magic more colors to the screen as were possible on the original machine.

After quite some tinkering with the firemware, I could actually free up some space in the logic fabric to start adding more functionality. As I found, I have some on-chip RAM to spare and finally figured out how to utilize the on-chip flash to store configuration data (not finished yet).
My plans now go more or less exactly into the direction of your proposal: To support a 16x16 large matrix of palette entries. Each entry will hold a 15 bit value for the Y-Pb-Pr signal. that can be individually  modified by some writes into the VICs register space.
As you said, this can be used to set up any palette you like with or without color blending effect. Also setting up RGB colors is possible, as the Y Pb Pr outputs can be adjusted totally independently.
RGsB is easy then, but RGBS would also be possible by disabling the sync signal on the G line altogether, and using the existing luma from the A/V connector instead.

Wow, that sounds super!  The more I've been thinking about it, the more I've been wanting to build one.  The FPGA's ground pad might be a challenge for me though...  That said, I have noticed one or two PCBs over on OSH Park get around this by putting a largish plated hole connected to GND directly underneath the IC's ground pad so it could be soldered from the bottom.  Not ideal if there isn't the room to route traces around it.  (I haven't looked at the PCB layout, so I'm not certain if that is an issue or not.)  It it is possible then it would reduce the DIY barrier down too just needing a decent soldering iron and a steady hand.
On the subject of possible palette reprogramming 'abuse', I'm not entirely sure that its too big of an issue.  As big scene gatherings I would assume that there would be a dedicated compo machine set up for showing productions.  I can think of a couple of ways that things could be kept 'honest' so to speak, but as your project is open source, there's nothing stopping somebody building a version of the firmware with such a thing disabled, that is if there's room in the FPGA to implement it at all.
If you're wondering what they are, both methods would have the palette registers locked on reset until either one:  A unlock sequence is written, causing the interface to override the screen for a few seconds with a pre programmed bitmap that'll give the game away or perhaps display a small watermark in one of the corners until the next reset...
Or two (possibly simpler to implement):  Have a CRC generator on the FPGA that generates a CRC value from the screen registers and pixel data, perhaps only count a few dozen scanlines in the center of the screen to allow a bit of freedom on the screen.  Take a regular C64 bitmap image showing whatever message and generate a CRC from it, use that as the register unlock key.  Now, to unlock the registers the programmer would have to display the bitmap to allow the CRC generator in the FPGA to generate the unlock key.  The upshot of this method is that the memory requirements are shifted over to the software and that the unlock image could be anything /you/ want...
Both are a little involved and probably not work the trouble to implement as like I said somebody with the right skills could easily cook up a fully unlocked firmware to use on their demo machine.  Then again, I'm not too sure that it would be that effective to cheat by reprogramming the palette on the fly.  Completely new colours would be spotted a mile away and called out and I think that reprogramming the palette on the fly in time critical situations would be difficult, seeing that each palette entry is 3 writes long and probably the register access is going to be register, palette addr and then data then that's another 3 writes.

My design actually has exactly such a hole under the IC, so I can hand-solder it. This works perfectly every time: After the pins are finished, turn up the iron a bit and feed solder into the hole. Even with a hotter iron it takes some time until the solder melts after the initial heat was sucked into the ground plane. Heat it up until you can stir the solder in the hole like soup in a pot.
For unlocking the setup mode, I imagine to write some magic number into a register and then toggle the mode switch at the back of the computer. As the switch is quite small and hard to reach, this should not go unnoticed.

That's great on both points.  I'm definitely going to have to build one now...

Following our discussion, I have created a new firmware version (2.0) that allows modification of the palette.  You can have a look at the project main page to see how it is done. Maybe you will indeed build a board for yourself...

I'm just about to start getting the parts together.  One quick question though, I'm pretty sure that the files I need to send off to the PCB producer (I'll most likely use OSH Park) are the 'Gerber.zip' found in the rfreplacement folder in this project for the main board and for the VIC-II adapter the gerbers are in the zipfile found in "A-VideoBoard/c64mod/c64board/"?

Yes, you are right with both  assumptions. Do not forget to order the VIC-II adapter with only 0.8 mm thickness.

If it helps anyone else, I've shared the OSH Park projects for the two boards so it's easier for anyone else to order.

https://oshpark.com/shared_projects/6aZcVYWO (Modulator replacement)
https://oshpark.com/shared_projects/ma5ec3tv (VIC-II adapter)


Feature/travis ci

This PR adds Travis CI into the PR workflow.

:tada: This PR is included in version 1.0.0 :tada:
Your semantic-release bot :package::rocket:


Use of retrievecontentpack on post install script of OS X installer

Update: Fri, 4 Mar 2016
This PR now does the following:

fixes #358
fixes #370
addresses #349

Hi @aronasorman - still a WIP which eventually fixes #358
(text copied from issue filed #358)
I modified the OS X installer's post install script to replace the kalite manage init_content_items --overwrite with kalite retrievecontentpack local en path-to-en.zip -- then built the .pkg, installed on a VM, and noticed that the ~/.kalite/content_databases/content_khan_en.sqlite on a target VM still has zero bytes.
I replaced it with the content.db from Pantry, it's about 12MB+, restarted kalite, but the sidebar didn't display anything.
What works
When I ran kalite manage retrievecontentpack local en path-to-en.zip in the target VM - it works!  The content_khan_en.sqlite now has 81MB+ in size.
Running the same .pkg on the target VM twice works too and it also generates the same size.
Questions

Any idea why the diff in size between the local .sqlite generated with retrievecontentpack versus the one in Pantry's is so big?
There is no tree of videos to download at Manage - Videos tab.

Setup of build:

used ka-lite/0.16.x branch
used installers/develop branch
build script now includes the contentpacks/en.zip along with the assessment.zip when building the .pkg

On the side, the KA-Lite.pkg is now 691MB+ (with contentpacks/en.zip and assessment.zip) and will require at least 1.3GB+ when installed on a target Mac OS X machine.

Thanks @aronasorman - the updated content.db you uploaded now works.  So I used it and comment-out the use of the retrievecontentpacks.  The KA-Lite.pkg is now lower at 405MB+ and requires 817MB+ to install.
However, I have these issues when I replaced the .sqlite with the new content.db and navigated KA-Lite as installed in a VM:

On the exercise page

[Error] SyntaxError: JSON Parse error: Unexpected identifier "undefined"
parse (bundle_common.js, line 55)
get_item_data (bundle_common.js, line 55)
...
n (bundle_common.js, line 824)
(anonymous function) (bundle_common.js, line 824)

On the Video download page, the video tree doesn't load and here's the only Console error:

[Error] Failed to load resource: the server responded with a status of 404 (Not Found) (loading.gif, line 0)

And on the sidebar, I can't see the unavailable items like video, I'd think that is a feature? :smile_cat:

Any ideas where I should look?

On the side, the KA-Lite.pkg is now 691MB+ (with contentpacks/en.zip and assessment.zip) and will require at least 1.3GB+ when installed on a target Mac OS X machine.

Use either, not both. So if you're using retrievecontentpack, then you can drop the assessment zip.

On the exercise page

Is this the 0.16.x branch? Once this is merged: https://github.com/learningequality/ka-lite/pull/4827
Hopefully it can fix that error.

Yup it's on the ka-lite/0.16.x branch.
For now, I will use the content.db from Pantry and keep the assessment.zip.

Tested the latest ka-lite/0.16.x and now the exercise page is working for Mavericks, Yosemite, and El Capitan, yey!
Setup of build:

used ka-lite/0.16.x branch
used installers/develop branch
build script only uses the content.db/en.zip and the assessment.zip when building the .pkg

From initial tests, only the video downloads page issue remains with this Console error:

[Error] Failed to load resource: the server responded with a status of 404 (Not Found) (http://<IP_ADDRESS>:8008/node_modules/jquery.fancytree/dist/skin-lion/loading.gif, line 0)

Hi @aronasorman - I removed the assessment.zip and this now only uses contentpacks/en.zip with the retrievecontentpack command.
Setup of build:

used ka-lite/0.16.x branch
this branch based on installers/develop branch
build script only uses the contentpacks/en.zip when building the .pkg

From initial tests, only the video downloads page issue remains with this Console error:

[Error] Failed to load resource: the server responded with a status of 404 (Not Found) (http://<IP_ADDRESS>:8008/node_modules/jquery.fancytree/dist/skin-lion/loading.gif, line 0)

Any idea why the videos tree is not loading?

Updates:

the video download page now works
we now put all "shared" files on the /Applications/KA-Lite/support/ folder
updated the OS X installer documentation (readme, release notes, license) and now uses only "Learning Equality" for copyright.

ToDo:

The images on the exercise page are missing (see screenshot).

/cc: @aronasorman

Hi @radinamatic - the last commit fixed the missing images as per my tests.
Can you verify that it indeed fixed it based on http://dungeon.learningequality.org:8085/browse/KL-MT16-67/artifact then I can merge this PR.

Latest commit also fixes #370
@radinamatic If successful, please use the Bamboo build at http://dungeon.learningequality.org:8085/browse/KL-MT16-68KL-MT16-68 for your tests.

Hi @radinamatic - The build 71 on Bamboo should contain the fixes for #370 - specifically the missing notifications.

:+1: @cpauya, downloading!

@cpauya, I tested build 74 on El Capitan, and I couldn't see the "unreliable" notifications anymore, but that version of OSX is one-big-PITA of a resource hogger, and screenshare actions lag behind my mouse clicks so much that I can't really assess if the display of the notifications is correct.
Couldn't test at all on Mavericks, see all the details in #4964.

@radinamatic Yup, using El Capitan on a VM sucks - I gave it 2GB of RAM and it wants some more.
Fixes on this PR is already piling-up, and testing them will be harder.
I think we should merge this now so I can make a build that will include the other fixes like https://github.com/learningequality/ka-lite/issues/4961#issuecomment-192047315.  Is that okay?

That sounds like a plan for me, but I'd also prefer if @aronasorman gave the "go/no go" order for the merge... :wink:

That's my preference too, I don't want to merge my own PR as much as I want, unless there's no one else to do it.
Any comments @aronasorman? :smile_cat:

Go merge!
On Mar 3, 2016 5:41 PM, "Cyril Pauya"<EMAIL_ADDRESS>wrote:

That's my preference too, I don't want to merge my own PR as much as I
want, unless there's no one else to do it.
Any comments @aronasorman https://github.com/aronasorman? [image:
:smile_cat:]
—
Reply to this email directly or view it on GitHub
https://github.com/learningequality/installers/pull/359#issuecomment-192050932
.