[Attestation Service] Change the API of CoCo-AS

This PR will refactor the CoCo-AS furthermore, which will help to let CoCo-AS run as a seperate attestation-service without any coupling with KBS.
The API of CoCo AS will be definitely not related to KBS protocol. In this PR, CoCo-AS gRPC service will have the following API
message AttestationRequest {
    Tee tee = 1;
    string evidence = 2;
    repeated string runtime_data = 3;   // plaintext of runtime-data
    repeated string init_data = 4;           // plaintext of init-data
    string hash_algorithm = 5;                // hash-algorithm of the
    repeated string policy_ids = 6;         // specified policy-ids that is used to check against
    string customized_claims = 7;          // extra claims that will be injected into the result token
}

Close #177
Related to #217
cc @jialez0

message AttestationRequest {

we may also need nonce?

message AttestationRequest {

we may also need nonce?

Nonce is something that will be bound to the report data. This API now gives a general parameter rubtime

message AttestationRequest {

we may also need nonce?

Nonce is something that will be bound to the report data. This API now gives a general parameter runtime_data to check the binding between any report data and the evidence.

This API now gives a general parameter runtime_data to check the binding between any report data and the evidence.\n\nAFAIUI, the AS cannot generate "expected report data" based on runtime_data (Tee pubkey) only if it does not know the nonce the attester used to generate the quote/report data.

You are right.
This api is now for user who knows the materials to calculate the report data.
We can add an extra API, which will give a nonce to the caller to get evidence. Then the user can call this attestation API.

I'm still reviewing, but the attestation requester should not be passing policy_id to evaluate against, that is insecure. The policy in RATS is supplied by the verifier owner, which is different then the attester.

@jepio policy_id in the api would help the caller to apply different use cases.
The key point is that the returned attestation token will have information about the policy ids that are used to check against the evidence. Current AS only has a fixed policy and cannot choose a specific one from API.
Doea this make sense?

@jepio policy_id in the api would help the caller to apply different use cases.
The key point is that the returned attestation token will have information about the policy ids that are used to check against the evidence. Current AS only has a fixed policy and cannot choose a specific one from API.
Doea this make sense?

I see. I now see that ITA also allows the user the option to pass the policy in a similar way. Why don't we do it same way as well:

policy id parameter is optional and then default policy is applied
policy id is hash/uuid (i think hashes are a better option as they are unique and unmodifiable), not name

@jepio policy_id in the api would help the caller to apply different use cases.
The key point is that the returned attestation token will have information about the policy ids that are used to check against the evidence. Current AS only has a fixed policy and cannot choose a specific one from API.
Doea this make sense?

I see. I now see that ITA also allows the user the option to pass the policy in a similar way. Why don't we do it same way as well:

policy id parameter is optional and then default policy is applied
policy id is hash/uuid (i think hashes are a better option as they are unique and unmodifiable), not name

This is a great point. I think this is related to how policies are indexed. Currently we only have OPA as policy engine and the regos are stored in filesystem so we directly retrieve them with the file name.
In this PR, I added the policy digest into the result JWT as you suggested. However how to specify which policy should be used might be another topic, as it is related to how it is stored. We can promote this in another PR. wdyt?

It's not secure to mix claims attested through runtime data (like tee-pubkey, nonce) with "custom claims" which are unattested and user controlled. So we really need to impose some structure on runtime data (like a json dictionary) that can be included in the attested token separately from any user supplied data.

It is a good idea that we should have a runtime data format/spec like undergoing initdata. This could help us to leverage report_data field in a more standard way.
But for custormized claims, my idea is that this would give ability to the user to embed some extra context information. Whether to use such claims is determined by the consumer of the JWT. In CoCo-AS/KBS model, the attestation requester and JWT consumer both are KBS, so it is secure for KBS itself to leverage the pub-key fields inside the claims.

Custom claims should instead be a "nonce" supplied by the user to be included in the attestation token.

From the perspective of opposing server replay attack, this is worth doing.

@jepio policy_id in the api would help the caller to apply different use cases.
The key point is that the returned attestation token will have information about the policy ids that are used to check against the evidence. Current AS only has a fixed policy and cannot choose a specific one from API.
Doea this make sense?

I see. I now see that ITA also allows the user the option to pass the policy in a similar way. Why don't we do it same way as well:

policy id parameter is optional and then default policy is applied
policy id is hash/uuid (i think hashes are a better option as they are unique and unmodifiable), not name

This is a great point. I think this is related to how policies are indexed. Currently we only have OPA as policy engine and the regos are stored in filesystem so we directly retrieve them with the file name.
In this PR, I added the policy digest into the result JWT as you suggested to ensure the integrity of policy recorded in the JWT. However how to specify which policy should be used might be another topic, as it is related to how it is stored. We can promote this in another PR. wdyt?

I think we need to address this before. We can't change the token format every week because without stability we won't see adoption. We should not be leaking the fact that policies are stored in filesystem into our token API - in a cloud deployment of CoCo AS the backend storage will be blob storage. So I think we need policy management APIs like ITA (https://docs.trustauthority.intel.com/main/restapi/restapi-policy-management.html). When a user registers a policy they need to either pass a UUID or the API returns a UUID. And then this UUID would be passed with the attestation request. WDYT?

It's not secure to mix claims attested through runtime data (like tee-pubkey, nonce) with "custom claims" which are unattested and user controlled. So we really need to impose some structure on runtime data (like a json dictionary) that can be included in the attested token separately from any user supplied data.

It is a good idea that we should have a runtime data format/spec like undergoing initdata. This could help us to leverage report_data field in a more standard way. A json map would help both protect the integrity and embed into extra claims.
But for custormized claims, my idea is that this would give ability to the user to embed some extra context information without integrity protection. Whether to use such claims is determined by the consumer of the JWT. In CoCo-AS/KBS model, the attestation requester and JWT consumer both are KBS, so it is secure for KBS itself to leverage the pub-key fields inside the claims.

No, it's not secure. Anyone can request a CoCo-AS token  passing an arbitrary public key in "custom claims", also one not stored in a TEE and then pass that token to CoCo KBS. So "runtime-claims" (attested through a REPORTDATA mechanism) need to be separated from arbitrary "custom claims" in the token. Anything else can't be trusted by consumers.

we can specify what it needs to look like to be valid. and besides: the code already parses the OPA output and checks that it matches this structure.

From: Xynnn_ @.>
Sent: Wednesday, December 6, 2023 10:10
To: confidential-containers/kbs @.>
Cc: Jeremi Piotrowski @.>; Mention @.>
Subject: Re: [confidential-containers/kbs] [Attestation Service] Change the API of CoCo-AS (PR #240)
@Xynnn007 commented on this pull request.

In attestation-service/bin/grpc-as/example.token.jsonhttps://github.com/confidential-containers/kbs/pull/240#discussion_r1416963055:

@@ -0,0 +1,47 @@
+{

"customized_claims": {

   "test_key": "test_value"

},
"evaluation-reports": [

   {

       "evaluation-result": "{\"allow\":true}",

This is the raw string output by OPA when using the default_policy.rego. I am afraid that we cannot assume what the result will look like. It is decided by the rego.
—
Reply to this email directly, view it on GitHubhttps://github.com/confidential-containers/kbs/pull/240#discussion_r1416963055, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABXINVSYEOFFFIYZQYS3TBTYIAZARAVCNFSM6AAAAAA7Y4ECW2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTONRXGA2DOMRZGE.
You are receiving this because you were mentioned.Message ID: @.***>

@jepio

about policy management. I agree with your ideas. I think we can implement them in another PR rather than this one. Currently (name, hash) pair is equal to (uuid, hash) pair, so after merging this PR we can still impl the policy part without changing the attestation api. In the token, it is always called policy-id.
Custormized claims. Makes sense. It requires that we should check the integrity of runtime data with the one in the evidence.
Let me added a separate commit to check whether protoc in GH runner supports optional in proto3.

@jepio Updated the PR. Added a separate commit where I use oneof key word to support both raw init/runtime data match and structured init/runtime data match. PTAL if it is good.

How does reference data fit into this? It looks like reference values aren't set via the attestation request or reported in the attestation results. The same policy could have very different results with different reference values.

Reference data seems a design challlenge. Reasons

We currently tightly depend on the OPA policy to check the reference data. That is to say, if the policy does not do the check, the reference data actually do not work.
@thomas-fossati once raised a good point that compound reference values are needed but still not supported by the corrent RVPS.

But overall, I agree with you that attestation token should have the reference data information inside. So let just temporarily ignore how 1 & 2 are handled, and just try to fix the token format, s.t. another field containing reference data information that was used when doing verification. wdyt?

Rebased and update the PR with the new API.
@jepio @fitzthum @jialez0 a decision left: whether the token can be provisioned without passing all OPA policies
@mythi Added the KBS side check for https://github.com/confidential-containers/guest-components/issues/366 here I will put a PR on guest-components side then.

Container build CI is not related to this PR. Will be resolved in a separate PR.


prepare code for testing

...in case participants do not bring their own, and maybe it is also better so that everyone is on the same page with the code. We would have code in two different languages (R and python) and 2-3 levels of experience:

one very basic where it is a script and you got to organize it first then test maybe one functionality

one bit more complex where there are already functions, and you need to create example data for functions (eg working on a csv file that you can mock)

and one advanced where a bit more involved set of functions (class) needs to be tested: I am thinking both unit test and integration test with different edge cases.

In all cases they would need to modify the original code too, we can add bugs in them or missing catches for unexpected values (eg division by 0)

R is to be finalized
python is in progress


Implement non blocking mutate functions

Definition of the problem
Offline requests block promises.
This will mean that code in then will never be called until the device becomes online.
return this.apollo.mutate<Task>().then((retry)=>{
	// Never called when offline.
	// This will be resolved only when online
}).catch((error)=>{
	// Never called when offline
	// This will be resolved only when online
});

Proposal 1
Return error when change is enqueued offline and resubmit every change using client.
return this.apollo.mutate<Task>().then((retry)=>{
   // Never called when offline.
   // This will be resolved only when online
}).catch((error)=>{
  if(error.conflict){
    // User will get error with conflict
  }
});

Proposal 2
Return another observable or promise in error
return this.apollo.mutate<Task>().then((retry)=>{
   // Never called when offline.
   // This will be resolved only when online
}).catch((error)=>{
  if(error.offlinePromise){
    offlinePromise.then((data)=>{
       // Will be called when we become back online
    });
  }
});

done in #10


callerid: Inject newContext() so it can be wrapped or replaced.

Exporting from internal change.

LGTM


Software page

Update the software page.

This looks like good content! Is it possible to add these as tiles in a grid to make the page a little denser?

I've changed to a grid layout. Will we eventually have a logo for every software or should we have a generic software icon for those without a logo?

Cool! I like the look of it as a grid. It would be nice if we had logos or representative images related to each, but if not, maybe we can make a placeholder graphic that looks like binary or code or something for ones we're missing

Yeah, I like that. How about either docs or [docs] to keep it short?

I made this icon in illustrator. Kind of a mix between the Mac terminal and traffic light buttons. It felt more fitting than beakers!

I like the logo! As we discussed, try to find a color scheme that looks good in both light and dark modes.


Is there any performance difference between submitting all primitives at once into the stream and submitting each primitive individually into the stream ?

I've created Alexnet network using two approaches.

For each layer,  i've created its corresponding primitive and submitted into the stream for execution. (i.e., after creating convolution primitive, it is submitted into stream and then ReLU primitive is submitted into the stream and so on)

For each layer,  i've created its corresponding primitive and push back to the vector(std::vector<mkldnn::primitive> net). In this way primitives of all layers are pushed into the vector(net). Finally this vector is submitted at once to the stream for execution.

Can i expect any performance improvement in second approach?

There's no difference in performance between these approaches currently. The stream was introduced to accomodate asynchronous execution necessary for programming some devices.

thanks for the information @vpirogov


Support global install

This should work:
npm install -g cloverfield
cf

But here's what happens:
npm install -g cloverfield
cf
-bash: cf: command not found

Want to fix this, @nkbt?

I just found that the npm command doesn't install all the cloverfield's files and dependencies, this is what I have in my global lib/node_modules directory:
 9 21:08 ..
 4  2015 LICENSE
 4  2015 README.md
 9 21:08 package.json

Maybe this is the reason why the npm command is not creating the link inside the bin directory.

I found that when I ran:
npm install -g cloverfield

No files were downloaded, but I did get the symbolic link output like normal.
However, if I run:
npm install -g cloverfield-tools/cloverfield

Cloverfield is installed properly, and both cf and cloverfield are executable. However, I get error messages from the index.js file. I am using nvm, so I do not know if that is an issue.

We have most of things ignored before publishing to npm. Also I don't think we published to npm recently (will check it).
I am going to fix this today later.

As I expected it is simply not published to npm. I created #20 with .npmignore and #19 to update to the most recent dependencies. After thy are in we can publish new version to the npm.

Thanks @nkbt
It's a good start, but we need to build the bin file that gets symlinked with Babel for Node support:
$ npm install -g cloverfield

><EMAIL_ADDRESS>install /Users/eric/.nvm/versions/node/v0.12.5/lib/node_modules/cloverfield/node_modules/babel/node_modules/chokidar/node_modules/fsevents
> node-gyp rebuild

  SOLINK_MODULE(target) Release/.node
  CXX(target) Release/obj.target/fse/fsevents.o
  SOLINK_MODULE(target) Release/fse.node
/Users/eric/.nvm/versions/node/v0.12.5/bin/cloverfield -> /Users/eric/.nvm/versions/node/v0.12.5/lib/node_modules/cloverfield/index.js
/Users/eric/.nvm/versions/node/v0.12.5/bin/cf -> /Users/eric/.nvm/versions/node/v0.12.5/lib/node_modules/cloverfield/index.js
<EMAIL_ADDRESS>/Users/eric/.nvm/versions/node/v0.12.5/lib/node_modules/cloverfield
├──<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>├──<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>└──<EMAIL_ADDRESS>                              --   --   (master  origin)   v0.2.0  
Erics-MacBook-Pro:cloverfield eric$ cf
/Users/eric/.nvm/versions/node/v0.12.5/lib/node_modules/cloverfield/index.js:4
import nomnom from 'nomnom';
^^^^^^
SyntaxError: Unexpected reserved word
    at exports.runInThisContext (vm.js:73:16)
    at Module._compile (module.js:443:25)
    at Module._extensions..js (module.js:478:10)
    at Object.require.extensions.(anonymous function) [as .js] (/Users/eric/Dropbox/dev/cloverfield/node_modules/babel/node_modules/babel-core/lib/api/register/node.js:214:7)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:501:10)
    at Object.<anonymous> (/Users/eric/Dropbox/dev/cloverfield/node_modules/babel/lib/_babel-node.js:144:25)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)

Hm, I used babel-node as interpreter here so it should work. At least it
worked on the other project of mine. I'll check more. If it still won't
work, will do it on the code level

babel-node is not need it anymore if we are using ESLint ^1.0.0, we could safely use the native ESLint parser since this new version has full support for ES6

@davegomez babel-node and babel-eslint are completely different things. And I am pretty sure eslint 1.0 still does not support object spread like:
const x = {
  ...y,
  z: 123
};

+1 @nkbt Just tested eslint 1.1.0 and object spread is not supported (cf https://github.com/eslint/eslint/issues/3271).
I don't know babel-node, but should the transpilation be done in the CI part ? Thus, the published package would be ES5.

We use babel-node to avoid transpilation step before publishing. Babel-node
does it in-memory.
--
Sincerely yours, Nikita Butenko
Google Talk<EMAIL_ADDRESS>Skype: nikita.butenko
Mobile: +61 450 264 801
On 11 August 2015 at 18:21, ThomasCrvsr<EMAIL_ADDRESS>wrote:

+1 @nkbt https://github.com/nkbt Just tested eslint 1.1.0 and object
spread is not supported (cf eslint/eslint#3271
https://github.com/eslint/eslint/issues/3271).
I don't know babel-node, but should the transpilation be done in the CI
part ? Thus, the published package would be ES5.
—
Reply to this email directly or view it on GitHub
https://github.com/cloverfield-tools/cloverfield/issues/18#issuecomment-129764982
.

Published as 0.3.0


Googlebilling

Google published a library for easy usage of payments. https://developer.android.com/google/play/billing/billing_java_kotlin
I did an implementation of gdx-pay using their library. It is very few code - less code, less errors! Can be used instead of OpenIAB and android-googleplay which uses the native libs.
This branch is based on my last PR.

Thanks for providing the PRs! Currently I my agenda is busy, I will look at them on Friday.

@MrStahlfelge we are using the old gdx-pay-android-googleplay library, and consider to switch to this.
One problem we now face, is that users who try to purchase using iDEAL continuously often fail to purchase the app. Do you know if the new API works better with iDEAL?

Hi, I don't know iDEAL but it looks like a banking service to use with Google IAP instead of credit cards?
Of course I would appreciate if you use the playbilling implementation, too. But I doubt the approach using the billing lib will avoid these problems as they are probably not technically. The billing lib from Google used in playbilling is "just" a convenience wrapper around the same API googleplay implementation uses, so purchase flow is the same.


How to load javascript and css

I'm trying to access css files using /{AppName}/css/bootstrap.css but i'm getting this error message:
"Failed to load resource: the server responded with a status of 404 (ERROR)"
What exactly do i need to do to make it works like in the Positron.Application example?

And changed Build Action to "Resource"

Oh nvm, it worked now. I just had to change every item's build action to "Resource" 👍


New allocation algorithm

Replaced algorithm in class arena with one that uses mmap()'d address spaces and demand paging for each semispace and initializes the current semispace of each arena at start time to avoid and the second semispace of each arena at swap time to avoid relying on Undefined Behavior or having an extra test in kore_arena_alloc().

That is correct. I tried to simulate the old GC triggering algorithm, however note that allocations > 1 MB are now accommodated in the GC'd arena.


make isoDateFromDateInput allow months as words

Spontaneous contribution after spotting discussion in x-gov about something related and already having the code handy
How useful it is for prototypes 🤷♂ so don't feel like you have to accept it! 🙌

Love this! Merged and released in v1.4.0. Thanks for contributing!


proper Python packaging

I discovered this code thanks to Anna ! It would be great to have it available as a Python package on pypi so that one can do pip install veering.
(I can make it happen if I have a green light)

Sure - I don't see why not... the code base is actively being worked on and we are not organized enough to have numbered releases or anything like that - would there be any issue with updating the pip version of the code?

Releasing on pypi is optional. With a proper Python package setup one can install the development version with the command pip install git+https://github.com/henryseg/Veering. That would already be a step forward for having people easily play with the code.

Releasing on pypi without versioning is not an option.

Dear Vincent -
I am going to assume that Henry and Anna are fine with this, so yes, have a green light.
I suggest the version number 0.0.0 so we can use pip install.
Tell me what I need to do...
best,
saul

Ah, Henry, you need to do this - go to the Veering main page, click on "Settings"(last on horizontal list of items), click on "Access" (in sidebar), click on "Collaborators & teams", and to the right of "Manage access", click "Add people or Add teams".  Add videlec as an admin.  Or make me an admin and I will do it.

I invited Vincent - it looks like for a personal repository there is
precisely one owner and no other kind of admin role.
On Thu, Jul 21, 2022 at 2:49 PM Saul Schleimer @.***>
wrote:

Ah, Henry, you need to do this - go to the Veering main page, click on
"Settings"(last on horizontal list of items), click on "Access" (in
sidebar), click on "Collaborators & teams", and to the right of "Manage
access", click "Add people or Add teams". Add videlec as an admin. Or make
me an admin and I will do it.
—
Reply to this email directly, view it on GitHub
https://github.com/henryseg/Veering/issues/7#issuecomment-1191509890,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACOJ3GJMVNTIGJSIVT23VQDVVFIM7ANCNFSM54HOPFLQ
.
You are receiving this because you commented.Message ID:
@.***>

veering_census.txt and veering_census_with_data.txt are the most important. Perhaps we could have a secondary "More_Data" folder, which isn't included in the standard install for the other files?

Let me start with these two files.

I pushed my changes on the setup branch. If you are fine with it I can merge it in master.

Of course.
Though there is still an issue with markdown rendering. distutils setup ignores long_description_content_type while setuptools is half broken. Could I changed the README syntax from markdown to rst ? It will make the packager life easier.

I think Saul wrote the readme, I've CCed him. I imagine that he will be
fine with whatever makes things work.
On Thu, Jul 21, 2022 at 6:14 PM Vincent Delecroix @.***>
wrote:

Of course.
Though there is still an issue with markdown rendering. distutils setup
ignores long_description_content_type while setuptools is half broken.
Could I changed the README syntax from markdown to rst ? It will make the
packager life easier.
—
Reply to this email directly, view it on GitHub
https://github.com/henryseg/Veering/issues/7#issuecomment-1191739808,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACOJ3GOP27WGHCNIAOFJSKTVVGANZANCNFSM54HOPFLQ
.
You are receiving this because you commented.Message ID:
@.***>

It turns out that setuptools is very much deprecated. Markdown should be perfectly fine.

It sounds like you are saying "Saul, you don't have to do anything." which makes me very happy.

The changes in the setup branch are not backward compatible as the code uses a lot of relative paths such as https://github.com/henryseg/Veering/blob/setup/veering/boundary_triangulation_with_snappy.py#L11-L12. Also many files have main functions which is a weird thing for a Python library.
I am not sure what to do about that.

I tend to use the main() functions as working notepads for what we are trying to calculate with a file, separate from the (ideally) stable rest of each .py file. I'd rather not do away with them entirely, as it's a lot easier to remember how to get the code working with examples of what we were doing in the main(). I could have entirely separate files for working, although that would increase the number of files floating around.
I suspect that Saul's style of using the sage command line to calculate with is probably more standard, and doesn't leave main() functions in the code. The downside with this as I see it is that the code he writes as he is calculating is often not saved, and we spend time rewriting it. Are there alternate ways to work that give the best of both worlds?

I understand why these functions main are there and that they might be useful in a development stage. Though, as part of a distributed library, I tend to believe that they create more confusion than help. If it is just for documentation purpose, it would make more sense to write documentation strings with specification and examples. The latter could automatically be tested with eg https://docs.python.org/3/library/doctest.html. This is how Python encourages users to write their library code.
I definitely support the idea of having code examples, notepads, code snippets, etc together with the code. A random person (including the ones who wrote the code) is more likely to copy/paste an example close from her purpose rather than reading function specifications. But I tend to separate such worked out examples in a disjoint folder such as scripts/ or examples/, or docs/. Such folder has no reason to be installed with the library.

Ok, we will move the main() functions into separate documents named *_working.py, or *_examples.py

I've started a "Scripts" directory, moved all of the "example code" to to it, and pushed.

detail : Python tends to use snake case for files and folder (ie Veering -> veering, Scripts -> scripts)

I did push a change which prepares for version 0.0

I put all .py files that were tested in test_suite (+ dependencies) inside the veering/ folder that should be considered as the "veering library". All other files are in scripts. I believe that you want more files in veering. Simply tell me which one and what test should be added to the test suite.
For testing, I strongly suggest to switch to pytest which is very popular these days
The README file is adapted to the new setup

I've used the new instructions in the README to install veering in sage.
I noticed that the module veering has no methods, which I thought was strange?
Anyway, I guessed that
from veering import test_suite
would work, and it does.
The test_suite completes correctly.
I also did a pass through the scripts.
I think that none of them are currently ready for prime time.

great that it works !
Indeed, one should import all relevant functions in the veering namespace. You need to choose what are the relevant functions and include them in the __init__.py as
from .module1 import functionX, objectY

Making the scripts work is I think the priority. For most of them, it is just a matter of changing the import statements.
Also, I believe that you would like some of the current scripts to be part of the veering library.
Switching to pytest is tedious but straightforward. I can do it.

Ok, I am confused.  Writing
from .module1 import functionX, objectY
suggests that we are going to allow functions of submodules to be functions of veering.  That feels like it violates the name space separation that we've evolved towards.  I was asking how the user is supposed to know the submodules of veering?

Ok, I am confused. Writing
"from .module1 import functionX, objectY"
suggests that we are going to allow functions of submodules to be functions of veering.

Indeed.

"You need to choose what are the relevant functions"
That feels like it violates the name space separation that we've evolved towards. I was asking how the user is supposed to know the submodules of veering?

The submodules are usually about code organization. To my mind, it makes sense to have the main functions being available in the main namespace (here veering). Just take the very standard numpy
>>> import numpy
>>> numpy.array([0, 1, 2, 3])
array([0, 1, 2, 3])

If you dislike such approach, you can consider having the list of submodules in the (not yet existing) documentation.
One can also discover submodules via introspection but it is not very user friendly.

Regarding scripts:
"For most of them, it is just a matter of changing the import statements."
I have fixed the import statements for all scripts, and checked that my scripts work. The scripts written by Henry or Anna will have to be checked by them (or by me, spending significantly more time...).  I suggest that we ignore these for the moment.
"Also, I believe that you would like some of the current scripts to be part of the veering library."
There are a few that might be promoted, later.  I suggest that we ignore these for the moment.
"Switching to pytest is tedious but straightforward. I can do it."
I have installed pytest, installed veering (both in sage and not in sage), fixed the tests in the test directory, and successfully run pytest at the command line in that directory.  However, I do not know how to run pytest under sage... approximately 55% of the test suite needs sage.

Oh, and I've checked that the test_suite still runs (even after all the changes!).  Whew.

"Switching to pytest is tedious but straightforward. I can do it."

I have installed pytest, installed veering (both in sage and not in sage), fixed the tests in the test directory, and successfully run pytest at the command line in that directory. However, I do not know how to run pytest under sage... approximately 55% of the test suite needs sage.

To run pytest in the sage environment do
$ sage -sh
(sage-sh) $ pip install pytest # if not already done
(sage-sh) $ python -m pytest .

Ok.  In the sage shell:

python -m pytest .

throws an error as follows:
/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: No module named pytest
But simply typing

pytest

in sage shell, the test directory, now works.  I had to uninstall veering and reinstall it (because the version was unchanged, I assume) for everything to work.
What should I do next?  I suppose that you want all of the tests in the test suite moved and rewritten.  Are you still willing to do that, or should I?  Also, how should we deal with tests that require sage?  Currently the test_suite pokes its environment a bit, and either does or doesn't do the test depending on the result.

Ok. In the sage shell:

python -m pytest .

throws an error as follows:
/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: No module named pytest
But simply typing

pytest

in sage shell, the test directory, now works. I had to uninstall veering and reinstall it (because the version was unchanged, I assume) for everything to work.

hmm, I think this is not calling the right pytest. Could you try which pytest? I mentioned in my earlier post that you have to install pytest inside the sage environment.

What should I do next? I suppose that you want all of the tests in the test suite moved and rewritten.

I find it cleaner with pytest but this is by no mean mandatory.

Are you still willing to do that, or should I?

I can definitely assist.

Also, how should we deal with tests that require sage?

There is a pytest directive importorskip for that purpose.

Currently the test_suite pokes its environment a bit, and either does or doesn't do the test depending on the result. Having all of the tests in one place sort of makes sense to me - it is easier to use environmental flags, and it is easier to write new tests (by copying from old ones).

I believe that correctly installed pytest in sage, following your instructions.  Here is a screen-shot for which pytest:

[~]$ sage -sh
Starting subshell with Sage environment variables set.  Don't forget
to exit when you are done.  Beware:

Do not do anything with other copies of Sage on your system.
Do not use this for installing Sage packages using "sage -i" or for
running "make" at Sage's root directory.  These should be done
outside the Sage shell.

Bypassing shell configuration files...
Note: SAGE_ROOT=/Applications/SageMath

which pytest
/Applications/SageMath/local/bin/pytest

I find it cleaner with pytest but this is by no mean mandatory.

If it is not mandatory, let's skip it.  We can fix the way we do testing in the future.  Perhaps in version 1.61803398875.

Dear Vincent - Is there anything left that really needs to be done?  Can we close this issue?

This looks like ready for a first beta release. Once the issue closed, I can proceed with making the release and uploading on PyPI. Unless you want me to teach you how to do that?

Excellent.  I am closing this issue.  If you have time tomorrow at 3pm UK time, let's chat about teaching me this.


Support for Account Moving from/to Takahe

Support for account moving, e.g. from Mastodon to Takahe (or the other way around).
See Mastodon docs "Moving or leaving accounts" for the possibilities Mastodon offers in this regard.
Priority (for me) would be moving from Mastodon to Takahe.

Yup, this is on our roadmap for approaching 1.0, though some initial work has already started.

I see it's not possible to export posts from Mastodon. Has anyone checked how difficult it could be to import posts on a Takahe instance? I am running a Mastodon instance, but I would rather run a Takahe one, since I work with Python. I still haven't found time to test Takahe tho.

I'd be interested in a database import for Akkoma, as well, but am more concerned with triggering other instances to migrate followers.

Would Takahe -> Takahe migration be implemented first? Or is that already possible?

It would be at the same time - I'd want to implement the account moving system that's already out there, so it would just let you move to/from anything else that supports it.

I'm also wondering the other way around: How could a process look like to migrate from Takahe to any other Fediverse software, let's say GoToSocial?

It's the same system - one of the reasons I have not done another release yet is that I've implemented inbound migration, but not yet implemented outbound, and it feels bad to have one and not the other.

Regarding exporting posts from mastodon, the Export Archive feature certainly includes them... https://docs.joinmastodon.org/user/moving/#export - very little tooling actually supports importing this archive though, not even mastodon itself. Supporting backfill-import of the archive would be very cool, though perhaps not within the same issue as the online account migration.


Cloud v3 pg_dump format changed

@MarkvanMents please ask about dates here if necessary, as this is for a release over a week ago.


refactor: export delete-permission.sh script

This PR exports the delete-permission.sh script

Closing since changes were merged with another branch


Do tiles have to be images?

If you have many large detailed geojson files, how do you load them in a tiled way depending on zoom level and position?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 669085c9-3219-fb26-c6b3-b35ec1800feb
Version Independent ID: 671a5d20-61e8-ced7-bdca-17c2f17f99f1
Content: Add a tile layer to a map
Content Source: articles/azure-maps/map-add-tile-layer.md
Service: azure-maps
GitHub Login: @rbrundritt
Microsoft Alias: richbrun

The approach in that case is to convert your GeoJSON files into a vector tile layer. Azure Maps supports the open standard for vector tiles defined by Mapbox. https://github.com/mapbox/vector-tile-spec There are many tools available for converting GeoJSON files into vector tiles: https://github.com/mapbox/awesome-vector-tiles
From there, simply pull your data in using the atlas.source.VectorTileSource class and use this as a data source within the Azure Maps SDK.

@onionhammer Thank you for your query.
Thank you @rbrundritt for helping on this.

@rbrundritt thanks, sounds like a good opportunity to add an azure map specific doc page for this scenario

There are plans for one, but we are waiting on a new major feature that will make this whole process a lot easier than it is today.

@rbrundritt Thanks again for your help
Are you able to add a bit more info on how to actually use VectorTileSource here? Since there isnt really any usage documentation on it, I"m struggling to actually utilize the manifest and myriad pbf files I've created on an azure map

Its pretty straight forward, once you create an instance of a VectorTileSource you can use it with any layer that can take in a "Source" nearly the same way you would use a DataSource. The main difference is you would specify the sourceLayer from the vector tile you want to render. Vector tiles allow you to have multiple data layers, called sourceLayer. Here is a code block:
var source = new atlas.source.VectorTileSource(null, {
	url: '<URL to the TileJSON endpoint for your vector tiles.>'	//A TileJSON file specifies the details of the vector tile service.
});

/*
//Alternatively, if you don't have a TileJSON you can create specify the details.
var source = new atlas.source.VectorTileSource(null, {
	tiles: ['<formatted URL>'],	//An array of one or more formatted tile URLs. Multiple URL's used if you have multiple endpoints (i.e. subdomains) for your service for performance.
	tileSize: 512,	//The diminsions of the tiles.
	minZoom: 0,	//The min zoom level tiles are available.
	maxZoom: 22,	//The max zoom level tiles are available.
	bounds: [-180, -85, 180, 85]	//The bounding box for where tiles are available.
});
*/

//Create a line layer and connect it to the source. Add the layer to the map.
map.layers.add(new atlas.layer.LineLayer(source, null, {
	sourceLayer: '<name of the data layer in the vector tile you want to render>'
}));

Here are some related docs:

https://docs.microsoft.com/en-us/javascript/api/azure-maps-control/atlas.source.vectortilesource?view=azure-maps-typescript-latest
https://docs.microsoft.com/en-us/javascript/api/azure-maps-control/atlas.vectortilesourceoptions?view=azure-maps-typescript-latest

AirMap usings Azure Maps and vector tiles. Here is their source code: https://github.com/airmap/js-azure-maps-plugin

@onionhammer We hope above response will provide more details and examples. Do let us know if you have further queries.
Since we have not heard back from you we will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly reopen the issue and continue the discussion.


4.x: Amends datasource-ucp to depend on ucp11 instead of ucp

This PR changes integrations/cdi/datasource-ucp/pom.xml to depend on ucp11 instead of ucp, since the pom.xml was previously amended (via https://github.com/helidon-io/helidon/commit/ef6649f236ecd3d35ec96a138c0735ad7a54856f) to include a dependency on our ojdbc integration rather than on the Oracle JDBC driver directly—a decision which had some unintended ripple effects.
Part of this change seems to have been to include the full production suite of artifacts related to the driver. Notably, the suite of artifacts also includes ucp11, along with several other jars related to observability and so on.
Our ojdbc integration excludes ucp from its dependency tree. But that dependency is never included (despite the Developers Guide For Oracle JDBC on Maven Central claiming (incorrectly) that it is (see "Use Case 1: I Want the Production Jars", second bullet point, and contrast with what the relevant pom.xml actually says)), so the exclusion is effectively a no-op. I'll fix this in a subsequent PR (though I'm not the original author, it seems simple enough).
Anyway, the net effect of all this, as #9349 indicates, is that a user of this project will currently end up with two copies of UCP on the classpath: one compiled with JDK 8 (ucp) dragged in by integrations/cdi/datasource-ucp, and another compiled with JDK 11 (ucp11) (dragged in by integrations/db/ojdbc).  The intent of this PR is to fix this problem and remove at least some of this confusion.
Closes #9349.

Partial mystery solved: the Oracle JDBC ojdbc11 artifact, at version <IP_ADDRESS>, has the following in its native-image.properties file it ships with:
  -H:EnableURLProtocols=http \
  -H:EnableURLProtocols=https

These lines are not present in the native-image.properties file shipped with ojdbc11 version <IP_ADDRESS>.10 (nor, I would think, should they be). So that explains that issue.

Closing; incorporated in #9441


Allow matrix to determine whether a given square is outside the data

This would speed up the building step of the k2-raster tree, since it would then be able to determine that certain recursive calls are unnecessary and save time on them.


Rework asynchronous tests documentation

🚀 Feature Proposal
https://jestjs.io/docs/asynchronous
It currently says "The most common asynchronous pattern is callbacks." This is wrong. We should put promises at the top, and callbacks last. We should probably lift up async-await as well, before the example returning promises.

If you want to work on this, please state so in this issue so not multiple people start working on it, creating redundant work 🙂

Please assign me this issue.

Thanks @adi611!

Hi @SimenB async-await mentions .resolves or .rejects. So should we lift it before the example returning promises? Because then .resolves and .rejects will come after async-await. Shouldn't we keep it after .resolves and .rejects?

I think we should just have a plain async-await section first, then we can bring in resolves and rejects afterwards

hey @adi611 , need some help, could you share how much you have been able to do it?

hey @adi611 , need some help, could you share how much you have been able to do it?

Hi do I need to change the CHANGELOG.md file?

nope doc changes are usually not added to the changelog , unless there is a change that needs to be notified, here we are just making a few back and forth changes

🚀 Feature Proposal
https://jestjs.io/docs/asynchronous
It currently says "The most common asynchronous pattern is callbacks." This is wrong. We should put promises at the top, and callbacks last. We should probably lift up async-await as well, before the example returning promises.
If you want to work on this, please state so in this issue so not multiple people start working on it, creating redundant work 🙂

Hi I have worked on this in #12555


Swift Potentially Uses the Wrong Swift Driver

I recently opened an issue on the Swift repo (https://github.com/apple/swift/issues/65461) but realised after some debugging that it probabl better fits here.
When trying to build an existing application with a custom toolchain based on the Swift 5.8 Release (https://github.com/apple/swift/commit/f9611bc8e745b1b9a98d494236adb6d6c739fe97) the application will not compile due to the Swift Package Manager trying to disallow the new driver.
I noticed that when executing swiftc -version the output seems to vastly differ. With Xcode's swiftc I get following output:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -version
swift-driver version: 1.75.2 Apple Swift version 5.8 (swiftlang-<IP_ADDRESS>.2 clang-14<IP_ADDRESS>.100)
Target: arm64-apple-macosx13.0

while with the custom compiler I get following output:
/Library/Developer/Toolchains/swift-LOCAL-2023-05-02-a.xctoolchain/usr/bin/swift-frontend -version
Apple Swift version 5.8 (swift-5.8-RELEASE)
Target: arm64-apple-macosx13.0

To me it looks like there are some issues with the custom compiled toolchain choosing the wrong driver. Is there any easy-fix for this?

Sorry, my bad. I put the wrong command there. Here is the corrected one:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -version
swift-driver version: 1.75.2 Apple Swift version 5.8 (swiftlang-<IP_ADDRESS>.2 clang-14<IP_ADDRESS>.100)
Target: arm64-apple-macosx13.0

Anyway, my problem still applied. I did some debugging with dtruss and noticed that the my custom toolchain does not check for swift-driver while the stock one does:
351- 1234/0x252e:     16496       1      0 sysctl([CTL_KERN, 4, 0, 0, 0, 0] (2), 0x16BC19140, 0x16BC18DA0, 0x0, 0x0)		 = 0 0
352- 1234/0x252e:     16500       4      3 sysctl([CTL_HW, 1, 0, 0, 0, 0] (2), 0x16BC19240, 0x16BC18DA0, 0x0, 0x0)		 = 0 0
353: 1234/0x252e:     16505       3      2 access("/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-driver\0", 0x0, 0x0)		 = 0 0
354: 1234/0x252e:     16507       2      1 access("/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-driver\0", 0x0, 0x0)		 = 0 0
355- 1234/0x2530:     17970:        0:       0 fork()		 = 0 0
356- 1234/0x2530:     17971       1      0 mprotect(0x104EB4000, 0x8000, 0x1)

I also noticed this details in the README:
https://github.com/apple/swift-driver/blob/main/README.md?plain=1#L14
Previous snapshots of Swift worked without any problems in that regard. Could it be that using a self-built "release" disables forwarding executions to the driver?

Could you please clarify how your build a custom toolchain, exactly?

I used the official instructions aka:
swift/utils/build-toolchain <prefix>

I used the official instructions aka:
swift/utils/build-toolchain <prefix>

Could you please share which <prefix> you are using?

Sure, sorry for leaving out this details. My bundle prefix is customCompiler.
Is there anything special you have in mind? Are there some names that are considered "forbidden"?

@thekief do you have a swift-driver executable adjacent to /Library/Developer/Toolchains/swift-LOCAL-2023-05-02-a.xctoolchain/usr/bin/swiftc?
I am having trouble using build-toolchain script locally, but I just triggered a toolchain build on a release/5.8 PR here:
https://github.com/apple/swift/pull/65587
And the resulting toolchain seems to have the new driver correctly installed and it does get used when calling swiftc from this toolchain.

Thank you for your help there. I had a look at your build and I noticed that some files on my side were kind of missing and broken most likely due to some issues in the build process.
We had some issues during our build times in the night and so far restarting/resuming the build process never caused any issues. That is the first time for me here. I will test the toolchains internally and potentially close the ticket soon

i will inform you until next Monday. Currently, we switched back to Xcode 14, circumventing any problems with this. I will write about the outcome as soon as possible.

@thekief have you had a chance to reproduce this again?

Related to this one here:
https://github.com/apple/swift-package-manager/issues/6970
I kinda forgot about this issue. Thank you for the reminder

I don't think that SwiftPM issue is related in any way, unless some details in the description of this one were omitted.

Sorry for the bad phrasing:
I wanted to retest it with the new version mentioned in the other ticket. As I cannot build the new version, I have no update on it yet.

I was able to build release/5.9.0 and the issue does not occure anymore.


Jbaker/optional checkbox

Alaska Airlines Pull Request
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes: #87
Summary:
Adds a new noCheckmark attribute. When used, selected options will not show a checkmark. Nor will padding to the left of all options be inserted to make room for checkmarks.
Type of change:
Please delete options that are not relevant.

[x] New capability
[ ] Revision of an existing capability
[ ] Infrastructure change (automation, etc.)
[ ] Other (please elaborate)

Checklist:

[x] My update follows the CONTRIBUTING guidelines of this project
[x] I have performed a self-review of my own update

By submitting this Pull Request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Pull Requests will be evaluated by their quality of update and whether it is consistent with the goals and values of this project. Any submission is to be considered a conversation between the submitter and the maintainers of this project and may require changes to your submission.
Thank you for your submission!
-- Auro Design System Team

Was there a design specification for this update? @leeejune is this correct? The nested options seems to be in really far?

This is the correct design spec.


CLI should report error when a non-existent config file is specified

I was looking at this and it seemed like it does -- at least at the preAction in /packages/schema/src/cli/index.ts.
But in the actual loadConfig function of config.ts there is no check (just an early return) so a little bit of a code smell here.
In my PR #354 I went ahead and moved the existence check to the loadConfig function with the correct CliError throw. I also adjusted the logic there a bit and wrote / tweaked some tests to better detect what sort of errors are being thrown in some scenarios.
Hopefully that's useful, couldn't really tell what this issue was really referring to besides ensuring a useful CliError is being emitted if the user-specified config file path does not exist.

Fixed in release alpha.111.


Accordiantab style behaving differently when open and close mode

If the page contains mutilple accorian tabs, one of the Accordiantab style behaving differently when open and close mode.
Behaviour : when the Accordiantab is opened , the overflow style is true if it contains form.
when the Accordiantab is closed , the overflow style is true if it contains form and also creating blank space at the end of footer page.
Issue Details
Type of Issue  (check one with "X")
[ ] Bug Report => Please search GitHub for a similar issue or PR before submitting
[ ] Feature Request => Please ensure feature is not already in progress
[ ] Support Request => Please do not submit support requests here, instead see: https://discourse.oss.antheminc.com/

Current Behavior
when the Accordiantab is opened , the overflow style is true if it contains form.
when the Accordiantab is closed , the overflow style is true if it contains form and also creating blank space at the end of footer page.
Expected Behavior
when the Accordiantab is opened or closed  ,it should not  create blank space at the end of footer page.
How to Reproduce the Issue
Steps to Reproduce

Code Snippet

Environment Details

Nimbus Version:
1.1.11

Browser:
Chrome , version: Version 72.0.3626.109

nimbus-sass-module "version": "1.18.34" provided fix for accordion closed height issue
added .card-holder {overflow: hidden} to constrain visible content within parent container


Feat/docs newsletters

Needs to be merged AFTER https://github.com/Mashape/getkong.org/pull/520 has been merged as this is a branch off of that

@travispamaral from the look of the Files Changed, this appears to be only email newsletter stuff, no EE plugin stuff?


[mesheryctl] platform: kubernetes support for system stop command

Signed-off-by: navendu-pottekkat<EMAIL_ADDRESS>Description
This PR fixes #2311
mesheryctl now checks for the platform of the current-context and if the platform is kubernetes, it will delete all deployments made by Meshery. The user can also use the --context flag to switch contexts.
The logic of the case when the platform is docker is unchanged.

As shown above the pods are terminated when the platform is kubernetes.
This is done using the manifest files present in the .meshery/manifests folder and the adapters specified in the config.yaml file.
Notes for Reviewers

[x] Yes, I signed my commits.

What a great write-up.

A couple lint items to look into...

@leecalcote Fixed lint errors. I always forget to lint check locally and only realise it until I see the red cross mark on the PR.

Quick check - do we warn the user and ask for confirmation in order to delete the Meshery deployment?

Quick check - do we warn the user and ask for confirmation in order to delete the Meshery deployment?

Nope. I also thought about it since we will be deleting the deployments in kubernetes as opposed to just stopping the containers in docker. But since we are not asking this for docker, I did not add it here.

@navendu-pottekkat, given the behavior differences, it's good to consider either changing the command name to delete or prompting the user for confirmation - https://docs.google.com/document/d/1xRlFpElRmybJ3WacgPKXgCSiQ2poJl3iCCV1dAalf0k/edit?disco=AAAALLkU8Ck

@leecalcote I think it would be best if we add a user prompt since stop makes more sense in the case of docker.

@leecalcote I have added the user prompt similar to what we have for the reset command. Please check and suggest if there are any changes needed to the PR


[$250] Chat - Emoji category selection buttons not working

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 9.0.30-4
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught during regression testing, add the test name, ID and link from TestRail: https://expensify.testrail.io/index.php?/tests/view/4935849
Email or phone of affected tester (no customers<EMAIL_ADDRESS>Issue reported by: Applause Internal Team
Action Performed:

Launch App
Tap on any chat
Open emoji picker near compose and tap on any icon to select section under search field

Expected Result:
Emoji picker should be scroll to selected section.
Actual Result:
Scroll emoji via icon doesn't works. Nothing happens when tap on any icon to select section under search field
Workaround:
Unknown
Platforms:

[x] Android: Native
[ ] Android: mWeb Chrome
[x] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
https://github.com/user-attachments/assets/bab87e7f-5d2d-43b5-8795-a65df1809535

View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~021833174513558591946
Upwork Job ID:<PHONE_NUMBER>558591946
Last Price Increase: 2024-09-16
Automatic offers:

allgandalf | Reviewer | 104018090

Issue OwnerCurrent Issue Owner: @allgandalf

@alexpensify  FYI I haven't added the External label as I wasn't 100% sure about this issue. Please take a look and add the label if you agree it's a bug and can be handled by external contributors

Proposal
Please re-state the problem that we are trying to solve in this issue.
Emoji category button doesn't work.
What is the root cause of that problem?
When we press the emoji category, we call scrollTo and pass the ref to the function.
https://github.com/Expensify/App/blob/4af620b439636f714ff72a26320ed57f9d2faac3/src/components/EmojiPicker/EmojiPickerMenu/index.native.tsx#L68-L74
scrollTo is a reanimated function to scroll synchronously which requires an animated component or RN built-in component.
However, we are using FlashList as the list component.
https://github.com/Expensify/App/blob/4af620b439636f714ff72a26320ed57f9d2faac3/src/components/EmojiPicker/EmojiPickerMenu/BaseEmojiPickerMenu.tsx#L105-L106
But the doc also mentioned that it would work if the component implements getNativeScrollRef for the new arch which doesn't exist on FlashList and they have no plan on adding it.
This previously worked fine because we have a patch to add the getNativeScrollRef to flash-list and recylcerlistview which is the deps of flash-list, but was removed when upgrading RN. (the linked PR is part of the RN upgrade)
Btw, we used an animated FlatList when the scroll code was introduced.
What changes do you think we should make in order to solve the problem?

Use the ref scrollToOffset directly.
emojiListRef.current?.scrollToOffset({offset: calculatedOffset, animated: true})

Change it on both index and index.native files. Then, we can change this to useRef
https://github.com/Expensify/App/blob/4af620b439636f714ff72a26320ed57f9d2faac3/src/components/EmojiPicker/EmojiPickerMenu/useEmojiPickerMenu.ts#L13
What alternative solutions did you explore? (Optional)
There is already an upstream PR that will add the function but it has been stale for months. So, we can re-add the patch back that adds the getNativeScrollRef, but only the patch for the flash list code.

I'm able to replicate this experience on my Android device. @allgandalf, please review the proposals and identify if one will fix this issue. Thanks!

@allgandalf, any update for these proposals? Thanks!

Waiting for the review here

@allgandalf could you please clarify why you did not choose @nyomanjyotisa's proposal?

Just FYI, I also write the alternative if we want to keep using the animated ref.

Okay, makes sense, thanks 👍

@flodnv To confirm, which solution should I use?

I feel like the main proposed solution (not the alternative) is best in regards to ensuring this regression does not come back?

Got it. PR is ready
cc: @allgandalf

Awesome, now we wait for this one to go to Prouduction.

Flagging that this one went into production today.

Something weird is going on with Upwork, I'm going to hold to confirm if there is an issue with the other payments I just sent out.

There is an issue an Upwork that is being investigated to address the payment process. I'm going to hold until there is a fix. Convo: https://expensify.slack.com/archives/C01GTK53T8Q/p1728319262830679

Payouts due: 10-04-2024

[ ] Contributor: $250 @allgandalf
[ ] Contributor+: $250 @bernhardoj

Upwork job is here.

@allgandalf - our team figured out the Upwork bug, and I tried to complete the payment via another payment path in Upwork. Their support team is working on a permanent fix for the bug. I'd really appreciate it if you could confirm today's payment was sent successfully. Thanks!

@bernhardoj please submit a request in Chat!

Requested in ND.

$250 approved for @bernhardoj


proposal: cmd/go: allow disabling gocache trimming

Proposal Details
We have our own docker image containing our Go toolchain. As an optimization, we also pre-compile the standard library (go build std) so that it is in the gocache.
Evidently, the various Go tools will trim the cache upon exit. In particular, they will delete any files in the cache that have not been modified in the last five days. While this may be reasonable for a cache on someone's host machine, it does not make sense for a short-lived container using a cache from its image, since the mtimes will all reflect when the image was built. At best, it trimming the cache is pointless because the container will be destroyed shortly, and at worst is actively counterproductive as those cached files may be used by subsequent commands.
Please add an environment variable to disable trimming the gocache upon command exit. The specific name is immaterial, but something like GOCACHE_AUTO_TRIM=0 would be fine.

Related Issues and Documentation

cmd/go: clean cache periodically #22642 (closed)
cmd/go: make cache keys for libraries in GOROOT insensitive to `-trimpath` #57346

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

CC @matloob

I don't know if this justifies its own environment variable (that would show up in go env, etc.) but maybe we can make it an undocumented # godebug setting?

@matloob If you mean something we can set via the GODEBUG environment variable, that would also be acceptable. We just need something we can easily configure globally in the base image.

Yes. That is what I mean.


Ch 4 Factorization Machines Endpoint Content_Type error

In Chapter 4, Factorization Machines Notebook in the "Deploy Model" section there is an issue after creating the endpoint.  While attempting to run this code:
'fm_predictor.content_type = "application/json" '
You get this error;
'AttributeError: can't set attribute'
If you try to comment it out & run the next block of code you get this error:
'AttributeError: "function" object has no attribute "CONTENT_TYPE"'
I attempted to follow @laurenyu's feedback: https://github.com/aws/sagemaker-python-sdk/issues/1126 but that code doesn't appear to work.  Please advise.

This appears to  be due to the removal of the methods in version 2.x of the SageMaker:
content_type and accept in the Predictor Constructor
The content_type and accept parameters are now no-ops in the following classes and methods:

sagemaker.predictor.Predictor

sagemaker.estimator.Estimator.create_model

sagemaker.algorithms.AlgorithmEstimator.create_model

sagemaker.tensorflow.model.TensorFlowPredictor

Please specify content types in a serializer or deserializer class instead.

You need to specify the serializer and deserializer in the deploy method instead. Here is a sample code:
from sagemaker.deserializers import JSONDeserializer
from sagemaker.serializers import CSVSerializer

predictor = estimator.deploy(
    initial_instance_count=1,
    instance_type="ml.m4.xlarge",
    serializer=CSVSerializer(),
    deserializer= JSONDeserializer
)

That's correct. Fixing these is on my to do list, once AWS re:Invent is out the way :)

Fixed.


better nyc default configuration

there are a few shortcomings in how nyc is configured by default in synthtool:
we don't display coverage when someone is running tests locally
right now we only generate the lcov report on disk, we should also output human readable information.
we don't show files unless they have tests written for them
we can address this issue by running with --all, but there's a bug with nyc currently that stops this from working (I am volunteering to fix this upstream).

Pretty sure we landed this!


spanish [AnimeFlv]

Source name
animeFlv
Source link
https://www3.animeflv.net/
Source language
spanish
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a title with source name.
[X] I have checked that the extension does not already exist by searching the GitHub repository and verified it does not appear in the code base.

Already added by @kodjodevf I appreciate it


Mount go mod cache

To speed up things when the mod cache is required.

@heartrobotninja do you see any issues with this?

We should keep an eye on future Go releases which may change the functionality of the cache. LGTM.


SSI - Define & implement log functionality  (ext: CXAR-1046)

+Benefit Hypothesis+
As part of the agreement negotiation and closure, the dataspace audit log rules need to get defined and implemented.
In the current solution agreement logs are stored on EDC level. Additionally it needs to get ensured that the agreement party authentication/authorization (means VC and VP details in the time of the agreement closure) is stored securely

+Beneficiaries+
< Who will benefit for that feature>

Components: CPLP, EDC, MIW
Sprints: N/A
Fix Versions: N/A
StoryPoints: N/A
Attachments: N/A

closed - tickets are managed and created on sig-release
closing this ticket


Added redirect for roles.md

This change is

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/a1380f0100a3cc459d960721f4990053feb1a343/
Edited pages:

v20.1/authorization.md


Initial commit of self-intersection check - IVS-45

This turned out quite a bit more elaborate than expected, but have a first look and let me know where you have questions.

I don't see a *.feature file. Did I miss it or is this a general template that will result in a series of GEM rules?

@civilx64  1000x apologies. Added it.

Note that the feature file and test files still need to be broken up per what we conclude.

@civilx64 I added a new joke called 'full_stack_rule' which means that in addition to the inst parameter you get a path (list) of all previous givens that trace back inst to the first given statement. This is necessary for the tesselated faces because a tesselated face is just a list of integer indices, and the vertex coordinate positions are defined one level higher.
https://github.com/buildingSMART/ifc-gherkin-rules/pull/229/commits/e64c55104f7478e81038e5862ff3812d93a409ae#diff-2c838ec4f24dc4e43b898ddaa7de4de8f6cc972099850db514753312625e105a


OBS-191 | Viewing domains page should not require Manage Domains priv…

…ilege
Checklist

[ ] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[ ] Links to related issues (if applicable)
[ ] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
[ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

Actually this looks incorrect.
The user should be able to view domains always, even if they cannot create domains.
I should have specified further.

Got it. We should remove the privilege checks for creating and managing domains since they should always be visible.


Release/2.4.1

Fixes

Coverage increased (+0.004%) to 99.458% when pulling fe9c016717c2c16e5e24636bb1454356da669538 on release/2.4.1 into 17bcc9bf32b9113120da05b5e8d8d67691cf446e on master.


ES6 class vs React.createClass

Hi, are there any plans of switching to components React.Component instead of React.createClass?

Down for that too

Would love to see a fork, and ideally one where mostly pure components are used versus extending React.Component, though I've had issues getting HMR working in RN with pure.


doc: net: Enhance Ethernet management documentation

The Ethernet management documentation was missing Overview
chapter.
Signed-off-by: Jukka Rissanen<EMAIL_ADDRESS>

Codecov Report

Merging #13784 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #13784   +/-   ##
=======================================
  Coverage   52.26%   52.26%
=======================================
  Files         307      307
  Lines       45404    45404
  Branches    10504    10504
=======================================
  Hits        23729    23729
  Misses      16886    16886
  Partials     4789     4789

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 51a00cf...1cc1ec4. Read the comment docs.


ES6 import fails in render method

Importing ReactGridLayout like this
import ReactGridLayout from "react-grid-layout";

and using it like this in render()
    <ReactGridLayout
      className="mainContainer"
      layout={layout}
      cols={12}
      rowHeight={30}
      width={1200}
    ><children/></ReactGridLayout>

compiles but leads to the following error in the browser console (and a white-screen):

proxyConsole.js:54 Warning: React.createElement: type is invalid -- expected a string (for built-in components) or a class/function (for composite components) but got: undefined.  You likely forgot to export your component from the file it's defined in.
Check the render method of GeneViewer.
at invariant (invariant.js:42)
at createFiberFromElementType (react-dom.development.js:8185)
at createFiberFromElement (react-dom.development.js:8130)
at createChild (react-dom.development.js:8832)
at reconcileChildrenArray (react-dom.development.js:9132)
at reconcileChildFibers (react-dom.development.js:9495)
at reconcileChildrenAtPriority (react-dom.development.js:10127)
at reconcileChildren (react-dom.development.js:10118)
at updateHostComponent (react-dom.development.js:10370)
at beginWork (react-dom.development.js:10609)
at performUnitOfWork (react-dom.development.js:12573)
at workLoop (react-dom.development.js:12682)
at HTMLUnknownElement.callCallback (react-dom.development.js:1299)
at Object.invokeGuardedCallbackDev (react-dom.development.js:1338)
at invokeGuardedCallback (react-dom.development.js:1195)
at performWork (react-dom.development.js:12800)
at scheduleUpdateImpl (react-dom.development.js:13185)
at scheduleUpdate (react-dom.development.js:13124)
at Object.enqueueSetState (react-dom.development.js:9646)
at PersistGate../node_modules/react/cjs/react.development.js.ReactComponent.setState (react.development.js:218)
at PersistGate._this.handlePersistorState (react.js:37)
at Object.dispatch (createStore.js:173)
at Object.rehydrate (persistStore.js:91)
at persistReducer.js:66
at
invariant @ invariant.js:42
createFiberFromElementType @ react-dom.development.js:8185
createFiberFromElement @ react-dom.development.js:8130
createChild @ react-dom.development.js:8832
reconcileChildrenArray @ react-dom.development.js:9132
reconcileChildFibers @ react-dom.development.js:9495
reconcileChildrenAtPriority @ react-dom.development.js:10127
reconcileChildren @ react-dom.development.js:10118
updateHostComponent @ react-dom.development.js:10370
beginWork @ react-dom.development.js:10609
performUnitOfWork @ react-dom.development.js:12573
workLoop @ react-dom.development.js:12682
callCallback @ react-dom.development.js:1299
invokeGuardedCallbackDev @ react-dom.development.js:1338
invokeGuardedCallback @ react-dom.development.js:1195
performWork @ react-dom.development.js:12800
scheduleUpdateImpl @ react-dom.development.js:13185
scheduleUpdate @ react-dom.development.js:13124
enqueueSetState @ react-dom.development.js:9646
./node_modules/react/cjs/react.development.js.ReactComponent.setState @ react.development.js:218
PersistGate._this.handlePersistorState @ react.js:37
dispatch @ createStore.js:173
rehydrate @ persistStore.js:91
(anonymous) @ persistReducer.js:66
Promise resolved (async)
(anonymous) @ persistReducer.js:65
Promise resolved (async)
(anonymous) @ persistReducer.js:61
computeNextEntry @ VM6992:2
recomputeStates @ VM6992:2
(anonymous) @ VM6992:2
dispatch @ createStore.js:165
dispatch @ VM6992:2
(anonymous) @ createEpicMiddleware.js:59
(anonymous) @ middleware.js:13
dispatch @ VM6992:2
persistor.persist @ persistStore.js:99
persistStore @ persistStore.js:102
./src/index.tsx @ index.tsx:63
webpack_require @ bootstrap 97c9be831c1e011b5fcb:669
fn @ bootstrap 97c9be831c1e011b5fcb:87
0 @ Loading.tsx:10
webpack_require @ bootstrap 97c9be831c1e011b5fcb:669
(anonymous) @ bootstrap 97c9be831c1e011b5fcb:715
(anonymous) @ bundle.js:719
proxyConsole.js:54 The above error occurred in the  component:
in div (created by GeneViewer)
in GeneViewer (created by Connect(GeneViewer))
in Connect(GeneViewer) (created by Route)
in Route (created by App)
in div (created by App)
in MuiThemeProvider (created by App)
in App (created by Route)
in Route (created by withRouter(App))
in withRouter(App)
in Router (created by ConnectedRouter)
in ConnectedRouter
in PersistGate
in Provider
Consider adding an error boundary to your tree to customize error handling behavior.
 You likely forgot to export your component from the file it's defined in.
Check the render method of GeneViewer.
at invariant (invariant.js:42)
at createFiberFromElementType (react-dom.development.js:8185)
at createFiberFromElement (react-dom.development.js:8130)
at createChild (react-dom.development.js:8832)
at reconcileChildrenArray (react-dom.development.js:9132)
at reconcileChildFibers (react-dom.development.js:9495)
at reconcileChildrenAtPriority (react-dom.development.js:10127)
at reconcileChildren (react-dom.development.js:10118)
at updateHostComponent (react-dom.development.js:10370)
at beginWork (react-dom.development.js:10609)
at performUnitOfWork (react-dom.development.js:12573)
at workLoop (react-dom.development.js:12682)
at HTMLUnknownElement.callCallback (react-dom.development.js:1299)
at Object.invokeGuardedCallbackDev (react-dom.development.js:1338)
at invokeGuardedCallback (react-dom.development.js:1195)
at performWork (react-dom.development.js:12800)
at scheduleUpdateImpl (react-dom.development.js:13185)
at scheduleUpdate (react-dom.development.js:13124)
at Object.enqueueSetState (react-dom.development.js:9646)
at PersistGate../node_modules/react/cjs/react.development.js.ReactComponent.setState (react.development.js:218)
at PersistGate._this.handlePersistorState (react.js:37)
at Object.dispatch (createStore.js:173)
at Object.rehydrate (persistStore.js:91)
at persistReducer.js:66
at

What's the solution for this? I am seeing same error. here is my code:
import * as React from 'react';
import GridLayout from 'react-grid-layout';

export default class MyGrid extends React.Component<any, any> {

    render() {

        return (
                                <GridLayout className="layout" cols={12} rowHeight={30} width={1200}>

a

b

c

                                </GridLayout>
        );
    }
}

Same error for me. What I deed is import GridLayout like that :
import * as GridLayout from 'react-grid-layout';
instead of :
import { GridLayout } from 'react-grid-layout';


invalid unordered_map<K, T> key

Hi - after upgrading to node 0.12 we seem to be getting the following error from the gulp-sass plugin.
We have had to roll back to 0.10 until we resolve this:
ERROR [karma]: { [Error: invalid unordered_map<K, T> key]
message: 'invalid unordered_map<K, T> key',
fileName: undefined,
lineNumber: undefined,
showStack: false,
showProperties: true,
plugin: 'gulp-sass',
__safety: { toString: [Function] } }
[10:26:17] Karma has exited with 1
Versions:
├─┬<EMAIL_ADDRESS>│ ├─┬<EMAIL_ADDRESS>│ │ ├─┬<EMAIL_ADDRESS>Is this something that I should be raising on the node-sass project instead?

Also got this bug.
Could not find a clear reason for that, happening in Foundation for Apps sass source.

Linked to the following?
http://discuss.prepros.io/t/libsass-error-invalid-unordered-map-k-t-key/287

I have found out what is causing this, by first guessing that maybe it was one of our scss files causing it and then individually going through them one by one compiling each one before narrowing down the cause even within that file.
So strangely here it is:
WORKS:
$map: (
test: 1
);
ERROR:
$map: (
notest: 1
);
If you have a sass map, where a key starts with 'no' - then you will see the following error:
invalid unordered_map<K, T> key
I have raised a bug on libsass as I presume that's where the bug is (https://github.com/sass/libsass/issues/934), however is gulp-sass able to at least report the file that has the issue?

Any luck on this @Haemoglobin?
I'm getting this issue on Foundation for Apps too

Yes - please follow the issue I linked to on libsass, it was an issue there that they have now fixed for the next release. The workaround was to quote the sass map keys for now.


Add IPv6 support and tests for nexd TCP proxy

commit<PHONE_NUMBER>2ded58a04a80ab4173d561729cf963
Author: Russell Bryant<EMAIL_ADDRESS>Date:   Wed Apr 19 01:28:29 2023 -0400
nexd: Add IPv6 tunnel IP to the userspace proxy device

The userspace proxy device was configured only with the IPv4 tunnel
address. This adds the IPv6 address to the device, as well.

Signed-off-by: Russell Bryant<EMAIL_ADDRESS>
commit 13f0f6148e30812f1f2cbbf620f72469d3f5f27f
Author: Russell Bryant<EMAIL_ADDRESS>Date:   Wed Apr 19 01:29:30 2023 -0400
e2e: Add IPv6 TCP proxy test coverage

Test TCP ingress and egress proxy rules that cover:

- v4 to v4
- v4 to v6
- v6 to v4
- v6 to v6

Signed-off-by: Russell Bryant<EMAIL_ADDRESS>
Part of #816

going to include this with #858


Add dep ensure before starting mattemost server

it seems that we must use dep ensure before make run to start mattermost server, other way it tell me a package is missing.
actual error:
`cmd/mattermost/main.go:9:2: cannot find package "github.com/mattermost/mattermost-server
/cmd/mattermost/commands" in any of:
/home/wsebai/go/src/mattermost-server/vendor/github.com/mattermost/mattermost-server/cmd/mattermost/commands (vendor tree)
/usr/lib/go/src/github.com/mattermost/mattermost-server/cmd/mattermost/commands (from $GOROOT)
/home/wsebai/go/src/github.com/mattermost/mattermost-server/cmd/mattermost/commands (from $GOPATH)
cmd/mattermost/main.go:15:2: cannot find package "github.com/mattermost/mattermost-server/imports" in any of:
/home/wsebai/go/src/mattermost-server/vendor/github.com/mattermost/mattermost-server/imports (vendor tree)
/usr/lib/go/src/github.com/mattermost/mattermost-server/imports (from $GOROOT)
/home/wsebai/go/src/github.com/mattermost/mattermost-server/imports (from $GOPATH)
cmd/mattermost/main.go:12:2: cannot find package "github.com/mattermost/mattermost-server/model/gitlab" in any of:
/home/wsebai/go/src/mattermost-server/vendor/github.com/mattermost/mattermost-server/model/gitlab (vendor tree)
/usr/lib/go/src/github.com/mattermost/mattermost-server/model/gitlab (from $GOROOT)
/home/wsebai/go/src/github.com/mattermost/mattermost-server/model/gitlab (from $GOPATH)
`

Hey @walidsebai, thanks for relocating the original issue and posting the detailed error.
Something doesn't seem right in the errors above: github.com/mattermost/mattermost-server /cmd/mattermost/commands is part of the repository in question, so it would never be part of vendor/ in the first place, and thus not affected by dep ensure.
Did you follow https://developers.mattermost.com/contribute/server/developer-setup/ to setup your environment? Can you describe any extra steps you took that led to this error?

yes i did follow , maybe i did the Edit /etc/security/limits.conf the last thing but don't thing that would affect the setup. i think it would be better if we recreate it ?

i think i understand the problem, it's my fault i downloaded the source code in go/src/ so yeah u are right sorry for the inconvenience, i'm closing this now.


Added Create survivors and zombies, make them attack, and print results. Corrected Soldier file name.

Looks good to me!


i18n(fr): update tutorials/add-content-collections.mdx

Description (required)
Updates a link in tutorials/add-content-collections.mdx (see #9687)

[!important]
#9825 needs to be merged first to resolve the link error

Related issues & labels (optional)

Suggested label: i18n

Lunaria Status Overview
🌕 This pull request will trigger status changes.

Learn more
By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.
You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

Locale
File
Note

fr
tutorials/add-content-collections.mdx
Localization changed, will be marked as complete.

Warnings reference

Icon
Description

🔄
The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.


Simplified the configuration using a single 'config' key in DI container

This PR simplify the configuration process using only one config key in the DI container.
This can be useful also for retrieving the configuration in any Controller, injecting just the DI container instance.
An example of configuration is as follows, I used 2 files for the general application configuration (config.php) and for the DI container (container.php):
// config/config.php
use App\Controller;
return [
    'routing' => [
        'routes => [
            [ 'GET', '/', Controller\HomePage::class ]
        ]
    ],
    'pdo_dsn' => 'sqlite:data/db.sqlite'
];

// config/container.php
use PDO;
use Psr\Container\ContainerInterface;
return [
    'config' => require 'config.php',
    // specific DI configurations (eg. PDO)
    PDO::class => function(ContainerInterface $c) {
        return new PDO($c->get('config')["pdo_dsn"]);
    }
];

In this way we just need to pass the config/container.php file to the DI container.
For instance, using PHP-DI we can use the following code:
use DI\ContainerBuilder;
use SimpleMVC\App;

$builder = new ContainerBuilder();
$builder->addDefinitions('config/container.php');
$container = $builder->build();
$app = new App($container);

@danielebarbaro and @adamculp thanks for the feedback. I applied the suggestions of @danielebarbaro.  Let me know if you have more comments, thanks!

@adamculp, @danielebarbaro I'm considering to move the info logger into debug for the following cases:

Executing $controller;
Found HaltResponse in $controller;
Bootstrap execution: %.3f sec

Leaving the HTTP request, execution time and memory consumption as info. I think this can be also useful for monitoring an application, without the need to have debug level. Thoughts?

I typically leave debug for things that are diagnostically relevant for someone, while info is used for items I really don't care about, but want available. The items you mention seem to be debug candidates. Make it so!

I completely agree. No doubts for me, you can go ahead with another commit


Add UI features from Demo work

Several small changes from added features during the demo:

Added the "demo" banner at the top of the site and made it sticky.
Updated some card labels in the Report view and adjusted the styles so the columns aren't fixed. Didn't specifically get feedback from UX on the behavior but this was requested from users.
Also made the summary cards more flexible for different content.
Added an overflow to the card in case the user is on a small screen. This was noticeable in the summary table.
Removed "on target reach" from the charts.
Updated the title.
Added Google Analytics.

Bug Fixes:

Added a margin at the bottom of the site so the feedback button isn't stuck over things at the bottom like charts.
Fixed fonts. There was an extra "./" in the font src that would work locally but screw up webpack with routes on the server. Also removed the extra font styles and just left sans serif.
Added more cases to the number to magnitude converter including handling the 0 case to make the logic simpler.
Updated BUILD files for some ChartGroups. These charts were removed from the current UI so they didn't fail the build.

This change is


FWIW I'm cool with the revert as is. If we ever need this again, we know where to find it


faq mobile view

faq mobile view

your page rendered nicely but there are a few things to take note of
Faq mobile

add shadow and color of the boxes( Questions ) to match design on figma board
put the logos inside an  tag
Use auto margin for flex box container to center it

Thank you,
please push the desktop view


Prerequisites for supporting domain detection classes

A few prereqs to keep the commits cleaner.  Notably:

Extending Reader to support repeatable reads
Making ResultIndicator able to behave as a bool
Fixing and improving geopackage hydrofabric implementation and test data
Adding some data formats

Curious if the failures are related to removing Fiona as a dependency? I'm AFK, so I've not pulled the branch and verified locally.

I'm working to see what's going on with the tests.

@aaraney, I think all the review comments have been properly addressed either in conversation, separate issues, or additional commits, and I have the tests passing now also, but let me know if I've missed anything.

Sorry, I didn't catch this until after i'd approved. I went to take a look at #565 and in passing noticed my mistake.


clightning: 0.8.0 -> 0.8.1

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://github.com/ElementsProject/lightning/releases.
meta.description for clightning is: '"A Bitcoin Lightning Network implementation in C"'.
meta.homepage for clightning is: '"https://github.com/ElementsProject/lightning"
Release on GitHub
Compare changes on GitHub

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 0.8.1 with grep in /nix/store/qkjxjn5cw8dsscg8lpr36hcf2zp90r38-clightning-0.8.1

directory tree listing: https://gist.github.com/b2d6157792ca88004862e9e5f7d19a3a

du listing: https://gist.github.com/2b41a323da33054df530c66f304779c6

Rebuild report (if merged into master) (click to expand)

3 total rebuild path(s)
1 package rebuild(s)
1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
0 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)
First fifty rebuilds by attrpath
clightning

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/qkjxjn5cw8dsscg8lpr36hcf2zp90r38-clightning-0.8.1 \
  --option binary-caches 'https://cache.nixos.org/ https://r-ryantm.cachix.org/' \
  --option trusted-public-keys '
  r-ryantm.cachix.org-1:gkUbLkouDAyvBdpBX0JOdIiD2/DP1ldF3Z3Y6Gqcc4c=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(r-ryantm's Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A clightning https://github.com/r-ryantm/nixpkgs/archive/3f3ead76fdb741a0dd71a694a7b24efa024848d8.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/qkjxjn5cw8dsscg8lpr36hcf2zp90r38-clightning-0.8.1
ls -la /nix/store/qkjxjn5cw8dsscg8lpr36hcf2zp90r38-clightning-0.8.1/bin

cc @jb55 for testing.

post-merge ACK, I checked the signatures. @mmahut I think we should have at least one signature check ACK before merging updates to financial software :sweat_smile: or at least until we have signature checking as a part of the derivation

@jb55 I did check that update as I was already playing with it few hours ago.
I'm sorry, I should have at least written something before merging it. I will be more careful next time.


Add usage documentation

Can you please add a short documentation for how to use the plugin once installed? I wasn't able to figure it out by myself - using Discourse 2.3.

Hi @fuerst, very sorry about the late response, I wasn't watching the repo.
I have added instructions on how to install and set up the plugin in eb411ec, let me know if this clears it up. Thanks for checking out the plugin!


 Kan dit samen besproken worden met de medicatie FHIR WG, zodat er zeker 1 lijst is?

WG advises to use the same list of values that is defined and used in the medication packages.

The route as defined in the logical model is broader than the traditional concept of "route". We strongly suspect that the other values, beyond the traditional concept, will be coded in another place than medication.

WG: to verify using concrete examples.

WG: Marie-Alexandra Lambot and Anne Nerenhausen are working on the SNOMED CT list for administration routes. It will be a transversal valueset that will be shared with the medication projects. For every context there will be a subset of the general refset  available.

WG: we will receive a document today.

updated in #205

WG: we wait for the input of the SteerCo.

Linked to the use of MedicationRequest as a contained resource in the ServiceRequest


Add Twitter links to truck listing

This fixes #1 (for trucks that are on twitter). Totally open to other ways to present / lay out this info :smile:

I think this unhelpfully out of date by now.


G2P Features search bug

When I post a request to features/search with name: "*" using the compliance dataset I get an Internal Server Error.
http post localhost:8000/features/search featureSetId=WyJicmNhMSIsImdlbmNvZGV2MTkiXQ name="*"

@david4096  just saw this now...

Thanks for raising this.

Thanks for looking into it @bwalsh! Considering this fact, we might reconsider the notion that regular expressions can be implemented neutrally.
That being said, what is the expected behavior? Should we guard for it and move on?

Hey @bwalsh let me know if you think you'll have time to write a guard for this.

I should be able to get to this by Mon 10/31

I cannot reproduce this error. I don't get any results though. I would expect the full set of results. If I don't include name, then I get the list of everything.
http post localhost:8000/features/search featureSetId=WyJicmNhMSIsImdlbmNvZGV2MTkiXQ

Here is the whole trace of the error:
Traceback (most recent call last):
  File "/home/david/git/server-1/env/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/david/git/server-1/env/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "ga4gh/server/frontend.py", line 604, in searchFeatures
    flask.request, app.backend.runSearchFeatures)
  File "ga4gh/server/frontend.py", line 467, in handleFlaskPostRequest
    return handleHttpPost(flaskRequest, endpoint)
  File "ga4gh/server/frontend.py", line 344, in handleHttpPost
    responseStr = endpoint(request.get_data())
  File "ga4gh/server/backend.py", line 911, in runSearchFeatures
    self.featuresGenerator)
  File "ga4gh/server/backend.py", line 568, in runSearchRequest
    for obj, nextPageToken in objectGenerator(request):
  File "ga4gh/server/backend.py", line 406, in featuresGenerator
    request, featureSet, parentId)
  File "ga4gh/server/paging.py", line 397, in __init__
    super(FeaturesIterator, self).__init__(request)
  File "ga4gh/server/paging.py", line 325, in __init__
    self._objectList = self._search()
  File "ga4gh/server/paging.py", line 418, in _search
    self._request.gene_symbol))
  File "ga4gh/server/datamodel/genotype_phenotype_featureset.py", line 146, in getFeatures
    for row in featuresResults.bindings:
  File "/home/david/git/server-1/env/local/lib/python2.7/site-packages/rdflib/query.py", line 178, in _get_bindings
    self._bindings += list(self._genbindings)
  File "/home/david/git/server-1/env/local/lib/python2.7/site-packages/rdflib/plugins/sparql/evaluate.py", line 350, in evalDistinct
    res = evalPart(ctx, part.p)
  File "/home/david/git/server-1/env/local/lib/python2.7/site-packages/rdflib/plugins/sparql/evaluate.py", line 236, in evalPart
    return evalProject(ctx, part)
  File "/home/david/git/server-1/env/local/lib/python2.7/site-packages/rdflib/plugins/sparql/evaluate.py", line 360, in evalProject
    res = evalPart(ctx, project.p)
  File "/home/david/git/server-1/env/local/lib/python2.7/site-packages/rdflib/plugins/sparql/evaluate.py", line 245, in evalPart
    return evalOrderBy(ctx, part)
  File "/home/david/git/server-1/env/local/lib/python2.7/site-packages/rdflib/plugins/sparql/evaluate.py", line 321, in evalOrderBy
    res = sorted(res, key=val, reverse=reverse)
  File "/home/david/git/server-1/env/local/lib/python2.7/site-packages/rdflib/plugins/sparql/evaluate.py", line 153, in evalFilter
    if _ebv(part.expr, c.forget(ctx)):
  File "/home/david/git/server-1/env/local/lib/python2.7/site-packages/rdflib/plugins/sparql/evalutils.py", line 50, in _ebv
    return EBV(expr.eval(ctx))
  File "/home/david/git/server-1/env/local/lib/python2.7/site-packages/rdflib/plugins/sparql/parserutils.py", line 194, in eval
    return self._evalfn(ctx)
  File "/home/david/git/server-1/env/local/lib/python2.7/site-packages/rdflib/plugins/sparql/operators.py", line 217, in Builtin_REGEX
    return Literal(bool(re.search(unicode(pattern), text, cFlag)))
  File "/home/david/git/server-1/env/lib/python2.7/re.py", line 146, in search
    return _compile(pattern, flags).search(string)
  File "/home/david/git/server-1/env/lib/python2.7/re.py", line 251, in _compile
    raise error, v # invalid expression
error: nothing to repeat
<IP_ADDRESS> - - [05/Dec/2016 15:46:14] "POST /features/search HTTP/1.1" 500 -

Any invalid regex will throw a 500, this probably affects description fields in g2p as well:
schemas git:(607_attributes) ✗ http post localhost:8000/features/search featureSetId=WyJicmNhMSIsImdhNGdoLWV4YW1wbGUtZGF0YS9jZ2QiXQ name="["
HTTP/1.0 500 INTERNAL SERVER ERROR
Content-Length: 61
Content-Type: application/json
Date: Tue, 06 Dec 2016 00:10:14 GMT
Server: Werkzeug/0.11.5 Python/2.7.12+

{
    "errorCode":<PHONE_NUMBER>,
    "message": "Internal Server Error"
}

The previous query is backed by an RDF datastore. From what I understand, it doesn't accept query strings so name="" is looking for a feature named '', so no error. On the other hand, http post localhost:8000/features/search featureSetId=WyJicmNhMSIsImdhNGdoLWNvbXBsaWFuY2UtZGF0YS9jZ2QiXQ name="*" does cause an error. It is trying a python regular expression. However, '' is invalid. The '.' is valid python regex as noted before. Therefore, I am going to treat this issue as a bug in not validating the python expression and resolve it by returning a bad request error.


[Question]: Different chunks returned in RAGFlow vs LangChain

Describe your problem
I've been testing queries in RAGflow using the embedding model (all-MiniLM-L6-v2) and 'General' method to chunk and embed documents and my ollama (Llama3.1 8B) for LLM inference. I am comparing the results to a streamlit/langchain app that I built myself and I'm getting very different results for the document chucks that are returned as part of the RAG query.
I have been comparing results returned during retrieval testing in RAGflow versus the chunks returned when I do the same search on my solution built on langchain+faiss. I seem to be getting much more accurate results with the langchain+faiss solution. Is there any way I can improve the accuracy of the chunks returned in RAGflow as it is not providing the LLM with the correct chunks at the moment?

Could you elaberate on your cases?
And please try this setup:

I am reviewing legal documents like large loan agreements and contracts. I have spent much time trying different chunking methods, delimiters, token numbers, keywords etc. From what I can see the 'problem' is not with the chunking of the documents but rather the results returned by the vector search.
I get similar chucks returned when I ask different questions which means I rarely get the correct answer. I have also noticed that the Vector similarity for the results are 0  so I'm not sure it is even doing a vector search

Weird!
Vector sim can't be 0 anyway.
What's your embedding model?

I thought it was weird too. I'm using the all-MiniLM-L6-v2 model that comes with 'full' build. I can see the document chunks and the embedding seems to run without any error, but when testing retrieval no vector search is performed.


404s on configuration update

Describe the bug
Update of a configuration while requests are being served. In this case a canary configuration update. So 2 separate SVCs.
To Reproduce
Will try to provide a focused example but want to post logs first to see if anyone can spot somehting.
Expected behavior
No 404s.
Versions (please complete the following information):

Ambassador: [e.g. 0.86.1]
Kubernetes environment [e.g. Minikube, bare metal, Google Kubernetes Engine]
Version [e.g. 1.16.3]

Ambassador DEBUG log including the configuration update 404 request.
partial_amb.log

We are seeing the same thing, and we're pretty sure it's an envoy issue https://github.com/envoyproxy/envoy/issues/9420


Fog seems to not be respected anymore

Fixed by adjusting the traceScene function to not rely on a light hit


load project file from link in htmlified project

is it possible to make htmlifer use a project file from a link, so I can upload the scratch project file somewhere, get the direct download link, then I put it in htmlifer, then when it runs, it gets the file from the link, so I can make the file way smaller by having htmlifer get the project file from the web instead of in the file.

From here
Related to #21

Oh, this was closed because the URL option is there, but not to exclude it from the file. Partial implementation, I see.


Update Page “about”

Automatically generated by Netlify CMS

👷 Deploy Preview for akulina processing.

Name
Link

🔨 Latest commit
c798cf50df33433e0dfe166b4ffc7caa84dd9ec7

🔍 Latest deploy log
https://app.netlify.com/sites/akulina/deploys/63a0ffac9d38120008842ac5


Request: specify static port in config for listener

I attempt to run my homebridge behind UFW Firewall. I noticed that this app launches the listener on a new port every time Homebridge restarts.
Is it possible to specify a static port in the homebridge config for this plugin to use the same port after each homebridge restart? This will allow HomeKit to continue communication with the xbox behind a UFW Firewall.

U can use  only range  of ports to used by Homebridge plugins:
https://github.com/grzegorz914/homebridge-xbox-tv/blob/master/sample-config.json
Set specific port for desired plugin is not possible on plugin site, this is implemented in homebridge itself.

Thank you so much for the fast reply. This should resolve my issue.


Remove setting of training schema when creating subscription

Removed the setting of training schema when creating subscription

@arsuryan - Please review