[meta] Standardize Storage Paths for K8S Components

For all v2 components that declare volumes, we should standardize on the location to make troubleshooting and expectations consistent. We should audit all components and make sure there is a sensible filesystem layout!
For example, minio should store data in /data/minio/..., postgres would use /data/postgres/...
Refs deis/deis#4809

I've noticed (as I read through various stuff) that in Kubernetes, they seem to treat /var/run/secrets as the de facto path for mounting secrets. Maybe /var/run/data would be a nice parallel for persistent data?
/cc @arschles @helgi and @krancour

I'm :+1: on /var/run/data for persistence

Is there any technical benefit to this other than consistency? For example in deis/postgres it makes sense to mount the minio AWS credentials in /etc/wale.d/env/ because those values are then used as environment variables when paired with envdir. Not sure if this is something we should target for beta, as this qould require changes both at deis/charts and in each respective repo just for consistency.

Audit of storage locations:

deis-dev/builder

minio-user secret: /var/run/secrets/object/store
builder-key-auth secret: /var/run/secrets/api/auth
objectstore-creds secret: `/var/run/secrets/deis/objectstore/creds

deis-dev/database

minio-user secret: /etc/wal-e.d/env
database-creds secret: /var/run/secrets/deis/database/creds
objectstore-creds secret: /var/run/secrets/deis/objectstore/creds

deis-dev/registry

registry-storage empty dir: /var/lib/registry
registry-creds secret: /var/run/secrets/deis/registry/creds

deis-dev/workflow

docker-socket dir: /var/run/docker.sock
minio-user secret: /var/run/secrets/deis/minio/user
builder-key-auth secret: /var/run/secrets/api/builder/auth
django-secret-key secret: /var/run/secrets/api/django
database-creds secret: /var/run/secrets/deis/database/creds

deis-dev/minio

minio-admin secret: /var/run/secrets/deis/minio/admin
minio-user secret: /var/run/secrets/deis/minio/user

deis-dev/router mounts no volumes
router-dev mounts no volumes
router mounts no volumes
deis-logger mounts no volumes

@bacongobbler what are the code changes necessary in the deis/database repo to move that?

Should be only three line changes: https://github.com/deis/postgres/blob/16d13629d72a512b9f99521de2064d5cf0f254b5/rootfs/docker-entrypoint-initdb.d/001_setup_envdir.sh#L3-L7

@bacongobbler thanks!

I have no clue why it mounts the docker socket. Probably from when we were doing new FS layers or maybe it's required to do the docker client stuff re: ports discovery and stuff. @mboersma @bacongobbler  ?

Yeah it was from when we pulled in the image, injected the envvars into the image then re-pushed to the registry. However I still think we need it: https://github.com/deis/workflow/blob/60041c52681a3de879079c80e3dfd25420f4cc85/rootfs/scheduler/init.py#L595-L598

@bacongobbler @helgi ok, I'll leave it in for now

@slack @bacongobbler @helgi turns out this will require a few non-trivial code changes. I recommend that we punt this to RC for the following reasons:

Releasing a beta without these changes won't hinder the individual-component upgrade path
The changes required pose a risk, but the reward seems only to be more standardized paths, which I don't think is worth it for beta

Thoughts?

:+1:. It ;s the same argument held over at https://github.com/deis/charts/pull/142. High risk, low reward, doesn't impact component upgrade path.

Do we have a handle on what needs changing? If the above list is good enough and shows us what needs to be done then we can easily post that as "here are all the paths" in docs and call it a day for beta, potentially even RC.
Punt to RC for now and lets revisit then

Ok, punting to RC1

@slack as of now we aare using a single storage secret for every component and Paths look more standard than before https://github.com/deis/workflow/blob/master/src/installing-workflow/configuring-object-storage.md#deisdatabase. what else can be done here moving to next sprint

Good enough!


add version for busy box

add the current version of busy box to our docker version latest shell script

PR merged


Flag Based Tests

There are settings within the Tenant UI that remove various elements, depending on levels of permission. Work on #717, has turned off tests that relate to components with write permissions to the Database or Ledger.
Can we write tests that look at the variables within the appropriate config files and turn on or off the correct tests? These files include:

local.json - Developing locally
values-pr.yaml - Deployed pull request
values-development.yaml - Deployed to development

Might be some misunderstanding, tests don't relate to the helm values in the yamls at all, those are deployment side, whereas the unit tests run non-deployed code. The code will never know anything about values.
It's not about turning on or off tests IMO. A unit test should just exercise code based on inputs to check expected output. So simply what's needed is the test exercising the component to go "based on this value in the config store (so that's easily mockable without needing to know anything about actual config deployment values), should this button be visible, or not?"
See @jamshale comment here https://github.com/bcgov/traction/pull/743#issuecomment-1660789719
this concept is already done in another test.

Might be some misunderstanding, tests don't relate to the helm values in the yamls at all, those are deployment side, whereas the unit tests run non-deployed code. The frontend code will never know anything about helm chart values.
It's not about turning on or off tests IMO. A unit test should just exercise code based on inputs to check expected output. So simply what's needed is the test exercising the component to go "based on this value in the config store (so that's easily mockable/changeable-in-scope without needing to know anything about actual config deployment values), should this button be visible, or not?"
See @jamshale comment here #743 (comment) this concept is already done in another test.

Hmmm... 🤔 ... Think I get what you're saying @loneil .
So maybe change this ticket to something like... Upgrade existing tests to account for the  showWritableComponents flag??

"Upgrade existing tests to account for the showWritableComponents flag" is perfect 😄

I can do this ticket. Could be reference for changing configs in the future.


Buttons and links seem to be gone...

Hi Giuilio
The example spa app looks quite different to the that it used to and doesn't seem to have any functionality any more. Maybe I'm doing something wrong but there's no links or buttons and it doesn't seem to do anything....
thanks

Closing in favour of https://github.com/gcanti/tcomb-spa-example/issues/12


Okhai/rot

[x] issue https://github.com/keep-starknet-strange/shinigami/issues/70
[x] follows contribution guide
[ ] code change includes tests

@b-j-roberts I think I am stuck.
If you have the chance, could you take a look at this, maybe I am missing something. There seems to be a problem with how the stack is being populated, and my tests are failing for the OP_ROT and OP_2ROT


feat: add alias for metadata cascade policy create

Fixes #458

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.002%) to 86.142%

Totals

Change from base Build<PHONE_NUMBER>:
0.002%

Covered Lines:
3980

Relevant Lines:
4476

💛  - Coveralls


[FLINK-25674] Add drop tables to be idempotent in case of retrials and the related test

This is to avoid the flakiness of the CassandraConnectorITCase#test*Pojo* tests due to retrials that try to recreate an already created (and potentially populated) table. This is the quick version that still uses Junit4 and RetryRule to quickly fix the Cassandra flakiness issue. The version that uses RetryExtension needs a migration of cassandra tests to junit5 and some parent tests are in flink-streaming and mutualized so it would be done in a next PR.
I also added a commit to replace all string literals by constants to avoid existing copy/paste.
R: @tillrohrmann
CC: @zentol

Thanks @zentol for quick review/merge. Time to monitor flakiness now :smile:


Remove TemplateHaskellQuotes from baseDynFlags

HSE is no longer used, so THQ is no longer needed.

Always nice to simplify! Thanks


Finalize Toolchain for project via Bitrise

This was done pre-project utilizing the Bitrise continuous integration for the 3rd year Mobile Dev assignment.
Small adjustments will need to be made.

Completed https://app.bitrise.io/app/bb27aba9fb1b6066#/builds


Feature request: Display metadata values from folder and global configured metadata

The dotnet/docs repo uses folder level metadata to simplify the authoring experience.
One downside of this is that it can be hard for newcomers to know the inherited values used. It would be a great feature to be able to see what values would be added to any article from the global and folder configuration.

I could help with this one! The metadata-controller.ts already has the constructs in place to fully resolve these values and they're used for the implicit metadata value update command.
What kind of behavior would you expect exactly, @BillWagner? Would you expect a right-click context menu item on a file -- and some sort of popup?

@BillWagner Is the existing Update Implicit Metadata Values feature sufficient?


eval,parser: allow padding fields in structs

Fixes neugram/ng#173.

Codecov Report

Merging #182 into master will increase coverage by 0.14%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##           master     #182      +/-   ##
==========================================
+ Coverage   50.38%   50.52%   +0.14%
==========================================
  Files          23       23
  Lines        9974     9985      +11
==========================================
+ Hits         5025     5045      +20
+ Misses       4448     4437      -11
- Partials      501      503       +2

Impacted Files
Coverage Δ

gengo/gengo.go
78% <100%> (+0.04%)
:arrow_up:

parser/parser.go
70.14% <60%> (-0.03%)
:arrow_down:

parser/scanner.go
69.97% <0%> (+2.43%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dcf621c...268971e. Read the comment docs.


How to configure programmatically git credentials for Jenkinsfile multibranch?

Hi,
I would like to know if there is some easy way to configure "global" git credentials (user:password) from the docker image to be used with the Jenkinsfile multibranch feature.
With Jenkins 1.6x I used an ugly .netrc file with credentials on the container, and it always works with Jenkins 2.x for legacy jobs, but when I'm creating the following Jenkinsfile in my project, I have a hudson.plugins.git.GitException: org.eclipse.jgit.api.errors.TransportException: https://xxx.git: not authorized
Do you have any idea to fix that properly?
Jenkinsfile:
#!groovy

node {
    catchError {

        stage 'Checkout'

        checkout scm
        sh 'git submodule update --init;'

        stage 'Build and deploy'

        sh 'make HOME=/tmp/backend deploy-registry'
    }
    step([$class: 'Mailer', notifyEveryUnstableBuild: true, sendToIndividuals: true, recipients<EMAIL_ADDRESS> }

PS: For now I've created the credentials by hand

That's probably because the git calls are now using the git java library, not the git command
You could create the credentials an the save the config file and the file containing the encryption password to the image, but it is a bad idea to save credentials in the image


Code blocks too wide

The background of code blocks sticks out quite a bit from the rest of the text. To my eyes this is quite jarring, but I saw you have code blocks on your own blog and don't seem to mind, so maybe it's just me! I thought I'd give you the feedback anyway, but feel free to close this right away if you don't think it's a problem. Love the theme in any case!

Apologies for the 'referenced issue' spam above. Added a link to my personal Etch fork's commit msg, didn't intend for it to spam this issue thread.


Update " Industry" section under Analysis

Provide an overview of the industry, including its size, growth rate, and major players. Highlight any current trends or developments that are relevant to SILK Corp's mission and values.

Discuss the specific target audience that SILK Corp aims to serve. Emphasize the inclusivity of the organization, welcoming individuals from diverse cultures, races, ages, genders, and religions.

Highlight the focus on women and children, as they play a significant role in SILK Corp's vision of creating a peaceful community.

Analyze the competitive landscape within the industry. Identify key competitors and their positioning, strengths, and weaknesses.

I have updated the issue "Industry" section under Analysis.

Hey, Can I work on this issue ? @N-eng66

@N-eng66, please provide the references link to the SILK Guide that helps to gather the information.

@Aini-Bashir, Okay  let me add that too.

@Akshatchaube01, You can start working on this issue. If you want to ask any question you can ask it here in comments.

Okay Thanks @N-eng66

For SILK Corp's mission and values, it would be helpful to know the specific industry the company operates in, as it will allow for a more targeted analysis of trends and developments that align with their objectives.

Hey @Akshatchaube01, please visit our website at https://silkcorp.org/ to learn more about SILK Corp. Additionally, make sure to visit the official SILK Corp Guide at https://guide.silkcorp.org/, where you can find detailed information about the company and the specific industry it operates in.

@Akshatchaube01, Go check out our contribution guidelines to get paid

@Akshatchaube01, Thank you for your contribution here. It would be beneficial to highlight any current trends or developments within the industry that are relevant to SILK Corp's mission and values. Identifying and discussing such trends would demonstrate the organization's awareness of industry dynamics and its ability to adapt to emerging needs. Identifying key competitors and discussing their positioning, strengths, and weaknesses would help us understand how SILK Corp differentiates itself and its competitive advantage in the market.

https://www.joorney.com/samples/e2-business-plan-sample-hospitality-business/
Page 9
Talk about which industry SILK can be under. You can also research NAICS Codes
https://www.census.gov/naics/


trasparent button did not show text when i put it in Left side of header

when i try this :
<Header>
     <Left>
         <Button>
             <Text>Back</Text>
         </Button>
     </Left>
 </Header>

it works correctly, but when i try transparent button its not work!
<Header>
    <Left>
        <Button transparent>
            <Text>Back</Text>
        </Button>
    </Left>
</Header>

It show nothing (i can find my button when random tap but cannot see Text).
I try to set color and position or width style to text but it doesn't work.
But if use <Right></Right> instead of <Left></Left> it works correctly

Ops !
its work when i changed <Text> fontSize

I was watching your docs , In your docs it doesn't work too add fontSize to your Text 😉🤣

Thanks...

Check release notes https://github.com/GeekyAnts/NativeBase/releases/tag/v2.4.5
Header's Left component will not render Text button for Android


test breaking the e2e test

this should fail e2e-aws-operator

@smarterclayton "could not wait for pod completion" seems like the wrong error here... if you look at the build log it's clear the e2e test failed (as expected):
https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/openshift_cluster-image-registry-operator/59/pull-ci-openshift-cluster-image-registry-operator-master-e2e-aws-operator/2
--- FAIL: TestClusterImageRegistryOperator (1.13s)
	cluster_image_registry_operator_test.go:25: unexpected error: imageregistries.imageregistry.operator.openshift.io "fake-image-registry" not found
FAIL
FAIL	github.com/openshift/cluster-image-registry-operator/test/e2e	1.148s
2018/11/07 01:15:52 Container test in pod e2e-aws-operator failed, exit code 1, reason Error
2018/11/07 01:23:49 Container teardown in pod e2e-aws-operator completed successfully
2018/11/07 01:23:49 Copying artifacts from e2e-aws-operator into /logs/artifacts/e2e-aws-operator
2018/11/07 01:23:50 Copied 6.35Mi of artifacts from e2e-aws-operator to /logs/artifacts/e2e-aws-operator
2018/11/07 01:23:55 Ran for 50m2s
error: could not run steps: could not wait for pod to complete: could not wait for pod completion: the pod ci-op-dibs34hg/e2e-aws-operator failed after 46m30s (failed containers: test):  unknown

Probably regressed recently when people changed the artifacts loop.  Open a
bug to CI-operator
On Nov 6, 2018, at 8:53 PM, Ben Parees<EMAIL_ADDRESS>wrote:
@smarterclayton https://github.com/smarterclayton "could not wait for pod
completion" seems like the wrong error here... if you look at the build log
it's clear the e2e test failed (as expected):
https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/openshift_cluster-image-registry-operator/59/pull-ci-openshift-cluster-image-registry-operator-master-e2e-aws-operator/2
--- FAIL: TestClusterImageRegistryOperator (1.13s)
cluster_image_registry_operator_test.go:25: unexpected error:
imageregistries.imageregistry.operator.openshift.io
"fake-image-registry" not found
FAIL
FAIL	github.com/openshift/cluster-image-registry-operator/test/e2e	1.148s
2018/11/07 01:15:52 Container test in pod e2e-aws-operator failed,
exit code 1, reason Error
2018/11/07 01:23:49 Container teardown in pod e2e-aws-operator
completed successfully
2018/11/07 01:23:49 Copying artifacts from e2e-aws-operator into
/logs/artifacts/e2e-aws-operator
2018/11/07 01:23:50 Copied 6.35Mi of artifacts from e2e-aws-operator
to /logs/artifacts/e2e-aws-operator
2018/11/07 01:23:55 Ran for 50m2s
error: could not run steps: could not wait for pod to complete: could
not wait for pod completion: the pod ci-op-dibs34hg/e2e-aws-operator
failed after 46m30s (failed containers: test):  unknown
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openshift/cluster-image-registry-operator/pull/59#issuecomment-436475341,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABG_p_aR5yVXDMUQK13PDpes0U3yVezhks5usj0zgaJpZM4YRjsu
.


Update ci.yaml

Thanks for filing the PR. I don't think this is a desired change we would like to make. There is no default wdio config and also we would particular run the one behind npm run test:local

I'm so sorry, I was clearly working in the wron window. Really really sorry!

No worries 😉


ProcBots: Move from spaces in source to tabs

We aren't following our own TS guide. We should be.

?  https://github.com/resin-io-modules/resin-procbots/pull/5#discussion_r99109043

@lurch Sorry, I have no idea what that's meant to mean.

Connects to #169


Fixed #22938 -- Modified session expiry date to allow cleanup of file based sessions

This PR fix https://code.djangoproject.com/ticket/22938
set SESSION_ENGINE to use django.contrib.sessions.backends.file

create few sessions, wait untill they expire (you may use set_expiry, to reduce the waiting time)
try django-admin.py clearsessions directly to clean out expired sessions.

Files should be deleted from your temp dir.

Could you add a regression test?

Sure, will come up with something

Personally, I am not following the logic in get_expiry_age https://github.com/django/django/blob/master/django/contrib/sessions/backends/base.py#L208 as is different then get_expiry_date https://github.com/django/django/blob/master/django/contrib/sessions/backends/base.py#L186. Why get_expiry_age duplicates the code rather then uses something like:
self.get_expiry_date()-timezone.now()

I don't know enough use-cases for both keyword args that's why this patch is enough for our project. Perhaps that could be reviewed at some point...

buildbot, add to whitelist.

It would be nice to note this in the 1.9 release notes as well.

Closing due to inactivity. Please send a new PR if someone can update it, thanks!

I am very sorry that was left over. New PR with all the comments addressed was opened in https://github.com/django/django/pull/5375


Beautify skill grid

A few visual tweaks:

Ensure that all skill boxes in a row have the same height (flex box approach that also works with smaller screen sizes).
Use correct hover icon for footer (pointer instead of grab).

After:

@almakantara Now that #189 has been merged I update my PR accordingly.

@almakantara Now that #189 has been merged I update my PR accordingly.

ok, thanks


Add AWS RDS instance discovery

This PR adds the possibility to automatically discover AWS RDS instances and offer them in a dropdown to prefill the host and port fields.

discovery is disabled by default
region and credentials can be automatically discovered from EC2 metadata or the usualy AWS SDK config methods (~/.aws/config, env variables, ...)
in read-only mode the endpoint for RDS clusters returned is the reader endpoint

Is there an issue with some background on why this feature should be added, scoping it, and discussing config plans? It seems like a fairly massive PR to dump on a project out of the blue.

Interesting, i had a very similar use case for Heroku where i had to connect to a few databases. I made  a hacky "addon" for pgweb, but all this work is not open sourced.

@bsx how are you running pgweb? on your dev machine or on the EC2 instance ?

I'm running it on EC2 instances in multi session mode for multiple users.

@bsx Checking in, did you try the approach i described above?

Sorry, haven't had the time yet but I'll definitely check it out

@bsx Another ping, have you tried anything that i outlined in the previous posts?

I've played around with the "Connect" featuee a bit. Seems promising so far. I've noticed however that it is not possible to combine it with the "Lock Session" feature, but maybe I'm misunderstanding the session lock thing.
I'm currently thinking about a "companion" app that handles auth and connection details so the user would probably never see the connection form of pgweb itself.

@bsx @Garbee take a look at #391 PR.
Pushed out an initial feature set with 2 providers: Heroku and AWS. Technically speaking, existing bookmarks feature could be implemented as a provider since all providers are using Bookmark model to return connection details.
I find it really easy to connect to Heroku now since i don't even have to configure the API token with CLI flag, pgweb will load the token from ~/.netrc file where Heroku CLI keeps its config.
Here's how to enable the feature:
pgweb \
  --sessions \               # Turn on multi-session mode
  --discovery \              # Turn on discovery feature
  --heroku \                 # Enable Heroku provider
  --heroku-token=... \       # Heroku API token (optional) -> HEROKU_TOKEN
  --aws \                    # Enable AWS provider
  --aws-access-key=... \     # -> AWS_ACCESS_KEY
  --aws-secret-key=... \     # -> AWS_SECRET_KEY
  --aws-region=us-east-1     # -> AWS_REGION

You'd need to checkout git branch discovery to test out the feature. Just clone the repo, switch branch and run make build.
Let me know what do you think.

Ping! Would love to hear some feedback here.

Closing in favor of #391


Pr mountain car

Added a simple task, the continuous mountain car, that doesn't require Mujoco to run. It's easy and quick to run and uses a normalisation wrapper that might be useful for other environments too.

Deprecated by #45


Consider updating to Tomcat 9 in 1.3

I've been having trouble hotswapping code with Tomcat 9. It works well with Tomcat 8.5.


Allow arrays on NER trim between conditions

Feature Use Case
Get such syntax instead of adding a new condition object.
{
  "complete_todos": {
    "entities": [
      {
        "type": "trim",
        "name": "todos",
        "conditions": [
          {
            "type": "between",
            "from": ["check", "complete", "tick"],
            "to": ["from", "from", "from"]
          }
        ]
      },
      {
        "type": "trim",
        "name": "list",
        "conditions": [
          {
            "type": "between",
            "from": ["the", "my"],
            "to": ["list", "list"]
          }
        ]
      }
    ]
  }
}

Feature Proposal
Iterate over from and to properties when facing NER between conditions while injecting trim entities.

 ✨ [1.0.0-beta.5] Allow array of elements for trim conditions

Done in https://github.com/leon-ai/leon/commit/7cf7f9791254e1950fe9128ce1b3a58079cc2ada.


kosmos shell editor

use http://www.npcole.com/npyscreen/ to starting from a schema do an autoform which allows people to edit the object.
Only for the the main object if complex type with subobjects.
how to call
j.clients.ssh.kds.ui()

➤ kristof de spiegeleer commented:
make sure to use the type info well of the schema so we can be smart to build the form
also support enumerators


Made a lot of revisions that'll benefit the group moving forward. Continue auth

Description
Please include a summary of the change and a link to which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[ ] Test A
[ ] Test B

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] My code has been reviewed by at least one peer
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

The changes will allow us to work with a much lesser chance of merge errors.


Sorter can not handle obejts e.g. dates when grouping by dates

It should be possible to also group by objets e.g. dates:
/**
		 * Returns a group object, at least containing a key property for group detection.
		 * May contain additional properties as provided by a custom group function.
		 *
		 * @param {sap.ui.model.Context} oContext the binding context
		 * @return {object} An object containing a key property and optional custom properties
		 * @public
		 */
		getGroup : function(oContext) {
			var oGroup = this.fnGroup(oContext);
			if (typeof oGroup === "string" || typeof oGroup === "number" || typeof oGroup === "boolean" || oGroup == null) {
				oGroup = {
					key: oGroup
				};
			}
...

I will provide a pull request to fix this issue

Hi ReinhardF,
thanks for reporting this and also thanks for providing a fix.
I informed the responsible developer, they will keep you updated.
Internal incident id:1580139846
Kind regards,
Janina

Hello ReinhardF,
the "object" type has intentionally been omitted, as objects most objects serialize as "[object Object]" and  you rather want to group by a specific property of an object, than the object itself. Please use a group-function returning the required key for grouping on objects.
Regards,
Malte


Cast per_page and page params to_i

Sometimes users will send params that are strings and not integers. This solves that.

Hey, thanks for the patch! Quick question: is the thing you're trying to avoid something like a user requesting /endpoint?page=hello&per_page=world? In my mind that should be a 400 BAD REQUEST as opposed to simply trying to cast those to integers.
Also, with regards to the hash syntax update… I do plan to do this, but I'd rather not break Ruby 1.8 compatibility until the next major version update.

@mikeastock Ping

So I actually ended up not needing this. But whenever you decided to break Ruby 1.8 compatibility I would love to help you cleanup syntax.

Great, thank you!


login username issue

I'm allowed to sign in with holybasil rather than Holybasil while my username of LeetCode is Holybasil

Hmm, looks wired. Will you be blocked if you sign in with Holybasil?

@jdneo Yes, but it is fine now😂


TabView: Separate border lines into being everywhere except the selected tab

The single border line that ran underneath the selected tab doesn't work when the selected tab color is partly transparent. This change breaks it up to run underneath all of the pieces of TabView separately -- which is a lot of places -- except for the selected tab.


Update the right python doc link for typing.Annotated

The current link is pointing to typing.Literal which seems to be a typo. please review

Change Summary

Related issue number

Checklist

[x] The pull request title is a good summary of the changes - it will be used in the changelog
[ ] Unit tests for the changes exist
[ ] Tests pass on CI
[x] Documentation reflects the changes where applicable
[ ] My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Thanks @gowthamgts


update to pydantic 2

Related Issue(s):

#593

Description:
This PR updates dependencies to Pydantic 2 and Stac-Pydantic 3
It removes the internal Stac, Search and Opertator Types in favor of Types from Stac Pydantic
PR Checklist:

[x] pre-commit hooks pass locally
[x] Tests pass (run make test)
[x] Documentation has been updated to reflect changes, if applicable, and docs build successfully (run make docs)
[x] Changes are added to the CHANGELOG.

@thomas-maschler , moving the STAC types away from the TypeDict implementations in stac_fastapi.types to Pydantic types could have performance implications. Specifically I'm worried about the STAC types encoding Client output. These were originally moved away from Pydantic models as they tended to be slow for large payloads, e.g. returning an ItemCollection from search that contains 1000 large Sentinel 2 Items. Running Pydantic validation against known valid Items that come out of PgSTAC as part of response causes additional latency without a lot of value. I know the Pydantic 2 backend changes means validation will be faster, but are we introducing the potential for increased latency in the API routes because of the move away from TypedDicts in e.g. BaseCoreClient in this PR? If so, could the upgrade be done for but leave the TypeDict usage as is? Or have you seen that these changes don't introduce additional latency due to the magic of Rust 🪄?

I can see your concerns regarding latency. I did not benchmark this myself. Others reported a speed improvement of x5 between pydantic 1 and 2. I was assuming that this would be enough to address latency concerns.
The benefits I saw in keeping the pydandic models are peace of mind, due to basic checks and setting of default values you wouldn't have to store in the DB. I have a use case, where data are stored in a legacy system and I have to translate them to STAC at runtime. Being able to push part of this into Pydantic makes the job easier.
If latency is still the main concern I could see several routes

Keep TypedDicts
Be explicit in the documentation that responses are not type checked and it is the responsibility of the use to do so.

Allow to skip validation for ItemCollections and Collections

from pydantic import SkipValidation
from stac_pydantic import api
from typing import Sequence

class ItemCollectionUnsafe(api.ItemCollection):
    features: Sequence[SkipValidation[api.Item]]

class CollectionsUnsafe(api.Collections):
    collections: Sequence[SkipValidation[api.Collection]]

With minimal code changes, users can use the Unsafe subclasses as their return model, telling pydantic to skip validation for items and collections when making bulk requests.

Add some configuration options to let the user choose between current implementation options 1 or 2 above.

Couple of notes from my pull request:

If typing List and Dict are being used this part will need to be changed as in it's current form it will miss off the Optional typing of nested Lists & Dicts.
The types.links seems to be covered by stac-pydantic links and link_factory.
The stac-pydantic Item/Collection/ItemCollection types are not json serialisable so will need to be json encoded/converted to dictionaries before they're returned from the API.

@rhysrevans3, regarding your first point, can you give me an example of when this fails? I cannot replicate it. I tried this, where the filter attribute is defined as Optional[Dict]

@pytest.mark.parametrize(
    "filter,passes",
    [(None, True), ({"test": "test"}, True), ("test==test", False), ([], False)],
)
def test_create_post_request_model(filter, passes):
    extensions = [FilterExtension()]
    request_model = create_post_request_model(extensions, BaseSearchPostRequest)

    if not passes:
        with pytest.raises(ValidationError):
            model = request_model(filter=filter)
    else:
        model = request_model(
            collections=["test1", "test2"],
            ids=["test1", "test2"],
            bbox=[0, 0, 1, 1],
            datetime="2020-01-01T00:00:00Z",
            limit=10,
            filter=filter,
            **{"filter-crs": "epsg:4326", "filter-lang": "cql2-text"},
        )

        assert model.collections == ["test1", "test2"]
        assert model.filter_crs == "epsg:4326"
        assert model.filter == filter

@thomas-maschler understood about the value of validation in the response classes for other use cases.
I think a solution where users could skip validation based on configuration would be great. In my previous testing (a year or so ago) it was the Pydantic validation that was causing slowdowns. I'd still want to validate that there's no other performance hits besides validation for using Pydantic models over TypeDicts before integrating these changes, but I have enough confidence that this would address the performance implications, and any unforeseen issues could be addressed in follow up work.
Would you be willing to add that SkipValidation configuration to this PR?

about performances, in TiPg we also went for the TypedDict way because pydantic validation was really slowing down the response (even with pydantic 2.0)

@thomas-maschler I found it was an issue for nested List/Dicts types: for example the sortby field in the SortExtensionPostRequest is
sortby: Optional[List[PostSortModel]]

But when I check the FieldInfo shows required=True
>>> from stac_fastapi.extensions.core.sort.request import SortExtensionPostRequest
>>>SortExtensionPostRequest.model_fields['sortby']
FieldInfo(annotation=Union[List[SortExtension], NoneType], required=True)

@rhysrevans3 this is b/c we didn't set a default value. Right now the model requires either None or a list of PostSortModel. If you set the default to None the FieldInfo will say required=False. However, None is filled in when sortby is not provided so it still works even without a default value.
Regarding your point 2.
The type.Links are no Pydantic Models and are not used anywhere in the package. I prefer to leave them in right now to not break too much. I am not sure if any of the backends currently rely on it.
Regarding your point 3.
All pydantic models are json serializable. either with .model_dump_json() (output as str) or .model_dump(mode="json") (output as dict). I added some test, directly returning the pydantic Item, Collection and ItemCollection and this works for me.
I am still working on the switch between TypeDict and Pydantic response models and will probably push my updates later today.

@thomas-maschler ah that makes more sense thank you for the explanation. I guess this just need to be filled in the backend I'm using.
Point 2 that sounds like a good idea. But don't the links in the api.landing_page used here need to be stac-pydantic Links? I guess with the validation removed this won't be an issue.
Point 3 sorry yes I meant json.dumps or orjson won't serialise them which I think fastapi uses for returning JSON/GEOJSON content. So was more of a note that the other repos will need to serialise or convert to a dict before returning the api routes. I think this effect the landing pages. however, I might be missing something obvious.

@lossyrob can you take a look at my recent changes, in particular 8118f10.

In types.config I added a parameter validate_response that defaults to False (same as now).
I added types.stac back in
I added types.response_model which implements the switch
In api.core I made sure we always use response_model classes and added some extra type-checking
I added additional tests to make sure this works correctly, in particular, test_response_model and test_app

I stumbled over some other bugs regarding field aliases in the filter extension that I didn't manage to fix. I will file an issue for that.
I also noticed that there are still some pydantic extension models that are either out of sync with stac-pydantic or not present in stac-pydantic. I will port them over and open another PR once they make their way through the system.

@rhysrevans3
regarding your point 3: please look at test_app.py in this PR. Serialization works just fine

@thomas-maschler sorry about that I've been getting TypeError: Object of type LandingPage is not JSON serializable on the landing page when trying to update the stac-fastapi-elasticsearch-opensearch to use this branch and assumed it was because of the switch to api.LandingPage type but this is my first time using pydantic models. I'll keep digging. Thanks for all your help!

I haven't looked at the backends. We use a custom one.
Try importing types.response_model.LandingPage and see if it works now.
If you want to use the Pydantic models set the envvar VALIDATE_RESPONSE=True

Hi everyone, I'm going to take over the review.
@thomas-maschler did you had anything to add or is this PR ready?

Thanks, this is ready. The most controversial part is probably the response model

I added a comment about adding something in the readme explaining the option to choose the response model.

@lossyrob can you take a look at my recent changes, in particular 8118f10.

In types.config I added a parameter validate_response that defaults to False (same as now).
I added types.stac back in
I added types.response_model which implements the switch
In api.core I made sure we always use response_model classes and added some extra type-checking
I added additional tests to make sure this works correctly, in particular, test_response_model and test_app

I stumbled over some other bugs regarding field aliases in the filter extension that I didn't manage to fix. I will file an issue for that. I also noticed that there are still some pydantic extension models that are either out of sync with stac-pydantic or not present in stac-pydantic. I will port them over and open another PR once they make their way through the system.

Thanks for this change, this for sure satisfies the concerns I brought up.
Looking forward to this making its way into a release soon!

Thanks a ton @thomas-maschler
This PR is massive and has many unrelated changes to pydantic update but it's too late to refactor so we'll just try to merge this.
I've posted some comment directly in the files themself, but wanted to ask one general question.
I see that we now have an option for the Model to be either TypeDict or Pydantic Model. This model is used  at two levels (if I understand well):

between the Backend and the Client (https://github.com/thomas-maschler/stac-fastapi/blob/pydantic2/stac_fastapi/types/stac_fastapi/types/stac.py#L490)

between the Client and the User (via FastAPI response) (https://github.com/thomas-maschler/stac-fastapi/blob/pydantic2/stac_fastapi/api/stac_fastapi/api/app.py#L238-L240)

Question:

I wonder if there is a real need to the Pydantic Model to be used at level 1, we could still get validation error when FastAPI would try to validate the response?
in https://github.com/thomas-maschler/stac-fastapi/blob/e24351ad2c182a790128d11423cc05870df7dd9f/stac_fastapi/api/stac_fastapi/api/app.py#L238-L240
if self.settings.enable_response_models is set to False we won't have OpenAPI schema hints for the response. In TiPG we overcome this by passing the model directly to the OpenAPI like https://github.com/developmentseed/tipg/blob/main/tipg/factory.py#L688 . Do you think we could do the same here (we can do it in another PR)

Oh, I somehow missed out on option 2. That makes things a lot clearer now.
Let's go with that approach only. It is less convoluted than what I did.
I will revert my introduced changes and make sure FastAPI uses the correct models.
Let me also see how I can inject the model to OpenAPI

ok, I got this reverted but I don't get my tests to pass. Am I doing something wrong?
Even if I set enable_response_models to True I get a 200 response with an invalid item.

ok, I got this reverted but I don't get my tests to pass. Am I doing something wrong?
Even if I set enable_response_models to True I get a 200 response with an invalid item.

🤯 this is so weird!

import pytest

from pydantic import BaseModel
from fastapi import FastAPI
from fastapi.exceptions import ResponseValidationError
from fastapi.testclient import TestClient

class model(BaseModel):
    value: int

def test_model():
    app = FastAPI()

    # should return 200
    @app.get(
        "/yo",
        response_model=model,
        responses={
            200: {
                "content": {
                    "application/json": {},
                },
                "model": model,
            },
        },
    )
    def yo():
        return {"value": 1}

    # response model
    # so validation should fail
    @app.get(
        "/ye",
        response_model=model,
        responses={
            200: {
                "content": {
                    "application/json": {},
                },
                "model": model,
            },
        },
    )
    def ye():
        return {"value": "b"}

    # No response model
    # so No validation
    # should return 200
    @app.get(
        "/ya",
        responses={
            200: {
                "content": {
                    "application/json": {},
                },
                "model": model,
            },
        },
    )
    def ya():
        return {"value": "b"}

    # SAME With `add_api_route` method
    def yi():
        return {"value": "b"}

    app.router.add_api_route(
        name="yi",
        path="/yi",
        response_model=model,
        responses={
            200: {
                "content": {
                    "application/json": {},
                },
                "model": model,
            },
        },
        methods=["GET"],
        endpoint=yi,
    )

    with TestClient(app) as client:
        resp = client.get("/yo")
        assert resp.status_code == 200

        with pytest.raises(ResponseValidationError):
            client.get("/ye")

        resp = client.get("/ya")
        assert resp.status_code == 200

        with pytest.raises(ResponseValidationError):
            client.get("/yi")

I've tried to reproduce this with a simple APP but without success 😬 , ☝ pass the tests

ahhhhhh it's because the endpoint response get's automatically wrapped in a Response so it passthrough the validation 😭
https://github.com/tiangolo/fastapi/blob/7e161b3f9e7ae5b6a37ab8ea13b6dd2c1c11eff9/fastapi/routing.py#L281-L284
so 👇 won't raise an issue
    @app.get(
        "/yu",
        response_model=model,
        responses={
            200: {
                "content": {
                    "application/json": {},
                },
                "model": model,
            },
        },
    )
    def yu():
        return JSONResponse({"value": "b"})

I'm pretty we can remove this
https://github.com/thomas-maschler/stac-fastapi/blob/d0762eb20dfba437376c9a8879524cac0ee9ce7e/stac_fastapi/api/stac_fastapi/api/routes.py#L26
because FastAPI will automatically do it, I'll make a PR on top of your PR

Phew, that was a beast. It kind of grew out of hand. Thanks for the help.
One thing left is updating the documentation. I believe, that once I turn that stone, new things will pop up and I prefer to make this a separate PR.

thanks a lot @thomas-maschler.
One thing that I want to do before we merge is make sure the performance is 👍, I'll try to set up some benchmark over the weekend

Do you think this issue is going to be fixed once this PR is merged?

@drnextgis Hey sorry for not seeing your ticket before, but yes it will be fixed. I stumbled on this to https://github.com/thomas-maschler/stac-fastapi/pull/2
I'll fix also for ~2.4 version of stac-fastapi

----------------------------------- benchmark 'Items': 7 tests -----------------------------------
Name (time in ms)           Min                Max               Mean             Median
--------------------------------------------------------------------------------------------------
Items Limit: (1)         1.0452 (1.0)       2.1031 (1.0)       1.1329 (1.0)       1.1137 (1.0)
Items Limit: (10)        1.7471 (1.67)      3.0766 (1.46)      1.8286 (1.61)      1.8087 (1.62)
Items Limit: (50)        4.5001 (4.31)     64.0863 (30.47)     4.9263 (4.35)      4.6144 (4.14)
Items Limit: (100)       8.0186 (7.67)     64.3674 (30.61)     8.6762 (7.66)      8.1762 (7.34)
Items Limit: (200)      14.8327 (14.19)    17.0791 (8.12)     15.1425 (13.37)    15.0344 (13.50)
Items Limit: (250)      18.1781 (17.39)    58.4482 (27.79)    19.1771 (16.93)    18.4300 (16.55)
Items Limit: (1000)     69.1144 (66.13)    69.9298 (33.25)    69.6401 (61.47)    69.6901 (62.58)
--------------------------------------------------------------------------------------------------

-------------------------------- benchmark 'Items With Model validation': 7 tests --------------------------------
Name (time in ms)                          Min                 Max               Mean             Median
------------------------------------------------------------------------------------------------------------------
Items With Model validation (1)         1.1480 (1.0)        2.3425 (1.0)       1.2314 (1.0)       1.2174 (1.0)
Items With Model validation (10)        1.7948 (1.56)       2.8828 (1.23)      1.9552 (1.59)      1.9323 (1.59)
Items With Model validation (50)        4.1826 (3.64)      38.1875 (16.30)     4.7179 (3.83)      4.2844 (3.52)
Items With Model validation (100)       7.2982 (6.36)      43.5653 (18.60)     8.6554 (7.03)      7.4836 (6.15)
Items With Model validation (200)      12.8544 (11.20)     47.9423 (20.47)    17.3208 (14.07)    13.2296 (10.87)
Items With Model validation (250)      16.0176 (13.95)     53.0070 (22.63)    21.2084 (17.22)    16.2596 (13.36)
Items With Model validation (1000)     61.4995 (53.57)    106.9911 (45.67)    86.3199 (70.10)    99.1915 (81.48)
------------------------------------------------------------------------------------------------------------------

Well, either the validation/serialization is not working or there is no difference 🤯

@vincentsarago I thought we had results here that seemed to make sense?

I believe we now test for correct validation.

I believe we now test for correct validation.

@thomas-maschler Yes

@vincentsarago I thought we had results https://github.com/stac-utils/stac-fastapi/pull/650#issuecomment-2042918120 that seemed to make sense?

@jonhealy1 absolutely, so this mean that validation does not impact the performance 🤯. I would still keep this optional but it might be great to mention this somewhere

Ok, I'll update the readme

thanks @thomas-maschler I was going update this branch tomorrow morning 🙏
let see if we can get this merged before the end of the week 😄

@vincentsarago when you say wait do you mean wait to merge?

@jonhealy1 sorry I meant, wait for the 3.0 release :-)
I'll merge this now 🥳

🤯 😍


Feature Request: modifier key to diable animations

First of all, thanks for the module!
Would it be possible to implement an option so that GMs can set a modifier key to disable token animations? This would be especially useful when the GM wants to move a token without revealing fog of war.

Added in 1.0.5

I don't know if this is the intention, but the animation is still played for the players.
This sort of defeats the purpose of OP:s request. Since the player will still get fog of war revealed if the GM holds the modifing key.

Fixed again in 1.1.0


[VS 2019] ModelBuilderPackage did not load correctly:  Could not load file or assembly "Microsoft.VisualStudio.Threading, Version=<IP_ADDRESS>"

Model Builder Version: 16.9.2
Visual Studion Version: 16.9
Bug description
Cannot load the Model Builder package. I cannot add a new Model Builder file or open a .mbconfig file. When I try to open a file I see the following error message:

The error in the ActivityLog.xaml shows:
  <entry>
    <record>587</record>
    <time>2021/12/08 17:24:11.319</time>
    <type>Error</type>
    <source>VisualStudio</source>
    <description>SetSite failed for package [ModelBuilderPackage]Source: 'Microsoft.ML.ModelBuilder'
    Description: Could not load file or assembly 'Microsoft.VisualStudio.Threading, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.Threading, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
File name: 'Microsoft.VisualStudio.Threading, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'

    at Microsoft.ML.ModelBuilder.ModelBuilderPackage.InitializeAsync(CancellationToken cancellationToken, IProgress`1 progress)
   at Microsoft.VisualStudio.Shell.AsyncPackage.<>c__DisplayClass20_0.<<Microsoft-VisualStudio-Shell-Interop-IAsyncLoadablePackageInitialize-Initialize>b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Threading.JoinableTask.<JoinAsync>d__70.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.VisualStudio.Services.VsTask.RethrowException(AggregateException e)
   at Microsoft.VisualStudio.Services.VsTask.InternalGetResult(Boolean ignoreUIThreadCheck)
   at Microsoft.VisualStudio.Services.VsTask.GetResult()

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].
</description>
    <guid>{A1473B8C-F40E-4509-8A84-66BBA12B3E7F}</guid>

80070002

    <errorinfo></errorinfo>
  </entry>

TO FIX: Upgrade VS 2019 to version 16.11 or newer.
The problem here is that in this PR we upgraded the VSSDK to version 16.10.31321.278 to use new color tokens. SDK version 16.10 is only available in VS version 16.10 and newer. The problem is fixed if using the newest version of VS 2019 - 16.11.
Of the ~2,200 people using the latest MB version with VS 2019, only 126 are still on version 16.9 and would see this problem on upgrade.

Model Builder team, we need to add a min version for 2019 extension install.

Install only available for 16.10 or newer:

Even with update to minor version, we are still seeing an SDK problem (not related to original root cause).
Description: Could not load file or assembly 'Microsoft.VisualStudio.Shell.15.0, Version=<IP_ADDRESS>

not repro this issue on latest main: <IP_ADDRESS>6901

ML.Net Model Builder: <IP_ADDRESS>6901
Microsoft Visual Studio Enterprise 2019: 16.11.9
can add new model builder or load existing one:


Looking for collaborators to help update and maintain Bad Ideas

I haven't touched this repository since I gave my KubeCon talk on it, mainly because the KCP project seemed to overlap enough that it should be able to address the same goals I had in mind for badidea.
While, I still think that KCP should be able to support the use cases I had in mind, it seems like there is a barrier for folks to use the current state of the KCP project to fit the same use cases as I had in mind for this project. As a result, I'd like to start an effort to update badidea, this time leveraging the KCP project as a base instead of the upstream components directly.
If you are interested in helping out with these efforts, please let me know. The plan is to try and schedule an initial kickoff meeting to jump start efforts in roughly a month.

I actually appreciate the minimalistic nature of this project.
I spent a few hours to upgrade the k8s.io/... deps to v0.25.0 with some small api changes here, and it is working.
Before starting a KCP based version, can we update the current one? I'd like to submit a pull request if you want it.


feat(compare_to): Use 2 different commit messages

What does this PR do?
Use different commit message when pushing the initial branch for compare_to test and the commit that modifies the compare_to: main content
Motivation
As sometimes gitlab displays both pipelines (the initial push and the modified commit) having both with the same commit name was confusing
Describe how to test/QA your changes
Pushed a "fake commit" to generate the compare_to test
Before:

After:

Possible Drawbacks / Trade-offs
Additional Notes

/merge


Fixes for 1.0

@stevengj, noticed this as an error trying to run DataFrames tests on 1.0

Thanks.  (Note that @static doesn't really accomplish anything here.)


fix(ref): fix ref prop normalization

Hey everyone!
I ran into inconsistencies with how template refs are treated between dev and prod here: https://github.com/vuejs/core/issues/12029
I have an element whose ref prop points to a non-ref object. Like this:

<template>

Hello

</template>

While this is obviously useless code, it has happened to me and several of our colleagues during development, by accident. In dev, this fails silently and everything keeps working, making it hard to spot that you did something wrong. During production, the code fails and often nukes the entire component or page. This inconsistency is quite dangerous as it causes unintended production bugs on code that worked in development.
This is NOT the same issue as in https://github.com/vuejs/core/issues/11373 and https://github.com/vuejs/core/issues/4866 - these issues are concerned about dev/prod differences when it comes to dynamically-bound refs (:ref="x").
The issue starts at the fact that prod compiles the templates differently than dev. The props object passed to the p element in the dev version is { ref: "p" }. However, in the prod version it becomes { ref_key: "p", ref: p }. This is where the problem gets injected into the framework since we're passing p as the ref. This works just fine if p is actually a Ref, but it breaks down when it isn't. Since the compiler can't differenciate these cases statically, the fix must lie in the runtime logic.
The earliest spot I was able to catch this issue in was in normalizeRef. The ref that is passed in is typed as:
type _ =
  | null
  | string
  | Ref
  | ((
      ref: Element | ComponentPublicInstance | null,
      refs: Record<string, any>,
    ) => void)

Which led me to conclude that the current code was erroneous; if ref is non-null, but is also not a string, Ref, or function, then it is simply an invalid ref and should be changed to null. Correct me if this fix is incorrect; i.e. there is some configuration in which the "fallthrough case" (where ref is used as-is, despite being no string, Ref, or function) is intended.

I don’t think this change is worthwhile.
First, this is incorrect usage. As mentioned in the documentation, developers should declare a ref to hold the element reference. This kind of incorrect usage can be caught during development because it is not an Element or a component instance when accessing p. see Playground
Secondly, this change only fixes the error in the production, but it does not make it work as expected. It also makes it harder for users to spot the issue since the error is no longer reported.

I see where you're coming from; it can be argued that since I'm using Vue in an undefined way, Vue does not owe me any guarantees and can also then behave in undefined ways.
We have run into this issue several times at work, and perhaps it may help if I show the last instance of this error I encountered to clear up the nature of the issue:

It started like this. I was rendering raw HTML in a paragraph element, but then wanted access to the raw text representation of it (so, textContent). So I created a ref to the paragraph element:

<template>

</template>

However, then I realized I didn't need to rely on Vue to do this, and simply created an off-screen paragraph element instead, which I then work with in a watchEffect:

<template>

</template>

This code now works as intended and is what we kept. My oversight here was not removing the ref="p" at the bottom (it is not needed anymore) - however, in dev, not only did my component work flawlessly, but there was also no warning in the console and no error in the typecheck or IDE.
Perhaps this clears up the usage for you: The code works as intended because p is used as it is defined in the script, and it not expected that p is overridden or set by the ref in the template below. It is simply that there is a faulty ref in the template that does nothing and is no longer needed.

If the behavior is to be kept like this (where this code can nuke an entire page in prod), then I think the error should also be equally as severe in development where it can be caught early by the developer. But since dev's current behavior is to do nothing (which I also think is fine), I argue that prod's behavior should not be more severe, since prod is a more high-stake environment. So that means prod should also do nothing, which is what the change in this PR achieves.

OK, I fully understand the situation you're facing. I think we should keep DEV and RPOD consistent.
reopen https://github.com/vuejs/core/issues/12029

during development, when setting the template ref, if the key exists on setupState but is not a ref

I agree that adding a warning for this case is definitely good as it nudges the dev to correct their mistake.
However, regardless of the issue at hand, is there anything incorrect about the code change in this PR? More specifically, could you give an example of when this branch of the code is taken and makes sense:

(excuse the screenshot)

This can happen when users manually assigns the ref property of an existing, already normalized VNode into the props of another one. It's unlikely to happen in template-based usage but is possible in projects using manually written render functions.


[IS-7.0.0] conditional authentication toggle doesn't get enable when a predefined conditional auth flow is selected

Describe the issue:
When we add a conditional login flow from Predefined flows, Conditional authentication doesn't get enabled automatically.
https://github.com/wso2/product-is/assets/25483865/ae2e1e60-c0d5-478a-82f0-f004e3afa144

How to reproduce:

Login to console
Go to Applications, create an application
Go to Login Flow tab of the created app
Select one of the predefined conditional auth flow
This will not enable the conditional authentication toggle

Version : IS-7.0.0

This has been fixed post IS7 release [1].
[1] https://github.com/wso2-enterprise/asgardeo-product/issues/22370


Don't abort on notify_on_error for poll. Instead simply schedule clos…

…ure with cancel.. Soft error

****************************************************************

libgrpc.so

     VM SIZE                                           FILE SIZE
 ++++++++++++++ GROWING                             ++++++++++++++
  +0.0%     +24 [None]                                 +320  +0.0%
  +0.4%     +48 src/core/lib/iomgr/ev_poll_posix.cc     +48  +0.4%
      +127%     +51 fd_notify_on_error                      +51  +127%

  +0.0%     +72 TOTAL                                  +368  +0.0%

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

[trickle] No significant performance differences

Objective-C binary sizes
*****************STATIC******************
  New size                      Old size
 1,976,546      Total (>)      1,976,498

 No significant differences in binary sizes

***************FRAMEWORKS****************
  New size                      Old size
10,771,320      Total (<)     10,771,321

 No significant differences in binary sizes

[microbenchmarks] No significant performance differences

Known issues : #16156, 16124

Thanks for reviewing!


2 Questions for PotatoePet/AC Team

Firstly I apologize for opening an issue for this as it's obviously not an issue. Github seems to be the place where the the dev/devs are most active so I'm asking here.

I currently do not have AC in my modpack for the sole reason that loaforcsSoundAPI and it seem to conflict. In the logs loaforcsSoundAPI says AC is the issue but I also saw a post from PotatoePet saying some sound related mod author was blaming AC when any lobby increasing size mod was causing issues however my sound replacer of choice (LethalResonance) seems to function fine with morecompany. Is the conflict fixable? Should I wait for a fix from loaforcsSoundAPI or from AC?

I would like to add a custom LC logo for my modpack. I am aware of some of the thunderstore drama but not all of it and since I have yet to use the 2 mods together I don't know how they function together in their current state. I saw some posts saying AC will close LC if the watermarkremover mod is added and other posts saying something along the lines of if the logo is changed AC will swap to a different mode and display "Powered by AC" so can I use a logo/watermark replacement mod with AC to display my modpack name? I am perfectly okay with it still saying powered by AC or something similar below the main menu logo and have no intention of removing any copyright notices

Sorry for the long read for 2 relatively straightforward questions I wanted to make myself as clear as possible.

Sound mods should work afaik
You can place a Logo.png in your plugins folder to replace the logo. If you want to name it differently you can configure this in ModPack.cfg of AC. If its part of a Modpack you could upload a Mod with just the Logo.png in it.

Firstly I apologize for opening an issue for this as it's obviously not an issue. Github seems to be the place where the the dev/devs are most active so I'm asking here.
1. I currently do not have AC in my modpack for the sole reason that loaforcsSoundAPI and it seem to conflict. In the logs loaforcsSoundAPI says AC is the issue but I also saw a post from PotatoePet saying some sound related mod author was blaming AC when any lobby increasing size mod was causing issues however my sound replacer of choice (LethalResonance) seems to function fine with morecompany. Is the conflict fixable? Should I wait for a fix from loaforcsSoundAPI or from AC?

logs for the loaforcsSoundAPI issue i dont mind reading the logs and trying to find where the error originates

You can place a Logo.png in your plugins folder to replace the logo. If you want to name it differently you can configure this in ModPack.cfg of AC. If its part of a Modpack you could upload a Mod with just the Logo.png in it.

I had no idea it was this easy I thought you had to use a mod like watermarkremover. Now I get why you dislike the mod especially if they remove your copyright. Thanks for the help!

logs for the loaforcsSoundAPI issue i dont mind reading the logs and trying to find where the error originates

I'll include my log in case you want to have a look yourself anyway but loaforcsSoundAPI points to this already made github issue


Fixed typo (Mangement)

Docs Build status updates of commit 4b715cd:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

biztalk/technical-guides/troubleshoot-biztalk-management-pack.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

@hubalazs Thank you for finding, and fixing these. We appreciate what you're doing.


README: make shorter and denser for v2.0

Signed-off-by: Hiroshi Miura<EMAIL_ADDRESS>

And for the title of this PR, it should be "make shorter and denser", not "dense".

I've updated suggested changes and also updated heading words simpler such as warning clause.


Feature/babel6 simple

A reduced change set rebased off master

That’s great, thank you very much!
The only thing I’d like to change is Lodash imports: I prefer import func from 'lodash/func' style.

Re the preference for lodash import syntax, I'd suggest that it's more in line with the direction that lodash itself is going to reference the module directly?

Modules, not packages. And now it’s very simple (in 3.x we had to write lodash/groupname/func, now it’s just lodash/func. I thinks it’s easier to maintain (you have to install just one lodash package) and only the required modules will be bundled by Webpack (so it’s as efficient as using multiple packages).

Removed all references to lodash.x packages and replaced _.pluck with _.map per discussion at https://github.com/underdash/underdash/issues/4#issuecomment-110026962


Mulit-GPU

Is there an option to run the training on multiple GPU (single node)?
I would like to make the training faster by (effectively larger batch size)?

Yes! You could find "--use_multi_gpu" and "--devices" in run_longExp.py. You just need to set those parameters for multi-gpu usage.

@yuqinie98 Thanks the multi-gpu works perfect for me now


ValueError: Catalog 'dev' does not exist in the metastore

Hey,
I am new to Databricks, and I am trying to test the mlops-stack bundle.
Within that bundle there is a feature-engineering workflow and I have a problem to make it run.
The main problem is the following.
the bundle specified the target to be $bundle.target which is in my case would be dev. I have created the dev catalog and within the project schema according to the template.
The issue is that when I run the workflow, the notebook fails at
fe.create_table(     name=output_table_name,         primary_keys=[x.strip() for x in pk_columns.split(",")] + [ts_column],  # Include timeseries column in primary_keys     timestamp_keys=[ts_column],     df=features_df, )
I am getting that:
ValueError: Catalog 'dev' does not exist in the metastore.
And I don't understand why?. If I ran the notebook manually through all purpose cluster it works.
I printed the available catalogs when I am running through workflow through bundle and I only got spark_catalog.

@bs-davidnagy can you confirm that you're using a UC-enabled cluster? And whoever you're running the workflow as (SP or yourself) has the right permissions on the catalog?

@arpitjasa-db  Thanks for the reply.
Regarding the metastore, it is UC attached.

the catalog created through the UI and for now all user has ALL_PRIVILEGES.

the the catalog and the underlying Schema details:

Currently I am using the job cluster specified in the resources. So effectively I did not changed anything on the workflow.
new_cluster: &new_cluster
  new_cluster:
    num_workers: 3
    spark_version: 15.3.x-cpu-ml-scala2.12
    node_type_id: Standard_D3_v2
    custom_tags:
      clusterSource: mlops-stacks_0.4

common_permissions: &permissions
  permissions:
    - level: CAN_VIEW
      group_name: users

resources:
  jobs:
    write_feature_table_job:
      name: ${bundle.target}-aifp_sample_project-write-feature-table-job
      job_clusters:
        - job_cluster_key: write_feature_table_job_cluster
          <<: *new_cluster
      tasks:
        - task_key: PickupFeatures
          job_cluster_key: write_feature_table_job_cluster
          notebook_task:
            notebook_path: ../feature_engineering/notebooks/GenerateAndWriteFeatures.py
            base_parameters:
              # TODO modify these arguments to reflect your setup.
              input_table_path: /databricks-datasets/nyctaxi-with-zipcodes/subsampled
              # TODO: Empty start/end dates will process the whole range. Update this as needed to process recent data.
              input_start_date: ""
              input_end_date: ""
              timestamp_column: tpep_pickup_datetime
              output_table_name: ${bundle.target}.aifp_sample_project.trip_pickup_features
              features_transform_module: pickup_features

The ${bundle.target} is dev
The job cluster created by the bundle-workflow is
{
    "cluster_id": "[REDACTED]",
    "creator_user_name": "[REDACTED]",
    "spark_context_id": "[REDACTED]",
    "driver_healthy": true,
    "cluster_name": "[REDACTED]",
    "spark_version": "15.3.x-cpu-ml-scala2.12",
    "azure_attributes": {
        "first_on_demand": 1,
        "availability": "ON_DEMAND_AZURE",
        "spot_bid_max_price": -1
    },
    "node_type_id": "Standard_D3_v2",
    "driver_node_type_id": "Standard_D3_v2",
    "custom_tags": {
        "clusterSource": "mlops-stacks_0.4"
    },
    "autotermination_minutes": 0,
    "enable_elastic_disk": true,
    "disk_spec": {},
    "cluster_source": "JOB",
    "enable_local_disk_encryption": false,
    "instance_source": {
        "node_type_id": "Standard_D3_v2"
    },
    "driver_instance_source": {
        "node_type_id": "Standard_D3_v2"
    },
    "effective_spark_version": "15.3.x-cpu-ml-scala2.12",
    "state": "TERMINATED",
    "state_message": "",
    "start_time": "[REDACTED]",
    "terminated_time": "[REDACTED]",
    "last_state_loss_time": 0,
    "last_activity_time": "[REDACTED]",
    "last_restarted_time": "[REDACTED]",
    "num_workers": 3,
    "default_tags": {
        "Vendor": "Databricks",
        "Creator": "[REDACTED]",
        "ClusterName": "[REDACTED]",
        "ClusterId": "[REDACTED]",
        "JobId": "[REDACTED]",
        "RunName": "[REDACTED]",
        "Team": "[REDACTED]",
        "CostCenter": "[REDACTED]",
        "Product": "[REDACTED]",
        "Owner": "[REDACTED]",
        "env": "Development"
    },
    "termination_reason": {
        "code": "JOB_FINISHED",
        "type": "SUCCESS"
    },
    "init_scripts_safe_mode": false,
    "spec": {
        "cluster_name": "[REDACTED]",
        "spark_version": "15.3.x-cpu-ml-scala2.12",
        "azure_attributes": {
            "availability": "ON_DEMAND_AZURE"
        },
        "node_type_id": "Standard_D3_v2",
        "custom_tags": {
            "JobId": "[REDACTED]",
            "RunName": "[REDACTED]",
            "clusterSource": "mlops-stacks_0.4"
        },
        "autotermination_minutes": 0,
        "enable_elastic_disk": true,
        "num_workers": 3
    }
}

I  don't understand why would a job cluster created through bundle resource, would have a different access to a catalog if the bundle executed by me. (So I am the user).

So I run, the notebook but also add the following command:
print(spark.sql("SELECT CURRENT_METASTORE()"))
and I get the following error:
[[OPERATION_REQUIRES_UNITY_CATALOG](https://docs.microsoft.com/azure/databricks/error-messages/error-classes#operation_requires_unity_catalog)] Operation CURRENT_METASTORE requires Unity Catalog enabled. SQLSTATE: 0AKUD
File <command-2336575342377498>, line 3
      1 spark.sql("CREATE DATABASE IF NOT EXISTS " + output_database)
----> 3 print(spark.sql("SELECT CURRENT_METASTORE()"))
File /databricks/spark/python/pyspark/errors/exceptions/captured.py:254, in capture_sql_exception.<locals>.deco(*a, **kw)
    250 converted = convert_exception(e.java_exception)
    251 if not isinstance(converted, UnknownException):
    252     # Hide where the exception came from that shows a non-Pythonic
    253     # JVM exception message.
--> 254     raise converted from None
    255 else:
    256     raise
    ```
    So I guess the cluster is not UC enabled. But I don't really understand how can I create, a non-UC Enabled cluster from a UC enabled workspace ?.

So I managed to run, I had to change the following on the cluster creation
    spark_version: 15.4.x-cpu-ml-scala2.12
    data_security_mode: "SINGLE_USER"

I don't really understand why the default cluster is custom/shared, if it unable to access the UC catalog during DBR is ML

@bs-davidnagy yeah great point, we updated our template code to include this field now, so that by default these clusters should be UC enabled.


Just only Create a USB Stick

Hi,
This is a greate Project, and i'dlike to use it for our Windows client deployment.
i do have one question, if it's possible to just only recreate the USB Stick, if no cleanup was done in a previous run. i've seen the USBImagingToolCreator.ps1 - but after execution only an empty stick was created.
do you have a hint for me?
thanks

@MKellyCBSD might be able to help with the USBImagingToolCreator.ps1 file

I have an updated version. I created a pull request #101 .

Resolved an issue with the Images folder not being copied over
A progress bar has been added. It advances in segments, so it may appear to be frozen at times, but it is functioning correctly
Added an option to disable Autoplay to prevent empty volume errors during script execution

This script has to be run from within the root folder containing the Drivers and Images folders.
This script also must be run from an Admin PowerShell console

I'll have to check this later. I initially uploaded a pull request. but then realized that that script is different than the script on the main branch. sorry for the confusion.

try the pull request I just submitted #102. Hope This helps. I tested this and it worked for me.

try the pull request I just submitted #103 to 2410.1. Hope This helps. I tested this and it worked for me.

Thanks a lot for the fast response!
yes, now it's working :)
Gerat job! Keep on developing the solution! :)


python3Packages.tensorflow: 2.4.1 -> 2.5.0

python3Packages.tensorflow: 2.4.1 -> 2.5.0

Motivation for this change
https://github.com/tensorflow/tensorflow/releases/tag/v2.5.0
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Added a release notes entry if the change is major or breaking
[ ] Fits CONTRIBUTING.md.

I need to update commit message.

If we want TF to be able to load keras weights with the new hdf5, we need TF 2.5.
We could have this working and backport it to 21.05 to resolve this issue: https://github.com/NixOS/nixpkgs/issues/124599

I've updated the sha256 hash for the source it seems to pass that problem.
But there's also the patch. The patch file for the setup.py is now different and I've updated that.
However I've noticed that TF2.5.0 also requires now keras-nightly:
     # Use keras-nightly at the moment.
     'keras-nightly ~= 2.5.0.dev',

This was not in v2.4.0, and so this is a new dependency that isn't even in Nixpkgs atm.
Also upgrading to v2.5.0 will require also updating all the binary versions too, not just the source derivation.

This will need to be packaged up as well!! https://pypi.org/project/keras-nightly/#history
Can someone else who is familiar with keras set this up?

Trying this, I'm getting
ERROR: /build/source/WORKSPACE:23:14: no such package '@com_github_grpc_grpc//src/compiler': BUILD file not found in directory 'src/compiler' of external repository @com_github_grpc_grpc. Add a BUILD file to a directory to mark it as a package. and referenced by '//external:grpc_cpp_plugin'

Disabling the TF_SYSTEM_LIBS seems to help it along for now

@yorickvP how did you get around keras-nightly?

@CMCDragonkai keras-nightly (and the other python dependencies) are not required during the build, as far as I can tell.

workspace.patch:
diff --git a/tensorflow/workspace3.bzl b/tensorflow/workspace3.bzl
index 16f7176d..0c0038b6 100644
--- a/tensorflow/workspace3.bzl
+++ b/tensorflow/workspace3.bzl
@@ -3,6 +3,14 @@
 load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

 def workspace():
+
+    http_archive(
+        name = "io_bazel_rules_docker",
+        sha256 = "1698624e878b0607052ae6131aa216d45ebb63871ec497f26c67455b34119c80",
+        strip_prefix = "rules_docker-0.15.0",
+        urls = ["https://github.com/bazelbuild/rules_docker/releases/download/v0.15.0/rules_docker-v0.15.0.tar.gz"],
+    )
+
     http_archive(
         name = "io_bazel_rules_closure",
         sha256 = "5b00383d08dd71f28503736db0500b6fb4dda47489ff5fc6bed42557c07c6ba9",

..... why did my script go forward with the merge...

Sorry for accidental merge.
There's still patch failures

What do we do?

Patch revert and continue with this MR?

I reverted the changes to master. Not sure if you can re-open a merged PR. I would just open a new one. Sorry for the inconvience.


issue #106: azure vm terraform module

Closes #106

The naming convention used inside the azure-vm module is based on what was currently done

The .tfvars is up to date with the lastest change from this PR

Re putting this PR to draft. Missing the setup to create a microsoft VM

Can we audit the bastion setup against best practices in a separate issue?
https://goteleport.com/blog/security-hardening-ssh-bastion-best-practices/


Perf: updated subscriber socket subsystem

Current dependencies on/for this PR:

main

PR #55   👈

This stack of pull requests is managed by Graphite.


Use without HSL?

Hi there, I'm curious to give these packages a try, but as far as I can see HSL is a hard dependency, and obtaining HSL requires submitting personal information into the HSL application form, which I am unwilling to do.
Is it possible to use this stack without HSL?
I see you are using an MIT license for this repo which is quite permissive, but the dependency on HSL appears to significantly limit usage.

OK great, thanks for the tip. I don't have time to try this now, hopefully if anyone gives it a go in future they can let us know here how well it works. Will close the ticket for now.


Refactor imputer components to remove unnecessary logic

Related to https://github.com/alteryx/evalml/issues/3999 (Note - will not yet close this issue, as this PR does not remove the nullable type logic; it only refactors the unnecessary logic)
Refactors the following imputers

imputer.py
simple_imputer.py - only pass columns with nans to the imputer
time_series_imputer.py - only pass columns with nans to the imputer
target_imputer.py

No linked issues found. Please add the corresponding issues in the pull request description.  Use GitHub automation to close the issue when a PR is merged

I believe the codecov failures are just because enough code was removed that coverage has lowered, but if you go into the codecov report, none of the changes show uncovered lines.

LGTM! great work 😄


Drop support for "use math"

Oh, I broke something and 262 found it! I'll take a look later.

This one is ready @bnoordhuis !

Don't worry about it, it happens.


List Component :: Can we modify the "no result" message

Hi,
is it possible to customize the "no result" message or provide different outputs for the list component?
Currently, I get the message "Keine Suchergebnisse gefunden ... ", but in my case, this is not correct. I have no data to fill the list initially (from api) and the current "no results" message implies that a filter or search has been used.
It would be helpful if we could have the option to change this message manually or we can switch to different messages for different scenarios (e. g. no initial data, no result after filtering, error from server or call api, ....)

Hey,
Instead of displaying an empty list, you can replace it with an IllustratedMessage. We’re already doing this in our mStudio. The IllustratedMessage has the advantage of filling the available space effectively and, with the help of the CTA (Call to Action), prompts the user to create a first item.
You can also find this under “List” and then “Guidelines,” in the “IllustratedMessage” section. We’re currently working on making documentation sections linkable, but here’s a screenshot of the section I mean:

Best regards Jan

ok, is fine for me - thanks

Thank you for your question!


gen/db/package.json deployer files not matching root package.json settings

I am attempting to set non-default values in the package.json migrations section to deploy to a common schema instead of public.
Each time I build using npx cds-dbm build the default values listed in project readme are written the the /gen/db/package.json file and therefore causing an scehma incorrect deployment.
screencast of issue:
https://www.screenpresso.com/=Annjc

Thanks for providing this really detailed video.
The issue here is, that the build module  doesn't respect the values from the root package.json, but instead uses the default values taken from the template.
I will look into this on the weekend and also update to the latest versions of the other npm packages.

Fixed in<EMAIL_ADDRESS>


chore(changelog): add missing note about TS 2.2

Fixes #4512

Also update issue template to ask for TypeScript version


How to run NASM Assembly on Windows 10 Insider 14332?

I've tried to run this simple assembly program on Windows 10 Insider Build with the Ubuntu Bash. However I always get the Error message: "bash: ./hello: cannot execute binary file: Exec format error".
So I've run said file on Ubuntu VM where it works without problems. Both operating systems are 64 bit. It turns out Windows can compile the code but not run it.
I use nasm -f elf32 hello.asm, gcc -m32 hello.o -o hello and ./hello as commands.
I appreciate any help.
section .text
global main
main:
mov eax, 4 ;system call number (sys_write)
mov ebx, 1 ;file descriptor (stdout)
mov ecx, mymsg ;message to write
mov edx, mylen ;message length
int 80h ;call kernel
mov eax, 1 ;system call number (sys_exit)
mov ebx, 0 ;exit with error code 0
int 80h ;call kernel
section .data
mymsg db 'Hello World!', 0xa
mylen equ $-mymsg

Currently WSL only supports amd64 binaries (elf64).  This is being tracked by #228 and #390 so I'm going to go ahead and close this out as a duplicate.
Thanks for trying out WSL!


[WIP][Tokenomics] Application Reimbursement Request

Summary
Implement Global Mint Reimbursement Request.
Specific changes:

Introduced the EventApplicationReimbursementRequest event
Added the TokenLogicModuleGlobalMintReimbursementRequest  TLM

Issue

#732
Related to #750
Superceed #740

Type of change
Select one or more:

[ ] New feature, functionality or library
[ ] Bug fix
[ ] Code health or cleanup
[x] Documentation
[ ] Other (specify)

Testing

[ ] Unit Tests: make go_develop_and_test
[ ] LocalNet E2E Tests: make test_e2e
[ ] DevNet E2E Tests: Add the devnet-test-e2e label to the PR.

Sanity Checklist

[ ] I have tested my changes using the available tooling
[x] I have commented my code
[x] I have performed a self-review of my own code; both comments & source code
[x] I create and reference any new tickets, if applicable
[x] I have left TODOs throughout the codebase, if applicable

@red-0ne I'm closing this PR but please open a new one with the work you're doing


Recipe led tests are picking up release candidate branch runs

Recipe led tests should be using main recipes.
Run:
https://luci-milo.appspot.com/raw/build/logs.chromium.org/flutter/led/flutter-try-builder_chops-service-accounts.iam.gserviceaccount.com/3f4abbc5ad760beeeb355cc19253868bf6ce986705a127c5a1de41bf74f53fbb/+/build.proto?server=chromium-swarm.appspot.com

@godofredoc What command(s) did you use to run this? Anything ususual outside of the regular led get-builder?

This was triggered by led recipe testing:

https://cs.opensource.google/flutter/recipes/+/main:recipes/recipes.py
https://cs.opensource.google/flutter/recipes/+/main:recipe_modules/recipe_testing/

Ah, so this is run on any recipes CL?

Yes, that is correct.

To summarize the problem - When opening a CL for recipes, the presubmit will look for any modified recipes, take the last green build, and replicate it using the modified recipe to make sure it is still passing. However, in some cases, the last green build is not on the same branch as the recipes CL is merging into, causing problems. In the case above, the last green build was on flutter-3.7-candidate.1, not main/master.
We need to change the testing to search for the last green build that is on the same branch as the CL is merging into.

Related: https://github.com/flutter/flutter/issues/120287

Waiting for release engineering to take a look at the pull request to fix this.
https://github.com/flutter/flutter/pull/120305 - fix pr

Finishing up a CL for this: need to write tests

^ This has been merged!

Closing, feel free to reopen


Add support for virtualized (xen) platforms (i.e. amazon ec2)

Prevent problems similar to issue 316
[31029.489919]   ***************************************************************
[31029.489922]   ***************************************************************
[31029.489924]   ** WARNING: Currently emulating unsupported memory accesses  **
[31029.489927]   **          in /lib/tls glibc libraries. The emulation is    **
[31029.489930]   **          slow. To ensure full performance you should      **
[31029.489932]   **          install a 'xen-friendly' (nosegneg) version of   **
[31029.489935]   **          the library, or disable tls support by executing **
[31029.489937]   **          the following as root:                           **
[31029.489940]   **          mv /lib/tls /lib/tls.disabled                    **
[31029.489942]   ** Offending process: zzzzzzzz (pid=16229)                   **
[31029.489945]   ***************************************************************
[31029.489947]   ***************************************************************

Thanks for this patch. However I read this from http://cs.felk.cvut.cz/~vaclam1/linux/ucebna/k310/data/doc/howto-xen-and-gentoo-en.html:
"Some software, in particular the glibc TLS library, is implemented in a way that will conflict with how Xen uses segment registers to circumvent a limitation of 32-bit x86 hardware platforms, causing poor performance whilst carrying out certain operations under Xen. This will result in a ~50% performance penalty running multi-threaded applications. To fix this, you must compile your system with the '-mno-tls-direct-seg-refs' flag."
So this flag is only required on x86? You should add an extra check so that this flag is only enabled for x86 systems.
And before we can merge your changes we need you to sign the contributor agreement at http://www.phusion.nl/forms/contributor_agreement
Could you follow the instructions there?

Added check to require option only on 32bit x86 Xen hosts
32ba4e21711cd8ba60cd6d28e9f1cd01ec490418
Contributor agreement signed

You should check for the file first and then cpu_architectures because checking for the file is the most inexpensive operation.

Added 0a5f1b01478f65f127f69f9a7f5f593b104fce94
but I think "supports_sfence_instruction?" called by "portability_cflags" already memoized "cpu_architectures".

Thanks, I've merged it now.


Multi proto file support

This PR introduces multiple proto files support in the form of directory uploads, as well as some refactoring in order to increase the test coverage of existing and new logic.
It is a fairly large PR in terms of file- and line-count, many of which are unit tests. Here is a general breakdown of the various aspects of this PR.
New database model
ProtoDirectory is a new database model that has been created, and it behaves in a manner similar to RequestGroup.
Proto file management business logic
Previously, much of the proto file management business logic was written within ProtoFilesModal. This is tricky to test because logic is intertwined within a React component. The business logic was extracted to network/grpc/proto-manager. Function were then added for directories addDirectory, deleteDirectory, as required by this feature. All of this logic is now unit tested, and the modal component greatly simplified. (In the future, this will also allow for adding the proto files and directories list alongside certificates, in workspace settings).
UI
Changes to the UI are primarily in the proto files list, which now need to show indents, read-only states and directories. This was achieved through extending the ListGroupItem component with indent/hover/select state logic (available in the storybook for this PR).
A generic ProtoListItem component, extends ListGroupItem and sets a common height and icon font size, specificly for the proto file management view.
Lastly, ProtoFileListItem and ProtoDirectoryListItem both extend ProtoListItem and include specific logic for each view, indenting and setting read-only states accordingly. (Only individual proto files at the root level can be renamed/reuploaded/deleted, and only directories at the root level can be deleted).
This is the overall component hierarchy of the proto file list items as described above:

ListGroupItem (in insomnia-components)

ProtoListItem (in insomnia-app)

ProtoFileListItem
ProtoDirectoryListItem

Upload a directory
When ingesting a directory, we follow a recursive algorithm.
For a given directory,

create a ProtoDirectory entity under the parent,
load entries from the directory on the filesystem,
if entry is a file, and has the .proto extension, create a ProtoFile entity with the file name and contents,
if entry is a directory, go to step 1 and set the current directory as the next parent

This algorithm starts with the workspace being the first parent.
A step also exists to sanity check the data. For example, if a directory is loaded but no .proto file exists in it or in a sub-directory, then that directory is of no use to us, so the database entry is deleted, keeping the database clean. There is a unit test in ingest-proto-directory.test.js using a fixture containing a sub-directory like this.
Parsing a proto file
When parsing a proto file for making requests, we must treat nested and individual files slightly differently. If a proto file has been selected, which is part of a directory tree, we must write the entire tree to the file-system, in order for the proto loader to successfully import relative paths within the proto file.
We do this by determining whether the selected proto file has any ancestor of type ProtoDirectory in the database. If so, we find that root directory, and then find all descendants of type ProtoDirectory or ProtoFile, and recursively write the tree onto the filesystem.
An individual file is written under <tempDir>/<id>.<modified>.proto to ensure we don't write more than necessary, but always have the latest contents as well. For directories, we follow a similar pattern, however the files and folders should keep their original names. Therefore, we use <tempDir>/<id>.<modified>/folder/root.proto, where the id and modified time are properties of the root folder.
Screenshots
Nothing uploaded

Individual file & directory uploaded
The directory uploaded exists at packages/insomnia-app/app/network/grpc/__fixtures__/library in this PR

root.proto selected

time.proto selected

hello.proto selected

It works! 🎉

Known limitations

100% sync & import/export support INS-399
Re-uploading a directory and not breaking existing requests (where possible) INS-400
The UX for viewing directories and their proto-files in the proto file modal is a bit lackluster, but this is an MVP as no updated designs exist for this area (that I know of).

Closes #2955

@sonicyeti I quite like the first option! Only concern is that right now there's no expand/collapse function in there, so could it be misleading? Could also use the folder icon (consistent with the sidebar)

@sonicyeti I quite like the first option! Only concern is that right now there's no expand/collapse function in there, so could it be misleading? Could also use the folder icon (consistent with the sidebar)

When importing a large directory that takes a while, after re-opening the modal and pressing import directory again, nothing happens, it'll just keep spinning (presumably until finished, the directory I was testing with takes tens of minutes to load, so I didn't verify), not sure if that should be handled differently?
You can also delete the new directory while it's still importing, and the spinner will keep running, not sure what state the DB ends up in there, if there's potential issues?

Yeah this is a great point; we can disable UI interactions but in order to keep the state in between closing and re-opening the modal, we'd need move the loading state into redux (or context), which seems like a more involved fix.
I'll look to address the second issue (deleting while adding), but will likely not fix the first one (large directory) right now. 🤔

When importing a large directory that takes a while, after re-opening the modal and pressing import directory again, nothing happens, it'll just keep spinning (presumably until finished, the directory I was testing with takes tens of minutes to load, so I didn't verify), not sure if that should be handled differently?
You can also delete the new directory while it's still importing, and the spinner will keep running, not sure what state the DB ends up in there, if there's potential issues?

Yeah this is a great point; we can disable UI interactions but in order to keep the state in between closing and re-opening the modal, we'd need move the loading state into redux (or context), which seems like a more involved fix.
I'll look to address the second issue (deleting while adding), but will likely not fix the first one (large directory) right now. 🤔

@sonicyeti I have made the styling changes in #3023

@sonicyeti I have made the styling changes in #3023


Fix authentication for users with non-ascii chars in their names

ldap uses base64 encoding iff there are non-ascii chars.
For some reason the query seems to then return the user_group_name
(src/authm_mad/remotes/ldap/authenticate) as ascii-encoded.
This is only relevant when a group is set in ldap_auth.conf, in which
case ldap.is_in_group is called.
By forcing the encoding to utf-8 the authentication succeeds.
Before this error was encountered:
Exception raised authenticating to LDAP
(UTF-8 regexp with ASCII-8BIT string)>
/usr/lib/ruby/vendor_ruby/net/ldap/filter.rb:755:in scan' /usr/lib/ruby/vendor_ruby/net/ldap/filter.rb:755:in parse_filter_branch'
/usr/lib/ruby/vendor_ruby/net/ldap/filter.rb:737:in
parse_paren_expression' /usr/lib/ruby/vendor_ruby/net/ldap/filter.rb:682:in parse'
/usr/lib/ruby/vendor_ruby/net/ldap/filter.rb:673:in initialize' /usr/lib/ruby/vendor_ruby/net/ldap/filter.rb:667:in new'
/usr/lib/ruby/vendor_ruby/net/ldap/filter.rb:667:in parse' /usr/lib/ruby/vendor_ruby/net/ldap/filter.rb:341:in construct'
/usr/lib/ruby/vendor_ruby/net/ldap/connection.rb:322:in search' /usr/lib/ruby/vendor_ruby/net/ldap.rb:753:in block (2 levels) in
search'
/usr/lib/ruby/vendor_ruby/net/ldap.rb:1228:in use_connection' /usr/lib/ruby/vendor_ruby/net/ldap.rb:752:in block in search'
/usr/lib/ruby/vendor_ruby/net/ldap/instrumentation.rb:19:in instrument' /usr/lib/ruby/vendor_ruby/net/ldap.rb:751:in search'
/usr/lib/one/ruby/opennebula/ldap_auth.rb:163:in is_in_group?' authenticate:85:in block in '
authenticate:59:in each' authenticate:59:in '
Could not authenticate user tavestbo

Thanks!


Add support for delta-source-fat-* directives

Closes #201

@jeanparpaillon I rebased, updated it to work with xdelta bounds checking code that was added, and removed some code that wasn't needed. It's in main now. Thank you very much for submitting this. My plan is to look at a couple more fwup updates this week and then make a release. When you get a chance, could you check that main continues to work for you with my changes?


WIP: Add support for comment preview on client side via showdown.js

Comes along with TeamNewPipe/website#64.

Isso changes to display a preview of the comments.

Add Markdown info icon

Sorry, that PR was meant to be to an other repository.