Styling on storybook sidebar in mWeb is broken

Details

Add a CSS to change sidebar background
Add ../assets/css to serve staticly
Call fonts.css from ../assets/css to make font work

Fixed Issues

$ https://github.com/Expensify/App/issues/7842
Tests

Run storybook and verify the font base is GTAmericaExp-Regular
Open storybook from mobile web and click sidebar navigation, verify the background is changed and the texts are visible

[x] Verify that no errors appear in the JS console

PR Review Checklist

Contributor (PR Author) Checklist

[x] I verified the PR has a small number of commits behind main
[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I clearly indicated the environment tests should be run in (Staging vs Production)

[x] I wrote testing steps that cover success & fail scenarios (if applicable)
[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests & veryfy they passed on:

[ ] iOS / native
[ ] Android / native
[x] iOS / Safari
[x] Android / Chrome
[x] MacOS / Chrome
[ ] MacOS / Desktop

[x] I verified there are no console errors related to changes in this PR
[x] I followed proper code patterns (see Reviewing the code)

[x] I added comments when the code was not self explanatory
[x] I put all copy / text shown in the product in all src/languages/* files (if applicable)
[x] I followed proper naming convention for platform-specific files (if applicable)
[x] I followed style guidelines (in Styling.md) for all style edits I made
[x] I followed the JSDocs style guidelines (in STYLE.md)

[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I corroborated the UI performance was not affected (the performance is the same than main branch)
[x] If I created a new component I verified that a similar component doesn't exist in the codebase

PR Reviewer Checklist

[x] I verified the PR has a small number of commits behind main
[x] I verified the correct issue is linked in the ### Fixed Issues section above
[x] I verified testing steps are clear and they cover the changes made in this PR

[x] I verified the testing environment is mentioned in the test steps

[x] I verified testing steps cover success & fail scenarios (if applicable)
[x] I checked that screenshots or videos are included for tests on all platforms
[x] I verified tests pass on all platforms & I tested again on:

[ ] iOS / native
[ ] Android / native
[x] iOS / Safari
[x] Android / Chrome
[x] MacOS / Chrome
[ ] MacOS / Desktop

[x] I verified there are no console errors related to changes in this PR
[x] I verified proper code patterns were followed (see Reviewing the code)

[x] I verified comments were added when the code was not self explanatory
[x] I verified any copy / text shown in the product was added in all src/languages/* files (if applicable)
[x] I verified proper naming convention for platform-specific files was followed (if applicable)
[x] I verified style guidelines were followed
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] I verified that this PR follows the guidelines as stated in the Review Guidelines
[x] I verified other components are not impacted by changes in this PR (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified the UI performance was not affected (the performance is the same than main branch)
[x] If a new component is created I verified that a similar component doesn't exist in the codebase

QA Steps

Go to https://staging.expensify.com/docs/index.html and verify the font base is GTAmericaExp-Regular (and not something obviously different like Times New Roman)
Go to https://staging.expensify.com/docs/index.html from mobile web and click sidebar navigation, verify the background is changed and the texts are visible

[x] Verify that no errors appear in the JS console

Screenshots

Web

Safari

Chrome

Mobile Web

Desktop

iOS

Android

@mollfpr I updated your QA steps a bit. Since QA happens in our staging environment, it doesn't really make sense to "Run storybook". Storybook is hosted at /docs/index.html, so I added a link to clarify that.

@roryabraham thank you for clearing things up!

Whoops, I can't merge this because we require all git commits to be signed. This is pretty easy to do and there's lots of documentation out there for how to set up commit signing. Once you have commit signing working, you can retroactively sign all these commits by:

git rebase -i HEAD~2
For each commit, enter this in vim: :%substitute/^pick/reword/ then save and quit.
Force-push your changes in this PR.

Here is one set of instructions you could follow: https://webdevstudios.com/2020/05/26/retroactively-sign-git-commits/

@roryabraham Done!

@roryabraham Sorry I rebase the commit again. Turn out my gitconfig that signingkey not setup properly.

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by @roryabraham in version: 1.1.42-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

Updates link to test

🚀 Deployed to production by @chiragsalian in version: 1.1.42-6 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Added function to get an error category for better error handling.

Groundwork to be able to categorize an error so that a error message with 'who to blame' instructions can be displayed by clients.

@freerdp-bot test


Multiple svelte updates:

Multiple svelte updates:

Merge svelte package.json with root package.json.
Move svelete code to be built/watched using webpack instead of rollup.
Remove svelete unneeded files like .prettierrc, yarn.lock, etc.
Update root to properly lint svelte source.

Closes #709
Closes #710

+1, looks good overall. Perhaps later we can merge together the webpack.config.js together and just have a single top-level one.

This might be possible but also possibly not because the way things are done to built the svelte stuff is slightly different that we do for the extension so that would have to looked into more.

No worries, this is already a nice consolidation.  If the webpack consolidation comes at the expense of a bunch of additional complexity, then it's not worth it.


FINERACT-2125: implement server side rate limit on Login Panel

Fineract has a reported security vulnerability, because it does not limit login attempts, effectively allowing brute force attacks on the login screen. These changes enforce a configurable limitation of allowed requests on certain API requests (currently applied for the login request only).
Note that the enforcement happens in each JVM separately, so for clustered deployment each node enforces limits separately. That should be considered when determining the configuration value on critical deployments.
Description
Describe the changes made and why they were made.
Ignore if these details are present on the associated Apache Fineract JIRA ticket.
Checklist
Please make sure these boxes are checked before submitting your pull request - thanks!

[X] Write the commit message as per https://github.com/apache/fineract/#pull-requests

[X] Acknowledge that we will not review PRs that are not passing the build ("green") - it is your responsibility to get a proposed PR to pass the build, not primarily the project's maintainers.

[X] Create/update unit or integration tests for verifying the changes made.

[X] Follow coding conventions at https://cwiki.apache.org/confluence/display/FINERACT/Coding+Conventions.

[X] Add required Swagger annotation and update API documentation at fineract-provider/src/main/resources/static/legacy-docs/apiLive.htm with details of any API changes

[X] Submission is not a "code dump".  (Large changes can be made "in repository" via a branch.  Ask on the developer mailing list for guidance, if required.)

FYI our guidelines for code reviews are at https://cwiki.apache.org/confluence/display/FINERACT/Code+Review+Guide.

@kjozsa ... how about using Resilience4j and not implementing our own? See here how this can be done by adding one dependency, using one annotation and a handful of application.properties entries...

@vidakovic I investigated the Resilience4j path, but I'm not sure it supports the usecase of rate limiting per source address. Are you sure this would work?

@kjozsa are you sure this is the right way to approach this security vulnerability? I mean if Fineract is being put behind a CDN/LB, the remote IP address will not be the actual client address but the CDN/LB's address.
How about just enforcing Fineract to use BCrypt with a minimum 8 characters, symbols, letters (upper,lowercase) and numbers as passwords?


Exit transition not working

See the following destinations with the default animations set below. The exit transition does not run and instead the enter transition runs every time. The animations are from here.
// MainActivity.kt

@OptIn(
    ExperimentalAnimationApi::class,
    ExperimentalMaterialNavigationApi::class
)
@Composable
fun Content() {
    val engine = rememberAnimatedNavHostEngine(
        rootDefaultAnimations = RootNavGraphDefaultAnimations(
            enterTransition = { materialFadeThroughIn() },
            exitTransition = { materialFadeThroughOut() }
        )
    )

    DestinationsNavHost(navGraph = NavGraphs.root, engine = engine)
}

// HomeScreen.kt

@OptIn(ExperimentalMaterial3Api::class)
@Destination(start = true)
@Composable
fun HomeScreen(navigator: DestinationsNavigator) {
    Scaffold(topBar = { SmallTopAppBar(title = { Text(text = "Home Screen") }) }) {
        Column {
            Text(text = "Hello World!")
            ElevatedButton(onClick = { navigator.navigate(AboutScreenDestination) }) {
                Text(text = "Navigate to About Screen")
            }
        }
    }
}

// AboutScreen.kt

@OptIn(ExperimentalMaterial3Api::class)
@Destination
@Composable
fun AboutScreen(navigator: DestinationsNavigator) {
    Scaffold(topBar = {
        SmallTopAppBar(
            title = { Text(text = "About Screen") },
            navigationIcon = {
                IconButton(onClick = { navigator.popBackStack() }) {
                    Icon(Icons.Filled.ArrowBack, contentDescription = null)
                }
            })
    }) {
        Column {
            Text(text = "About Screen!")
        }
    }
}

Hi @essarn !
tbh I haven’t had much time to play with animations yet, I just exposes Accompanist apis through my own. That said, I believe you should look at popEnterTransition and popExitTransition, since when you click that back arrow button, you are popping and not exiting. Exiting should be applied to your Home Screen when you navigate from Home to About though, I can’t be sure looking at that video if it is or not.
Can you please let me know if this helps you? I’m really confident that if there is an issue it is not with Compose Destinations, since I pretty much just call Accompanist APIs.
But I am of course keen to help you to help me get to the bottom of this to be sure 🙂

Hello @raamcosta
By default, the pop transitions should be the normal ones according to the docs (https://google.github.io/accompanist/navigation-animation/#usage) so that would not be the problem.
I have a bit of a hard time understanding the order in which the transitions run. What happens when there is both an exit transition for the initial destination and an enter transition for the target destination? In my tests, it seems that the exit transition never runs.
For example; in this case, only the enter transition runs when navigating to the AboutScreen but not the exit transition when navigation back. I.e there is no fade out when navigating back.
@OptIn(ExperimentalMaterial3Api::class)
@Destination(start = true)
@Composable
fun HomeScreen(navigator: DestinationsNavigator) {
    Scaffold(topBar = { SmallTopAppBar(title = { Text(text = "Home Screen") }) }) {
        Column {
            Text(text = "Hello World!")
            ElevatedButton(onClick = { navigator.navigate(AboutScreenDestination) }) {
                Text(text = "Navigate to About Screen")
            }
        }
    }
}

@OptIn(ExperimentalMaterial3Api::class)
@Destination(style = AboutDestinationTransitions::class)
@Composable
fun AboutScreen(navigator: DestinationsNavigator) {
    Scaffold(topBar = {
        SmallTopAppBar(
            title = { Text(text = "About Screen") },
            navigationIcon = {
                IconButton(onClick = { navigator.popBackStack() }) {
                    Icon(Icons.Filled.ArrowBack, contentDescription = null)
                }
            })
    }) {
        Column {
            Text(text = "About Screen!")
        }
    }
}

@OptIn(ExperimentalAnimationApi::class)
object AboutDestinationTransitions : DestinationStyle.Animated {
    override fun AnimatedContentScope<NavBackStackEntry>.enterTransition() = materialFadeThroughIn()

    override fun AnimatedContentScope<NavBackStackEntry>.exitTransition() = fadeOut()
}

Hey again @essarn !

By default, the pop transitions should be the normal ones according to the docs (https://google.github.io/accompanist/navigation-animation/#usage) so that would not be the problem.

 I was not remembering this was the case, we can even check the constructor of RootNavGraphDefaultAnimations:
class RootNavGraphDefaultAnimations(
    override val enterTransition: DestinationEnterTransition = DestinationEnterTransition { EnterTransition.None },
    override val exitTransition: DestinationExitTransition = DestinationExitTransition { ExitTransition.None },
    override val popEnterTransition: DestinationEnterTransition = enterTransition,
    override val popExitTransition: DestinationExitTransition = exitTransition,
): NavGraphDefaultAnimationParams

And this explains why the video you posted shows the behavior where whether you are navigating into a screen or popping into one, the animation is the same: the screen "exiting" kinda translates a bit and fades and the screen "entering" translates in the other direction and fades in. At least this is what I can see. Now, I don't know what the actual animations do, but it feels correct to me, now looking at it again.
Basically what happens in your first setup is that both screens have the same enter and exit animations, so:

When you navigate from HomeScreen to AboutScreen:

HomeScreen's exit transition runs
AboutScreen's enter transition runs

When you navigate back to HomeScreen:

AboutScreen's pop exit (same as normal exit in your case) transition runs
HomeScreen's pop enter (same as normal enter in your case)  transition runs

I just tested your second setup, and the fadeOut does run, however, the HomeScreen doesn't have any animations set, so the screen immediatelly shows up above the AboutScreen and the fading doesn't actually show for a long time, but if you set a breakpoint in the fadeOut call, you'll see that it runs when you exit the AboutScreen.

Thank you for all the help @raamcosta!
What you wrote makes sense and it works now. Thanks again 😄

Glad I could help! 🙂


Building and previewing structures is broken

I think this is due to a bug in the selection code, as we're using the voxel position of the ground.
We don't want to build in the ground...

It's now previewing correctly, but always shows as "occupied". When building, we then crash.


Ingestion of data failing in case of topic have GBs of data

@subkanthi
after creating table to clickhouse, connector is not putting records to clickhouse, topic have 100+ GiB of data using debezium postgres connector for data ingestion to kafka,  have tried changing few configurations for clickhouse sink connector as below
max.request.size: 1048576
heartbeat.interval.ms: 3000
session.timeout.ms: 120000
fetch.max.bytes: 10485760
max.poll.interval.ms: 300000
max.poll.records: 100
request.timeout.ms: 300000
offset.flush.timeout.ms: 120000

clickhouse-sink-connect-cluster-connect-558964f84c-pfxf5 clickhouse-sink-connect-cluster-connect 2022-12-13 09:57:15,288 INFO ClickHouseSinkConnector() (com.altinity.clickhouse.sink.connector.ClickHouseSinkConnector) [DistributedHerder-connect-1-1]
clickhouse-sink-connect-cluster-connect-558964f84c-pfxf5 clickhouse-sink-connect-cluster-connect 2022-12-13 09:57:24,385 INFO [clickhouse-sink-connector|task-0] preCommit(task-0) 20 (com.altinity.clickhouse.sink.connector.ClickHouseSinkTask) [task-thread-clickhouse-sink-connector-0]
clickhouse-sink-connect-cluster-connect-558964f84c-pfxf5 clickhouse-sink-connect-cluster-connect 2022-12-13 09:58:24,385 INFO [clickhouse-sink-connector|task-0] preCommit(task-0) 20 (com.altinity.clickhouse.sink.connector.ClickHouseSinkTask) [task-thread-clickhouse-sink-connector-0]
clickhouse-sink-connect-cluster-connect-558964f84c-pfxf5 clickhouse-sink-connect-cluster-connect 2022-12-13 09:59:24,695 INFO [clickhouse-sink-connector|task-0] preCommit(task-0) 20 (com.altinity.clickhouse.sink.connector.ClickHouseSinkTask) [task-thread-clickhouse-sink-connector-0]
clickhouse-sink-connect-cluster-connect-558964f84c-pfxf5 clickhouse-sink-connect-cluster-connect 2022-12-13 10:00:27,190 INFO [clickhouse-sink-connector|task-0] preCommit(task-0) 20 (com.altinity.clickhouse.sink.connector.ClickHouseSinkTask) [task-thread-clickhouse-sink-connector-0]
clickhouse-sink-connect-cluster-connect-558964f84c-pfxf5 clickhouse-sink-connect-cluster-connect 2022-12-13 10:01:21,571 INFO [AdminClient clientId=adminclient-8] Node -1 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-8]
clickhouse-sink-connect-cluster-connect-558964f84c-pfxf5 clickhouse-sink-connect-cluster-connect 2022-12-13 10:01:21,580 INFO [AdminClient clientId=adminclient-8] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-8]
clickhouse-sink-connect-cluster-connect-558964f84c-pfxf5 clickhouse-sink-connect-cluster-connect 2022-12-13 10:01:30,771 INFO [clickhouse-sink-connector|task-0] preCommit(task-0) 20 (com.altinity.clickhouse.sink.connector.ClickHouseSinkTask) [task-thread-clickhouse-sink-connector-0]
clickhouse-sink-connect-cluster-connect-558964f84c-pfxf5 clickhouse-sink-connect-cluster-connect 2022-12-13 10:01:39,684 WARN [clickhouse-sink-connector|task-0] WorkerSinkTask{id=clickhouse-sink-connector-0} Commit of offsets timed out (org.apache.kafka.connect.runtime.WorkerSinkTask) [task-thread-clickhouse-sink-connector-0]
clickhouse-sink-connect-cluster-connect-558964f84c-pfxf5 clickhouse-sink-connect-cluster-connect 2022-12-13 10:02:36,610 INFO [clickhouse-sink-connector|task-0] preCommit(task-0) 20 (com.altinity.clickhouse.sink.connector.ClickHouseSinkTask) [task-thread-clickhouse-sink-connector-0]
clickhouse-sink-connect-cluster-connect-558964f84c-pfxf5 clickhouse-sink-connect-cluster-connect 2022-12-13 10:02:49,461 WARN [clickhouse-sink-connector|task-0] WorkerSinkTask{id=clickhouse-sink-connector-0} Commit of offsets timed out (org.apache.kafka.connect.runtime.WorkerSinkTask) [task-thread-clickhouse-sink-connector-0]
clickhouse-sink-connect-cluster-connect-558964f84c-pfxf5 clickhouse-sink-connect-cluster-connect 2022-12-13 10:04:02,221 INFO [clickhouse-sink-connector|task-0] [Consumer clientId=connector-consumer-clickhouse-sink-connector-0, groupId=connect-clickhouse-sink-connector] Disconnecting from node 1 due to request timeout. (org.apache.kafka.clients.NetworkClient) [kafka-coordinator-heartbeat-thread | connect-clickhouse-sink-connector]
clickhouse-sink-connect-cluster-connect-558964f84c-pfxf5 clickhouse-sink-connect-cluster-connect 2022-12-13 10:04:02,221 INFO [clickhouse-sink-connector|task-0] [Consumer clientId=connector-consumer-clickhouse-sink-connector-0, groupId=connect-clickhouse-sink-connector] Cancelled in-flight FETCH request with correlation id 1378 due to node 1 being disconnected (elapsed time since creation: 31999ms, elapsed time since send: 31999ms, request timeout: 30000ms) (org.apache.kafka.clients.NetworkClient) [kafka-coordinator-heartbeat-thread | connect-clickhouse-sink-connector]

also tried changing
    buffer.flush.time.ms: 30000
    buffer.max.records: 10000
    buffer.flush.timeout.ms: 120000

@HelloSunilSaini are you sure the topics are defined correctly in the sink connector, also do u find the topics created by debezium.
Here is the example of the debezium configuration.
configure_postgres_employees.sh
{
  "name": "test-connector",
  "config": {
    "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
    "tasks.max": 1,
    "database.hostname": "postgres",
    "database.port": 5432,
    "database.user": "postgres_user",
    "database.password": "postgres",
    "database.server.id": 5432,
    "database.server.name": "SERVER5432",
    "database.dbname": "test",
    "table.include.list": "public.Employee",
    "plugin.name": "pgoutput",

    "snapshot.mode": "initial_only",
    "snapshot.locking.mode": "minimal",
    "snapshot.delay.ms": 10000,
    "include.schema.changes":"true",

    "database.history.kafka.bootstrap.servers": "${KAFKA_BOOTSTRAP_SERVERS}",
    "database.history.kafka.topic": "${KAFKA_TOPIC}",

    "key.converter": "io.apicurio.registry.utils.converter.AvroConverter",
    "value.converter": "io.apicurio.registry.utils.converter.AvroConverter",

    "key.converter.apicurio.registry.url": "http://schemaregistry:8080/apis/registry/v2",
    "key.converter.apicurio.registry.auto-register": "true",
    "key.converter.apicurio.registry.find-latest": "true",

    "value.converter.apicurio.registry.url": "http://schemaregistry:8080/apis/registry/v2",
    "value.converter.apicurio.registry.auto-register": "true",
    "value.converter.apicurio.registry.find-latest": "true"
  }
}

Yes using similar kind of configuration and topics are present in kafka created by debezium and data also present in kafka in my case tables are also getting created in clickhouse and for few topics which have less data, data also flushed to clickhouse i am facing this issue only for topics which have data in GBs
For below configurations using default values
    "snapshot.mode": "initial_only",
    "snapshot.locking.mode": "minimal",
    "snapshot.delay.ms": 10000,
    "include.schema.changes":"true"

@subkanthi I have already tested this flow with another postgresql database which have lesser data trying this now with a database which have 1000+ GBs data and here i am facing this issue

@subkanthi I have already tested this flow with another postgresql database which have lesser data trying this now with a database which have 1000+ GBs data and here i am facing this issue

Interesting, do u actually see data in the topics?

#4 looks like related

@subkanthi I have already tested this flow with another postgresql database which have lesser data trying this now with a database which have 1000+ GBs data and here i am facing this issue

Interesting, do u actually see data in the topics?

Yes

We are not using buffer.max.records and buffer.flush.timeout.ms as the part which checks these is commented out
https://github.com/Altinity/clickhouse-sink-connector/blob/613269cc33252502657a15ac89fa9d79c873a425/src/main/java/com/altinity/clickhouse/sink/connector/executor/ClickHouseBatchRunnable.java#L219

We have noticed that something similar, increasing the memory limits helped us.
JAVA_OPTS="-Xms1G -Xmx5G"

Hi @HelloSunilSaini, any updates on this?
have you solved this issue?


Error upon clicking information icon in most validation entries

 Issue by evansiroky
Tuesday May 15, 2018 at 04:33 GMT
Originally opened as https://github.com/catalogueglobal/datatools-ui/issues/149

In most feed version validation issue detail items, clicking on the information icon will result in an internal javascript error.
Observed in at least the following validation types:

"Required table empty"
"Service unused"
"Date no service"

 Comment by landonreed
Tuesday May 15, 2018 at 18:01 GMT

@evansiroky, I believe this commit should fix this issue: 5e05a5a3c8b76e8c96d918832257c362fdff3342


Update intro section so it's less wordy

Intro section is too long - need to make it less wordy

Fixed in 24d725c55e954c3e94dba3f48547649f11401dc4


A Python Stack Trace Error Occurs if the cdk-assets NPM Package is out of date

Subject of the issue
If the installed cdk-assets NPM package is out of date, the CDK handler errors with a full Python stack trace.
Your environment

sceptre-cdk-handler v2.0.0

Expected Behaviour
The handler should return a managed error like:
[2023-03-03 17:09:21] - dev/cdk - Launching Stack
[2023-03-03 17:09:22] - dev/cdk - Stack is in the PENDING state
[2023-03-03 17:09:22] - dev/cdk - Creating Stack
sceptre-example@ C:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\sceptre-example
└── (empty)

C:\Program Files\nodejs -> .\
└──<EMAIL_ADDRESS>
[2023-03-03 17:09:50] - Publishing CDK assets
[2023-03-03 17:09:50] - Error: Canot read asset manifest 'C:\Users\Simon\AppData\Local\Temp\cdk.outNA6wTy\CDKStack.assets.json': Cloud assembly schema version mismatch: Maximum schema version supported is 22.0.0, but found 30.1.0. The cdk-assets package is probably out of date.

Actual behaviour
[2023-03-03 17:09:21] - dev/cdk - Launching Stack
[2023-03-03 17:09:22] - dev/cdk - Stack is in the PENDING state
[2023-03-03 17:09:22] - dev/cdk - Creating Stack
sceptre-example@ C:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\sceptre-example
└── (empty)

C:\Program Files\nodejs -> .\
└──<EMAIL_ADDRESS>
[2023-03-03 17:09:50] - Publishing CDK assets
Error: Canot read asset manifest 'C:\Users\Simon\AppData\Local\Temp\cdk.outNA6wTy\CDKStack.assets.json': Cloud assembly schema version mismatch: Maximum schema version supported is 22.0.0, but found 30.1.0
    at AssetManifest.fromFile (C:\Users\Simon\AppData\Roaming\nvm\v18.12.1\node_modules\cdk-assets\lib\asset-manifest.js:24:19)
    at AssetManifest.fromPath (C:\Users\Simon\AppData\Roaming\nvm\v18.12.1\node_modules\cdk-assets\lib\asset-manifest.js:43:30)
    at Object.publish (C:\Users\Simon\AppData\Roaming\nvm\v18.12.1\node_modules\cdk-assets\bin\publish.js:7:40)
    at C:\Users\Simon\AppData\Roaming\nvm\v18.12.1\node_modules\cdk-assets\bin\cdk-assets.js:32:25
    at Object.handler (C:\Users\Simon\AppData\Roaming\nvm\v18.12.1\node_modules\cdk-assets\bin\cdk-assets.js:56:15)
    at Object.runCommand (C:\Users\Simon\AppData\Roaming\nvm\v18.12.1\node_modules\cdk-assets\node_modules\yargs\build\index.cjs:446:48)
    at Object.parseArgs [as _parseArgs] (C:\Users\Simon\AppData\Roaming\nvm\v18.12.1\node_modules\cdk-assets\node_modules\yargs\build\index.cjs:2697:57)
    at Object.get [as argv] (C:\Users\Simon\AppData\Roaming\nvm\v18.12.1\node_modules\cdk-assets\node_modules\yargs\build\index.cjs:2651:25)
    at main (C:\Users\Simon\AppData\Roaming\nvm\v18.12.1\node_modules\cdk-assets\bin\cdk-assets.js:43:9)
    at Object.<anonymous> (C:\Users\Simon\AppData\Roaming\nvm\v18.12.1\node_modules\cdk-assets\bin\cdk-assets.js:59:1)
Traceback (most recent call last):
  File "C:\Users\Simon\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Simon\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\Scripts\sceptre.exe\__main__.py", line 7, in <module>
    sys.exit(cli())
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\click\core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\click\decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre\cli\helpers.py", line 43, in decorated
    return func(*args, **kwargs)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre\cli\launch.py", line 62, in launch_command
    exit_code = launcher.launch(prune)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre\cli\launch.py", line 105, in launch
    code = code or self._deploy(deploy_plan)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre\cli\launch.py", line 197, in _deploy
    result = deploy_plan.launch()
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre\plan\plan.py", line 170, in launch
    return self._execute(*args)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre\plan\plan.py", line 29, in wrapped
    return func(self, *args, **kwargs)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre\plan\plan.py", line 55, in _execute
    return executor.execute(*args)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre\plan\executor.py", line 52, in execute
    stack, status = future.result()
  File "C:\Users\Simon\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\_base.py", line 439, in result
    return self.__get_result()
  File "C:\Users\Simon\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\_base.py", line 391, in __get_result
    raise self._exception
  File "C:\Users\Simon\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre\plan\executor.py", line 59, in _execute
    result = getattr(actions, self.command)(*args)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre\hooks\__init__.py", line 107, in decorated
    response = func(self, *args, **kwargs)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre\plan\actions.py", line 213, in launch
    status = self.create()
    response = func(self, *args, **kwargs)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre\plan\actions.py", line 91, in create
    create_stack_kwargs.update(self.stack.template.get_boto_call_parameter())
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre\template.py", line 224, in get_boto_call_parameter
    return {"TemplateBody": self.body}
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre\template.py", line 97, in body
    body = handler.handle()
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre_cdk_handler\cdk.py", line 230, in handle
    template_dict = builder.build_template(context, self.sceptre_user_data)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre_cdk_handler\cdk_builder.py", line 141, in build_template
    self._publish_artifacts(manifest_artifact.file, environment_variables)
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre_cdk_handler\cdk_builder.py", line 56, in _publish_artifacts
    self._run_command(
  File "c:\Users\Simon\Documents\GitHub\Sceptre\sceptre-cdk-handler\.venv\lib\site-packages\sceptre_cdk_handler\cdk_builder.py", line 64, in _run_command
    result = self._subprocess_run(
  File "C:\Users\Simon\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'npx cdk-assets -v publish --path C:\Users\Simon\AppData\Local\Temp\cdk.outNA6wTy\CDKStack.assets.json' returned non-zero exit status 1.

The challenge with providing the sort of error messaging you're requesting here is that the originating error message has been raised in Node resulting in a nonzero exit code. As far as the Python code knows, its subprocess has returned a nonzero. That's about it.
The only way to be more specific would be to read the output of the node stacktrace, parse it (somehow) or search for certain strings in it, and then reraise an error. I don't think it's appropriate for this handler (and definitely isn't maintainable) for us to keep trying to check for all the possible errors that Node/CDK/CDK-assets might raise and reraise specific errors accordingly.
As to wanting to bury the stacktraces, I also don't think that's a good idea. Since we're invoking a subprocess here, there's a number of issues that might take place and, without that stacktrace, it'd be very difficult to diagnose what's going on here. With this stacktrace info, you can see the EXACT command that's failing and even retry it directly for triage. Furthermore, with the Node stacktrace that's pushed out above it, you can even see what was going on within Node. All of that is invaluable information for diagnosing any number of issues that might arise when using this handler.

Actually, I thought a bit more about what you're saying here. The only part of the Python stacktrace that's actually useful is the message, not the traceback itself. I've updated this in my PR #10 in this commit: https://github.com/Sceptre/sceptre-cdk-handler/pull/10/commits/00efe562fc3b493a3706145428348d5eefdaa41f. We won't hide the JS stacktrace (because that's actually the really helpful bit), but it won't include the Python stacktrace.
If Sceptre is run with the --debug flag, though, it should still show the full stacktrace, which would be helpful.


Important qube template update 1.0.4 released!

A new release of the main template in qube has just been released. This automated pull-request attempts to apply the relevant updates to this Project.
Please make sure to merge this pull-request as soon as possible. Once complete, make a new minor release of your Project.
For more information on the actual changes, read the latest qube changelog.

See
https://github.com/qbicsoftware/qube-cli/issues/227


update readme and add requirements.txt

Attempting to check out the latest changes I was unable to run with python ./blinka.py (I can find the error message if that is expected to work, perhaps I was in wrong directory or something).
I saw the comment about entry points and found that using python -m blinka to run commands does work for me.
There were two pip requirements I had to install as well to get it running, I've added those to a requirements file.

Per the (poorly formatted) README:
## Installation Details

pip install pywin32 (only on Windows systems)
pip install semver
pip install pyserial
pip install dirsync
pip install freetype-py
pip install boto3

This should probably be the list for requirements.txt (not sure what to do about pywin32.)
As for the running instructions, yeah, those are probably out of date. On my machine I did a pip3 install -e . so that gave me a top-level blinka that I use. None of this I documented though.

Cleaned up the rest including the platform conditional for pywin32 in 676bbc4dac2b5d96a82c2b245c5235d37fe297e0


[Server] Remove permissions from current session / API token, when user is edited and got fewer roles

Given token is generated with A, B, C permissions
And user permissions are changed to only A, B
Then existing token should be limited to A, B

This actually needs to be rechecked, possibly it was already implemented.


Ability to delete songs

I'm sure this is going to be added later so this is just a reminder.

This is already listed on the project board.
https://github.com/lolPants/beatsaver-reloaded/projects/1#card-22613795
Please do not report missing features as stated in #1


"Block this site" overlay "Translate this page"

Expected Behavior
Buttons display side by side.
Actual Behavior
They're overlapped.
Steps to Reproduce the Problem

Use English as the default language of Google Search.
Search for "最新 バージョン Android"

Specifications

OS: macOS Catalina
Browser: Google Chrome 81
Other extensions that might interfere: Not sure

@PoiScript
Sorry for my late reply & thank you for your bug report.
I finally reproduced this 'overlay' bug on macOS Catalina.
I never found it on Windows 10, so it may be platform-specific.
It will be fixed in the next version.

@iorate
Faced this problem several months ago. Now failed to reproduce this bug. There is a feeling that this periodically happens due to a poor Internet connection.
OS: Windows 10 (x64)
Browser: Mozilla Firefox

@Fikafusik
This bug should be fixed in v3.8.0. Please give it a try. (Currently under review)


Problem with automatically purge images from an Azure container registry

I'm trying to run scripts from this article.
I've prepared test repositories, little changed first example and try to run it:
az acr login --name myacrname $PURGE_CMD="mcr.microsoft.com/acr/acr-cli:0.1 purge \ --registry {{.Run.Registry}} --filter 'hello-world:.*'" az acr run \ --cmd "$PURGE_CMD" --registry myacrname --debug
The script is executing, it comes to "Packing source code into tar to upload ..." after that freezes and then tells "Access is denied: '\Documents and Settings'". Ok, I'm creating shortcut for "Documents and Settings". Run. Then it tells "Access is denied: '\$Recycle.Bin\S-1-5-21-705238614-490741011-620655208-39416\$RCEIG1Q'". Hmm.. Ok, I'm clearing $Recycle.Bin. Run. Now it tells me "Permission denied: '\hiberfil.sys'"
What am I doing wrong?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 40e52f8e-a63e-3e85-6c84-f881db5d3b6c
Version Independent ID: 1290c892-ea9d-d12c-be84-7ed75f28fca0
Content: Automatically remove image resources in Azure Container Registry
Content Source: articles/container-registry/container-registry-auto-purge.md
Service: container-registry
GitHub Login: @dlepow
Microsoft Alias: danlep

@Boombasteg Thanks for the feedback. We are investigating into the issue and will update you shortly.

Try to run in Azure Cloud Shell:

@Boombasteg - Thanks for your feedback. I notice you are running the command in a PowerShell environment. The examples in the article are formatted for a Bash shell, so need a little adaptation to run in PowerShell. A note should be added to the article.
So in your Cloud Shell example, the error is due to the '\' character which is used as a line break in Bash. This error can be resolved by removing the '\' line break, or substituting the PowerShell backtick '`'.
However, once you fix that, you'll see errors because of missing parameters in the purge command (needs an --ago parameter) and az acr run command (needs a source context). For example, I found the following worked in a PowerShell shell:
az acr run  --cmd "mcr.microsoft.com/acr/acr-cli:0.1 purge --registry {{.Run.Registry}}  --filter 'hello-world:.*' --ago 1d" --registry myregistry /dev/null

Please let us know if this helps? Thanks

Thank you Dan, your example is work!


sea-orm-cli passwordless database user causes "No password was found in the database url" error

when you try to generate an entity using the sea-orm-cli generate entity --database-url "mysql://root@localhost/<dbname>" command it panics with the error thread 'main' panicked at 'No password was found in the database url'
is there some other way to log into a database user without a password?

Hey @jokil123, could you try mysql://root:@localhost/<dbname>?

mysql://root:@localhost/<dbname> will return the same error.
mysql://root: @localhost/<dbname> throws error returned from database: 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)

As a temporary workaround, I can confirm a random password works for Postgres (like postgres://Matt:anythingworks@localhost:5432/my_db). According to @jokil123 though, it doesn't seem to work for MySQL.

Thanks! @Madoshakalaka
It has been fixed on latest master.
As a temporary solution, you can install the fixed version of sea-orm-cli by running
cargo install sea-orm-cli --git https://github.com/SeaQL/sea-orm.git


Optimization of GAIA's execution engine

Hi @bmmcq, is there any further work related to this issue?

if yes, please add detailed description or list to-do items in the description.
otherwise, please close this issue.

GAIAX is on the way.


The behavior of ctrl+f was changed in June_2016

VSCode: Version: 1.3.0
OS Version: Windows 7

Steps to Reproduce:

Select text (multiline)
ctrl + f or ctrl + h

Previously, it enables find in selection by default.
But it disables find in selection by default since June_2016 update.
Could I revive the previous behavior?
I could not find the setting by searching in the default setting file by find, search or select.
My 2 cents, the new behavior is inconvenient for me.

Find in selection is now available under this button:

With the support for multi-line search added in June, the auto-seeding of the search scope from the editor selection on find field focus was causing a lot of issues where the search scope was constantly seeded from multi-line search results.
I can look into adding an option to add the ability to get the old behaviour back.

Thank you very much.
I'm looking forward to 2016_July update.
For what it's worth:

Visual Studio 2013 has the behavior as same as the previous behavior of VSCode.
Sublime Text has this option (auto_find_in_selection) and it has the support for multi-line search.

:+1: I'm thinking to differentiate between selections caused by a find result vs manual "hand-crafted" user selections. If the selection was caused by jumping to the next find match (even if it is multiline) it would not auto-set the search scope. If the selection was created explicitly, and then the focus goes into the find input, then the search scope gets auto-set.

The fix is:
when the find field receives focus the search scope gets seeded (same as before) only if the editor selection is not the current find match (new condition, added to accommodate multiline find)

This issue seems to appear again in October 2016 (version 1.7).

@mysticatea This was independently of me changed by @roblourens in 79bc7a814603d541502a604e0f984aee7abba90f . Please bring up your points with the other project members that took this decision in #12109.


How to customize container image references when using Operators/OLM?

Type of question
Are you asking about community best practices, how to implement a specific feature, or about general context and help around the operator-sdk?
This is a general context question relating to the use of Helm-based Operators and OLM.
Question
I have a use case to deploy applications in various 3rd party airgapped environments. This is defined as one without Internet connectivity, but K8s clusters nodes do have access to a container registry and an internal private network. I have create a set of Helm based Operators from existing Helm charts and would like to use OLM to manage their lifecycle.  However, I've build these images referencing my own private container registry, so OLM catalog image, image references within the CRs/Helm charts all point to my repo.  Here is an example of such usage:
image: my-private-registry.com/app1
tag: v1

If I want a user to download the catalog image and install my Operator on their airgapped setup, I must first push all container images to their repo (catalog, operator, etc).
My question is how then do I adjust the container image references programmatically to point to this new customer image registry so that the user can successfully install the catalog, operator and application?
The main focus is that whatever environment I install my Operators into, airgapped or not, they will never have access to my-private-registry.com/ and I will always need to adjust where the images are being pulled from.  To date via searching I have not figured out a good programmatic approach.  Any help is appreciated!
** Additional Context**
These applications (containers, Helm charts or operators) will never be publicly accessible, meaning that we will not be publishing them on the Web for Internet based consumption. Thus the original container image references will be to wherever my build system is (my-private-registry.com) and we assume we will not know the destination image registry until installation time.

/assign

@mmellin does helm not provide utilities to manage chart values?
Otherwise have you looked into kustomize? You can define variables and apply them to a set of base manifests. I don't think this plays nicely with Helm charts though.

Hi @estroz, thanks for your reply.
We are currently using Helm charts and yes we have no trouble here.  However with OLM the manifests no longer use Helm to install, and furthermore, are "baked" into the containers (such as CatalogSource in the bundle/CSV).  Image references are statically included within a CSV for an operator. This and other manifests are included in a "bundle" and then built into a CatalogSource container.
So I would need to update the CR for the CatalogSource (not too hard), update the CSVs located within the CatalogSource container (hard - focus of this question), and update the CRs related to the apps (medium-hard) if I wanted to point the docker pull for all containers to another location.
I was hoping that OLM may have included a way to do this.  It seems the assumption is that all your containers are located somewhere publicly reachable and the image references will never be changed.  If this is the working assumption, then there are use-cases for which OLM is not a good fit.
I have not looked into kustomize but will since you have suggested it.

@mmellin thanks for clarifying, I didn’t fully understand the issue. I highly recommend re-opening this issue in the OLM repo. They may have more insight into how such a situation could be handled.

@estroz I will do so! Thanks.

@mmellin hopefully filing an issue with OLM helped. If you still think there's something the SDK should be doing that it isn't, feel free to reopen this issue.


Update aws-core, http-client-spi, ... to 2.17.164

Updates

software.amazon.awssdk:aws-core
software.amazon.awssdk:http-client-spi
software.amazon.awssdk:netty-nio-client

from 2.17.104 to 2.17.164.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "software.amazon.awssdk" } ]

labels: library-update, early-semver-patch, semver-spec-patch, commit-count:1

Superseded by #118.


RequirementMachine: Redo concrete contraction after splitting concrete equivalence classes

This fixes an edge case where we start with the following requirements:
- U : P
- T : P
- T.[P]A == C
- T == G<T.[P]A>
- U.[P]A == T.[P]A

and end up with the following set of minimal rules (where the type witness for [P]A in the conformance G : P is C):
- U.[P] => U
- U.[P:A] => T.[P:A]
- T.[concrete: G<C>] => T
- T.[concrete: G<C> : P] => T

Since U.[P]A and T.[P]A are concrete, we split the abstract same-type requirement into two requirements, and re-run minimization:
- U : P
- T.[P]A == C
- U.[P]A == C
- T == G<C>

The concrete conformance rule T.[concrete: G : P] => T does not correspond to a requirement, so it was simply dropped, and the above rules violate post-contraction invariants; T.[P]A is not a valid type parameter because there is no conformance requirement T : P in the minimized signature.
We can fix this by re-running concrete contraction after splitting concrete equivalence classes. After contraction, the above requirements turn into
- U : P
- C == C
- U.[P]A == C
- T == G<C>

Which correctly minimizes to
- U : P
- U.[P]A == C
- T == G<C>

Both concrete contraction and concrete equivalence classes are hacks, and we should think of a way to directly express the transformations they perform with the rewrite system.
Fixes https://github.com/apple/swift/issues/61192.

@swift-ci Please smoke test

@swift-ci Please test source compatibility


fix: Catalog release issues addressed

Closes #31
Closes #28
Closes #26
Closes #25

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: tangollama:x: devfreddyYou have signed the CLA already but the status is still pending? Let us recheck it.

:tada: This PR is included in version 1.2.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


NIFI-10616 fix some unreproducible bits

Summary
NIFI-10616
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[X] Apache NiFi Jira issue created

Pull Request Tracking

[X] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[X] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[X] Pull Request based on current revision of the main branch
[X] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[ ] Build completed using mvn clean install -P contrib-check

[ ] JDK 8
[ ] JDK 11
[ ] JDK 17

Licensing

[ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[ ] Documentation formatting appears as expected in rendered files

waiting for ANTLR3 upstream solution seems to take ages: I don't think it's worth waiting for it

Thanks for the reply @hboutemy, based on the status of ANTLR 3 https://github.com/antlr/antlr3/pull/209, moving forward with this solution looks good for now.


Depreciated option

-externalFeedCredentials argument should be marked as depreciated, as it does not correctly parse passwords from Service Connections. Instead, a NuGetAuthenticate task should be used.
Explanation:
In a password from a Service Connection, NuGetAuthenticate will correctly transform a '&' to '&' since '&' is an escape character in xml.
-externalFeedCredentials will simply leave the '&' as is, causing an exception to be thrown when restoring the packages.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0fa37a3f-6551-16ad-e164-a4ac522cdaab
Version Independent ID: a3fd6935-2b99-92ef-4eb3-3249fca01f8a
Content: NuGet restore, pack, and publish task - Azure Pipelines
Content Source: docs/pipelines/tasks/package/nuget.md
Product: devops
Technology: devops-cicd-tasks
GitHub Login: @steved0x
Microsoft Alias: sdanie

@DaDarkWizard Thank you for the feedback.
This argument is currently available per reference documentation and will need to be addressed by the product team, please move the issue here.
Including @steved0x for awareness.

Created the issue here.

This task is marked as deprecated in our new task reference and points to the new task.
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/nuget-command-v2?view=azure-pipelines#remarks


[Plugin] Container image registry & trivy & image cache services

closing issues ，any discussion can be reopened.


[regression in 2.37] getting text of element ignores the value of child's input

Simplified example:

    I have <input type="number" value="2"/> out of 2 stamps

With htmlunit 2.36 when we get the text of the paragraph, the value was I have 2 out of 2 stamps
With htmlunit 2.37 the value is I have out of 2 stamps
The value of an <input> is being ignored now. This is regression and it seems like a bug.
Please, let me know if you need more information.

Am 6. Februar 2020 13:10:53 MEZ schrieb Slava Semushin<EMAIL_ADDRESS>
Simplified example:

   I have <input type="number" value="2"/> out of 2 stamps

With htmlunit 2.36 when we get the text of the paragraph, the value was
I have 2 out of 2 stamps
With htmlunit 2.37 the value is I have out of 2 stamps
The value of an <input> is being ignored now. This is regression and
it seems like a bug.
Please, let me know if you need more information.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/HtmlUnit/htmlunit/issues/128

Can you please add the method you are using to get the text

@rbri Thank you for quick response!

Can you please add the method you are using to get the text

That's good question. I'm using Robot Framework keyword that seems like is executing WebElement.getText():
https://github.com/MarketSquare/robotframework-seleniumlibrary-java/blob/7164c2587b00f19208e888bd924739dd052366c5/src/main/java/com/github/markusbernhardt/seleniumlibrary/keywords/Element.java#L392
Also I'm using htmlunit-driver and if I'm not mistaken, getText() is implemented in the following way:
https://github.com/SeleniumHQ/htmlunit-driver/blob/122070661e7e309a849f59c0e616aa28b0b4c512/src/main/java/org/openqa/selenium/htmlunit/HtmlUnitWebElement.java#L463-L467
And here is getVisiableText():
https://github.com/HtmlUnit/htmlunit/blob/f4e710378e8fcfc58eeee33bc848f9d044ef4659/src/main/java/com/gargoylesoftware/htmlunit/html/DomNode.java#L821-L824
I'm not 100% sure that that's correct and I hope that you might know even better what has been changed as you maintain these 2 libraries. If you need more assist, let me know and I'll run it under debugger to be fully ensure.
At this moment, to me it seems like it's not htmlunit issue but rather htmlunit-driver (if that's the case, could you transfer the issue to the appropriate repo?). Also I see that getText() implementation has been changed twice, so I guess that one of these changes might be a cause: https://github.com/SeleniumHQ/htmlunit-driver/commit/f3cba476e9de3aa5fbcc419edefe749c05c55312 https://github.com/SeleniumHQ/htmlunit-driver/commit/dcab0f9e0ffc3a15625dfa8866997b42eb6b2e0c

You are right, there was some change done in 2.37 (because of https://github.com/SeleniumHQ/htmlunit-driver/issues/80)
But now the implementation is compatible to the other selenium drivers.
Have added a test case that proves, Selenium WebDriver with real browsers are producing the same result.

But now the implementation is compatible to the other selenium drivers.

@rbri Does it mean that this won't be fixed?
I can get text but there will be no input value. I can get input value but it won't have outside text and I can't just glue them together as an input value in a middle of text. Could you give me a pointer on how to get get a text like it was before?

@php-coder can you verify that you will get the same result when using selenium with real browser instead of htmlunit

@rbri Thanks! I'll try it this week and report back about results.

can you verify that you will get the same result when using selenium with real browser instead of htmlunit

@rbri You were right: in Chrome it returns the same result: The text of element 'id:number-of-stamps-block' should have been 'I have 2 out of 2 stamps', but it was 'I have out of 2 stamps'.
Thanks, I'm closing the issue then.


[Bug]: Incorrect focused state while dragging the column sequence in table widget

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
When I try to change the sequence of columns for the table widget then the focus state does not appear for the column which was dragged but instead the column above the dragged one gets focused.
Below video demonstrates this behaviour:
https://user-images.githubusercontent.com/14138515/146379775-fb01c6d5-0f41-4d7c-8e57-ee62badf6f4e.mp4
The above video will also demonstrates that it appears to happen in both the ways i.e. dragging from top to bottom or bottom to top.
Steps To Reproduce

Drag and drop the table widget on the canvas.
Under the columns property of the table widget, try to change the sequence by first clicking the textfield to edit the row label, then drag the clicked row(mouse down state) down.
Similarly, try to move a column up in the sequence.

Environment
Production
Version
Cloud

@keyurparalkar please add labels to issues!

Great catch @keyurparalkar. How did you notice this?

@dilippitchika I was working on another issue: #9188 Over there I noticed this issue.

Good Catch @keyurparalkar, lets take this up with the current errort 👍

The fix for this issue is there in PR https://github.com/appsmithorg/appsmith/pull/10046


Silent failure when sparse training matrix has > 2^31 non-zero elements

When training data is in the form of a scipy.sparse.csr_matrix and the number of non-zero values exceeds 2^31 - 1 (haven't checked precisely but max_int32 is the only thing that makes sense here), model predictions suddenly become awful. I strongly suspect there's a 32 bit integer used as an index into the non-zero elements of the sparse matrix, which ends up mixing up data from different columns incoherently.
Environment info
Operating System: Ubuntu 14.04.4
CPU: 4x Intel Xeon 24 core processors
Python version: 2.7.6
LightGBM built from source on commit 44ed4270eef68f0dd2ba093d4e5aa34460ddeecc
Reproducible examples
Note the predictions improve as we add more training data until the number of non-zero entries is too large, at which point the predictions are very close to 0.5 and the model is providing essentially no predictive power:

Script to produce examples:
import matplotlib
matplotlib.use('Agg')

import numpy as np
import lightgbm as lgb
import scipy.sparse
from scipy.special import logit, expit
import matplotlib.pyplot as plt

np.random.seed(0)

def true_proba(X):
    return X[:, 0]

def simulate_y(X):
    probs = true_proba(X)
    return (np.random.random(len(X)) < probs)

def simulate_X(nrows, n_sparse_cols, non_zero_frac):
    X = np.zeros((nrows, n_sparse_cols + 1))
    X[:, 0] = np.random.random(nrows)
    for i in range(n_sparse_cols):
        X[:, i+1] = (np.random.random(nrows) < non_zero_frac)
    return X

def get_data(nrows, n_sparse_cols, non_zero_frac):
    X = simulate_X(nrows, n_sparse_cols, non_zero_frac)
    y = simulate_y(X)
    X = scipy.sparse.csr_matrix(X)
    return X, y

def run_study(figfile, params, nrows, n_sparse_cols, non_zero_frac=0.2, n_test=1000):
    X, y = get_data(nrows, n_sparse_cols, non_zero_frac)
    print "With %s rows, got %s non-zero entries"%(nrows, X.nnz)
    data = lgb.Dataset(X, label=y)
    model = lgb.train(params, data, num_boost_round=10)
    X_test = simulate_X(n_test, n_sparse_cols, non_zero_frac)
    preds = model.predict(X_test)
    plt.plot(true_proba(X_test), preds, 'o')
    plt.title("nnz in train = %s"%(X.nnz))
    plt.xlabel("True probability")
    plt.ylabel("Model prediction")
    plt.savefig(figfile)
    plt.clf()

params = {
    'learning_rate': 1.0,
    'objective': 'binary',
    'max_depth': 2,
}

for nrows in [1000, 10000, 100000, 5400000, 5300000]:
    run_study('results_%s.png'%(nrows), params, nrows, 2000)

I'll be happy to provide further environment information if needed and can re-run modified versions of the script if more output or model information is needed.

@bauks Thank you for reporting this and special thanks for the reproducible example!
Unfortunately, I can't run your snippet even for nrow=5400000 due to the lack of the RAM. How much RAM is on your machine?
@guolinke From the Python side my assumption is only this line:
https://github.com/Microsoft/LightGBM/blob/44ed4270eef68f0dd2ba093d4e5aa34460ddeecc/python-package/lightgbm/basic.py#L884

@StrikerRUS Yes unfortunately the example uses about 90GB of RAM. I used a machine with around 360GB of RAM. I tried reducing the memory usage by constructing the sparse matrix directly instead of from the full 2d matrix, but it ends up using a similar amount of memory.

@bauks Hmmm, I tested on the machine with 112GB. Will try again.

@bauks BTW, is this issue presented in case of dense data (numpy/pandas)?

@StrikerRUS Hmm this script tries to use only as much memory as is required but I didn't see a noticeable drop in memory consumption (still around 90GB):
import gc
import matplotlib
matplotlib.use('Agg')

import numpy as np
import lightgbm as lgb
import scipy.sparse
from scipy.special import logit, expit
import matplotlib.pyplot as plt

np.random.seed(0)

def true_proba(X):
    return X[:, 0].toarray()[:, 0]

def simulate_y(X):
    probs = true_proba(X)
    return (np.random.random(X.shape[0]) < probs)

def simulate_X(nrows, n_sparse_cols, non_zero_frac):
    # Pre-allocate enough to confidently hold all non-zero elements
    n_rand_expected = nrows * n_sparse_cols * non_zero_frac
    nalloc = int(nrows + n_rand_expected + 5 * np.sqrt(n_rand_expected))
    data = np.ones(nalloc, dtype=np.float32) # We can leave the sparse data as ones
    row_ind = np.zeros(nalloc, dtype=np.int32)
    col_ind = np.zeros(nalloc, dtype=np.int32)

    # Setup the single dense column
    data[:nrows] = np.random.random(nrows).astype(np.float32)
    row_ind[:nrows] = np.arange(nrows, dtype=np.int32)
    col_ind[:nrows] = np.zeros(nrows, dtype=np.int32)
    ndata = nrows

    # Add the sparse columns
    row_range = np.arange(nrows, dtype=np.int32)
    for i in range(n_sparse_cols):
        new_col = (np.random.random(nrows) < non_zero_frac)
        n = new_col.sum()
        row_ind[ndata:ndata + n] = row_range[new_col]
        col_ind[ndata:ndata + n] = (i+1) * np.ones(n, dtype=np.int32)
        ndata += n
        if i%10 == 0:
            gc.collect()

    assert ndata <= nalloc
    return scipy.sparse.csr_matrix((data[:ndata], (row_ind[:ndata], col_ind[:ndata])))

def get_data(nrows, n_sparse_cols, non_zero_frac):
    X = simulate_X(nrows, n_sparse_cols, non_zero_frac)
    y = simulate_y(X)
    return X, y

def run_study(figfile, params, nrows, n_sparse_cols, non_zero_frac=0.2, n_test=1000, dense=False):
    X, y = get_data(nrows, n_sparse_cols, non_zero_frac)
    print "With %s rows, got %s non-zero entries"%(nrows, X.nnz)
    if dense:
        X = X.toarray()
        print "Dense X has shape {}".format(X.shape)
    data = lgb.Dataset(X, label=y, free_raw_data=True)
    model = lgb.train(params, data, num_boost_round=10)
    X_test = simulate_X(n_test, n_sparse_cols, non_zero_frac)
    preds = model.predict(X_test)
    plt.plot(true_proba(X_test), preds, 'o')
    if hasattr(X, 'nnz'):
        nnz = X.nnz
    else:
        nnz = (X != 0).sum()
    plt.title("nnz in train = %s"%(nnz))
    plt.xlabel("True probability")
    plt.ylabel("Model prediction")
    plt.savefig(figfile)
    plt.clf()

params = {
    'learning_rate': 1.0,
    'objective': 'binary',
    'max_depth': 2,
}

for nrows in [1000, 10000, 100000, 5400000, 5300000]:
    #run_study('results_%s.png'%(nrows), params, nrows, 1000, non_zero_frac=0.4, dense=True)
    run_study('results_%s.png'%(nrows), params, nrows, 2000)

Note the 2nd last line is commented out but converts to dense numpy matrices before passing the training data to lightgbm. When I run that version and comment out the last line, I find that even when the number of non-zero elements is larger than 2**31 the model behaves well, so this seems like just an issue with initializing from sparse matrices:

@bauks Unfortunately, your code fails on my machine at the step with nrows=5400000:
C:\Miniconda3\lib\site-packages\ipykernel_launcher.py:29: RuntimeWarning: overflow encountered in long_scalars
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-e5eec86a161e> in <module>()
      7 for nrows in [1000, 10000, 100000, 5400000, 5300000]:
      8     #run_study('results_%s.png'%(nrows), params, nrows, 1000, non_zero_frac=0.4, dense=True)
----> 9     run_study(params, nrows, 2000)

<ipython-input-2-535e61a28d6b> in run_study(params, nrows, n_sparse_cols, non_zero_frac, n_test, dense)
     42
     43 def run_study(params, nrows, n_sparse_cols, non_zero_frac=0.2, n_test=1000, dense=False):
---> 44     X, y = get_data(nrows, n_sparse_cols, non_zero_frac)
     45     print("With %s rows, got %s non-zero entries"%(nrows, X.nnz))
     46     if dense:

<ipython-input-2-535e61a28d6b> in get_data(nrows, n_sparse_cols, non_zero_frac)
     37
     38 def get_data(nrows, n_sparse_cols, non_zero_frac):
---> 39     X = simulate_X(nrows, n_sparse_cols, non_zero_frac)
     40     y = simulate_y(X)
     41     return X, y

<ipython-input-2-535e61a28d6b> in simulate_X(nrows, n_sparse_cols, non_zero_frac)
     27         new_col = (np.random.random(nrows) < non_zero_frac)
     28         n = new_col.sum()
---> 29         row_ind[ndata:ndata + n] = row_range[new_col]
     30         col_ind[ndata:ndata + n] = (i+1) * np.ones(n, dtype=np.int32)
     31         ndata += n

ValueError: could not broadcast input array from shape (1080249) into shape (0)

@bauks BTW, why not to create sparse matrix simply by https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.random.html#scipy-sparse-random ?

@StrikerRUS Thanks for pointing out scipy.sparse.random, that leads to much more simple code. The simulate_X function can be replaced with:
def simulate_X(nrows, n_sparse_cols, non_zero_frac):
    dense_col = scipy.sparse.csr_matrix(np.random.random((nrows, 1)).astype(np.float32))
    sparse_random = scipy.sparse.random(nrows, n_sparse_cols, density=non_zero_frac,
                                        format='csr', dtype=np.float32)
    return scipy.sparse.hstack([dense_col, sparse_random], format='csr')

Unfortunately it looks like this method is slower and requires more memory; it used around 140GB and took 2 hours to run the 5400000 case. The results are similar, with the 5400000 case showing very poor predictions. If you have a suitable machine you could try running it; perhaps the error you encountered before won't show up.
Do you know if that warning was only for the 5400000 case? If so perhaps it's related to the failure. The failure indicates row_range[new_col] has shape (1080249), so we should also have n = 1090249. For row_ind[ndata:ndata + n] to have shape (0) it would mean we somehow did not allocate enough rows in row_ind, which would be quite startling (at 5 sigma) unless there's a bug somewhere else or something related to the warning is affecting row_ind.
Could you print out nalloc, data.shape, row_ind.shape and col_ind.shape after allocating them? You could also double the size of nalloc before creating the arrays for safety. And perhaps within the loop, just before the line that failed, print out row_ind.shape, ndata, n, row_range.shape, new_col.shape, and row_range[new_col].shape.

@StrikerRUS
I checked the cpp code, and think it is fine..
I don't have a machine to test that, can you provide the types
        ptr_indptr, type_ptr_indptr, __ = c_int_array(csr.indptr)
        ptr_data, type_ptr_data, _ = c_float_array(csr.data)

in https://github.com/Microsoft/LightGBM/blob/master/python-package/lightgbm/basic.py#L888-L889
especially for  type_ptr_indptr.
Thanks very much.

Hi @guolinke,
Hopefully the following helps:
csr.shape is (5400000, 2001)
csr.data is a numpy array with length<PHONE_NUMBER> and dtype float32.
csr.indices is a numpy array with length<PHONE_NUMBER> and dtype int64.
csr.indptr is a numpy array with length 5400001 and dtype int64.
Running
ptr_indptr, type_ptr_indptr, __ = c_int_array(csr.indptr)
ptr_data, type_ptr_data, _ = c_float_array(csr.data)

gives
type_ptr_indptr = 3,
type_ptr_data = 0,
type(ptr_indptr) = lightgbm.basic.LP_c_long,
type(ptr_data) = lightgbm.basic.LP_c_float


How to make jazzer get coverage of another process?

I would like to use jazzer to test a program A, while during program A I start several programs by ProcessBuilder, whose coverage I want jazzer to monitor as well.
Does jazzer support the case I mention?

Using coverage of external applications is currently not possible in Jazzer.
Is there a reason to not fuzz the other applications directly? This could potentially uncover problems otherwise hidden by the way they are started by the first application.

@bertschneider Thanks for your comment!

Is there a reason to not fuzz the other applications directly?

Because in my program, how other external applications run is determined by the first application's actions through inter-process communication. I would like get total coverage instead of that of only the first application or the other applications.


Link to irc on web site and github and show some respect to contributors!

I created a very serious issue https://github.com/ark-lang/ark/issues/436 but it was closed with no justification https://github.com/ark-lang/ark/issues/436#issuecomment-115468786 and when I told that justification is Proper Software Development Practice in https://github.com/ark-lang/ark/issues/437 I was then told that

The issue tracker is not the place for philosophical discussions (try irc).

and

If you have a proposal and a good argument for it, then open an issue and request feedback.

https://github.com/ark-lang/ark/issues/437#issuecomment-115503856. Well first of all there is no link to the irc on the home page and there is no link to the irc in the github so how am I supposed to go to the irc when this irc is a total mystery? When I do have a proposal that keywords be changed and I had a perfectly good argument for it I opened an issue and requested feedback but it was closed with absolutely no feedback and just unhelpful

This is not going to be changed.

comment https://github.com/ark-lang/ark/issues/436#issuecomment-115468786! If you want people to use your lang then you need to show RESPECT when people contribute and follow PROPER SOFTWARE DEVELOPMENT PRACTICE. If PROPER SOFTWARE DEVELOPMENT PRACTICE like explaining why all issues closed with full descriptive description is not followed then people see this project and this lang and they think it is amateur lang.

Hi. We appreciate all of the issues you've posted, everyones contributions helps make Ark a better language. However, we would appreciate if you were to keep more personal matters like this away from the issue handler, and talk to us about it on IRC or via email. We would prefer to keep the issue handler for discussions relevant to the languages development, and most importantly technical details relevant to the actual language.
And yes, we'll put a link to the IRC on the website.

We don't disrespect you, we just closed an issue that didn't make any sense in our eyes.
As felixangell said, please only open issues about the language itself.
You can join us on irc.freenode.net in #ark-lang.

@raoulvdberge I did open issue about the language itself. It was about keywords and them being ambiguous and unclear but it was closed with no justification. That is DISRESPECTFUL and does not follow PROPER SOFTWARE DEVELOPMENT PRACTICE! If you or @felixangell or @MovingtoMars are not understanding it then ask for clarification but never close discussion that is good and proper! Never! That is not proper and not respectful!

@BigBoneDaddy Yes, but you opened an issue about us being disrespectful, and this one is partly relevant to the language -- but also points out "showing respect".

@BigBoneDaddy If you'd like to discuss us being disrespectful, join our IRC channel.

@BigBoneDaddy Or email us.

We closed this issue again because:

we made clear earlier why we closed your original issue about the keywords
it's not relevant to the language

@raoulvdberge Discussing ambiguous and unclear keywords is relevant to the lang.

@felixangell Showing respect is important part of software dev. Please show respect it is is what software devs do.

@BigBoneDaddy Yes, it is relevant to the language so we gave you an answer and closed the issue. As for these issues, they are not as relevant, especially the one that asks for "respect".

@felixangell Respect is always important in software dev. I show respect. I ask that you do the same. Respect is always relevant. Always!

@BigBoneDaddy Always!


[image_picker][ios] image path is change after update application version

Steps to Reproduce
I checked in iPhone(iOS) emulator.

use image picker

ImagePicker.pickImage(source: ImageSource.gallery).then((_image){
  print(_image.path);
});

select image and check image.path
add number version in pubspec.yaml ( 1.0.0 -> 1.0.1 )
select same image in 2. and check image.path

in my emulator the image path has being changed like this

before update : /Users/user/Library/Developer/CoreSimulator/Devices/2AAAB012-FACF-4F29-A28D-285ABEAA4FFB/data/Containers/Data/Application/06BF15C9-34F8-4C9B-8EFD-6755B37335DA/tmp/image_picker_130E95AA-86A7-4909-8B3D-A2719704A259-48298-000015C757019416.jpg
after update : /Users/user/Library/Developer/CoreSimulator/Devices/2AAAB012-FACF-4F29-A28D-285ABEAA4FFB/data/Containers/Data/Application/40EBCCD5-602A-4EA2-AECE-FCAD888DA506/tmp/image_picker_70EF6C82-4A03-45EC-9663-7AAAB6B7B5ED-49406-000016007D01DDC1.jpg

In my application, I save image path to database and show it by list view.
But when I update my application, the path has been changed, so I can't read the same Image.
I think image path is always same ( because android is same ).
Target Platform:
iOS
Target OS version/browser:
iOS 13.3
Devices:
iPhone 11 Pro Max
Logs
$ flutter doctor -v
[✓] Flutter (Channel stable, v1.12.13+hotfix.5, on Mac OS X 10.15.2 19C57, locale ja-JP)
  • Flutter version 1.12.13+hotfix.5 at /Users/user/Desktop/development/flutter
  • Framework revision 27321ebbad (4 weeks ago), 2019-12-10 18:15:01 -0800
  • Engine revision 2994f7e1e6
  • Dart version 2.7.0

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
  • Android SDK at /Users/user/Library/Android/sdk
  • Android NDK location not configured (optional; useful for native profiling support)
  • Platform android-29, build-tools 29.0.2
  • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
  • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)
  • All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 11.3)
  • Xcode at /Applications/Xcode.app/Contents/Developer
  • Xcode 11.3, Build version 11C29
  • CocoaPods version 1.8.4
[✓] Android Studio (version 3.5)
  • Android Studio at /Applications/Android Studio.app/Contents
  • Flutter plugin version 42.1.1
  • Dart plugin version 191.8593
  • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)
[!] Connected device
  ! No devices available

I think this issue is similar problem
https://github.com/flutter/flutter/issues/23957

I resolve myself that using relation path

I faced it, Could you please post your solution? I need your help.
Thanks @hatano0x06

@uzumaki258
ok
if you can see the file path in other version, your can now how does it change
previous version file path
path after get image: /private/var/mobile/Containers/Data/Application/6FC924DB-4D50-420F-AE26-389C37A3F63F/tmp/image_picker_6990DBDB-EB4D-489F-B5E8-A5C1A8CAA003-3521-00000391FF88C78F.jpg
getApplicationDocumentsDirectory path: /var/mobile/Containers/Data/Application/6FC924DB-4D50-420F-AE26-389C37A3F63F/Documents

current version file path
path after get image : /private/var/mobile/Containers/Data/Application/47EF6330-6C6A-4ECE-82D1-DB2FDCF53EE7/tmp/image_picker_11DC60E6-C1B8-42A4-BA27-3A9BB715BA2E-3528-000003939D6B1DD1.jpg
getApplicationDocumentsDirectory path : /var/mobile/Containers/Data/Application/47EF6330-6C6A-4ECE-82D1-DB2FDCF53EE7/Documents

your can see the hash code in image path will change, if you change version.
So, the good way is save the lastast image path split by "/" ( image_picker_11DC60E6-C1B8-42A4-BA27-3A9BB715BA2E-3528-000003939D6B1DD1.jpg).
And if you read, create the path from getApplicationDocumentsDirectory ( be careful your have to replace some words) and connect the path your save it.

private/var/mobile

Understood, thank you so much


Fix verified files after changing version number

The unit tests won't work anymore because of the version number. This fixes it.

Hi @fnajera-rac-de thanks for your contribution !!!


[Feature Request] Support UPDATE command with Deletion Vectors

Feature request
Which Delta project/connector is this regarding?

[x] Spark
[ ] Standalone
[ ] Flink
[ ] Kernel
[ ] Other (fill in here)

Overview & Motivation
This task is part of the Deletion Vectors to speed up DML operations. Refer to the https://github.com/delta-io/delta/issues/1367 for the design and details.
The scope of this issue is to support the UPDATE command with Deletion Vectors (DVs). In the current implementation, UPDATE rewrites all files having at least one row that matches the UPDATE condition. The rewrite copies over unmodified rows. This could be an expensive operation if the UPDATE is touching only a few rows per file. Using DVs, the copy can be avoided.
This FR follows [Feature Request] Support DELETE command with Deletion Vectors. Click on the link to find more info.
Further details
The current implementation of the UPDATE has the following steps:

Step 1: Find the data files in the current snapshot that have rows matching the UPDATE condition.
Step 2: Replace the TahoeFileIndex in the original table scan with a TahoeBatchFileIndex that contains the file list generated in Step 1.
Step 3: Rewrite the files using the scan created in Step 2.
Step 4: Use the Delta protocol to remove all files from Step 1 and add new files generated in Step 3.

UPDATE with DVs modifies the above steps as follows:

Step 1: (not changed) Find the data files in the current snapshot that have rows matching the update condition. Call the file list generated as candidate files.
Step 2: Replace the TahoeFileIndex in the original table scan with a TahoeBatchFileIndex that contains the candidate files generated in Step 1. In addition, we also modify the Scan and TahoeBatchFileIndex to select a few additional metadata columns.

_metadata.file_path - the absolute path of the data file
row_index - number corresponds to the position of this row in the data file from which it is read.

Step 3.1 (process unmodified rows): Scan files using the scan created in Step 2, write DVs and output (AddFile(new_dv), RemoveFile(old_dv) pairs for files that are partially updated, or RemoveFile for those that are fully updated.
Step 3.2 (process modified rows): Rewrite files using the scan created in Step 2, but write only the updated rows.
Step 4: Write file actions produced by Step 3.1 and Step 3.2 to Delta log.

The "avoid copying rows" benefit described in the "Overview & Motivation" section is implemented in Step 3.1.
Pull requests

Feature
PR

UPDATE produces DVs (w/ and w/o CDC)
https://github.com/delta-io/delta/pull/1942

Metrics
Pending

Tests
Pending

Willingness to contribute
The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?

[x] Yes. I can contribute this feature independently.
[ ] Yes. I would be willing to contribute this feature with guidance from the Delta Lake community.
[ ] No. I cannot contribute this feature at this time.

@xupefei this is done .. right? can we close it?

Does this also works with MERGE statements?

Merge implementation is separate from UPDATE/DELETE. DV support for MERGE is tracked here: https://github.com/delta-io/delta/issues/2296


Optimize yml extraction in CompatibleConvertFrom-Yaml

RE: the script-level variable being used as a module variable.
I can't use simple a $YmlPresent = $null outside of the function, as the way the scoping works out we still end up running get-command repeatedly.
I think this is a safe pattern. Welcome feedback.

The following pipelines have been queued for testing:java - templatejava - template - testsjs - templatenet - templatenet - template - testspython - templatepython - template - tests You can sign off on the approval gate to test the release stage of each pipeline. See eng/common workflow

I don't have much context on this issue or the module installation work that preceeded it...
Would it make sense for us to install modules from an Azure Artifacts feed that upstreams to PowerShell gallery? That might be faster and more consistent... likely at the cost of efficiency in CI given that yq is already on the agents. https://learn.microsoft.com/en-us/azure/devops/artifacts/tutorials/powershell-upstream-source?view=azure-devops

I don't have much context on this issue or the module installation work that preceeded it...
Would it make sense for us to install modules from an Azure Artifacts feed that upstreams to PowerShell gallery? That might be faster and more consistent... likely at the cost of efficiency in CI given that yq is already on the agents. https://learn.microsoft.com/en-us/azure/devops/artifacts/tutorials/powershell-upstream-source?view=azure-devops

That's definitely a way to avoid the issue with installing a module!

@weshaggard it looks like Ben and Dan think that I should always use the powershell-yaml, but to use a public feed of ours with upstream to psgallery instead of directly installing the module.
My thoughts if we want to do this:

Add powershell-gallery as upstream source to azure-sdk-tools public feed
pull the yaml package into the feed from upstream
Update `CompatibleConvertFrom-Yaml and Module helper to utilize the yaml version always

if I have to pull other packages from upstream I will at this point as well

Do you have any protests to above? If yes, good with that feed?

@weshaggard it looks like Ben and Dan think that I should always use the powershell-yaml, but to use a public feed of ours with upstream to psgallery instead of directly installing the module.
My thoughts if we want to do this:

Add powershell-gallery as upstream source to azure-sdk-tools public feed

pull the yaml package into the feed from upstream

Update `CompatibleConvertFrom-Yaml and Module helper to utilize the yaml version always

if I have to pull other packages from upstream I will at this point as well

Do you have any protests to above? If yes, good with that feed?

I have not issues with trying out this approach. I suspect we don't even need to add PS gallery as an upstream if we simply include the package we one on the feed. I'd also be OK creating a feed specifically for PS if that is easier.

I have not issues with trying out this approach. I suspect we don't even need to add PS gallery as an upstream if we simply include the package we one on the feed. I'd also be OK creating a feed specifically for PS if that is easier.

I do feel that we should use the azure-sdk-tools public feed, so I'll take action and make this happen. Thanks @weshaggard

I know @benbp is also looking into this so please coordinate

Yeah @scbedd I'm already working on this. But it can be separate from this work regardless, since they are both achieving different things (reliable downloads vs. not downloading too much)

https://github.com/Azure/azure-sdk-tools/pull/9392

Yeah @scbedd I'm already working on this. But it can be separate from this work regardless, since they are both achieving different things (reliable downloads vs. not downloading too much)

#9392

I'm going to wait on #9392, then update this PR to remove the optional usage of yq. My install loop changes work just fine.

@scbedd I should have it in today, a couple more changes pending.

@scbedd I should have it in today, a couple more changes pending.

Ok I'm finally using it. Relying on always installing the module now.

It's failing due to merge conflicts on the auto merging branches. Just going to copy this to another PR.


feat: added groq provider

Added groq provider
Converted content object list in message (ToolResponse message) to string to match api schema.
Removed the duplications in providers that compatible wit openai api schema
Extracted open ai related utils function to a new file openai_utils
Added integration tests for google and groq providers and extract the utils to mock_server.rs

Note:
I could not get the capture screenshot working with groq provider as when the message include the toolResult with image the api returns bad request. So far I found they can categorised into 2 scenarios as below:

For most non-vision models, it does not accept the user message with image although in the doc it says it does.
Error message:
{ "request_id": "req_01jfckkap9fcdv2c4cr4tepnfw", "created_at": "2024-12-18T10:04:15.434Z", "error": {  "message": "message[0].content must be a string",  "type": "invalid_request_error" } }

I also tried llama-3.2-11b-vision-preview, it does not accept the payload including the user message with image and system message at the same time.
Error message:
{ "request_id": "req_01jfcmfx6yf0zbjt4028v4gkcd", "created_at": "2024-12-18T10:19:51.904Z", "error": {   "message": "prompting with images is incompatible with system messages",   "type": "invalid_request_error" } }

I tried with the python cli, it does not work either but with different reason. I did not look in-depth.  I feel this feature does not work originally and we can enhance this in the future when we have more usages with groq

@lifeizhou-ap I wouldn't worry about image support for groq and llama models for now if they are proving a problem.


Unexpected HTTP errors on _data calls cause blank routes without any caught errors

What version of Remix are you using?
1.12.0
Are all your remix dependencies & dev-dependencies using the same version?

[X] Yes

Steps to Reproduce
Force a route's _data request to fail with an HTTP error and a response that remix doesn't expect (e.g. some string). This could happen when there is something wrong with e.g. CDN, proxy or load balancer configuration, an error that is generated outside or remix's control. Then make a client side navigation to that route.
Here's a diff from base create-remix app with express server where I respond with an error from express handler: https://github.com/celebro/remix-http-error-example/commit/3e644df76e4e673a86cbdaecae4e138386ccb682
Expected Behavior
Remix should trigger ErrorBoundary with some generic error.
Actual Behavior
Remix renders an empty route, without any indication of an error.

Same problem with Remix and Cloudflare CDN Messages (Rate Limit, WAF etc.). How are WAF errors or Captchas to handle that the client receives via the data request (_data) ? Depending on the status code (>=500?) you might have to redirect to the complete html page to see the original message or the captcha?

Hm, yeah this is tricky.  The current behavior is as such because you can return anything from your loader, and as long as you don't throw that will make it through as loader data:
export function loader() {
  // This will make it to `useLoaderData`
  return new Response('whatever you want', {
    status: 500,
    headers: {
      'Content-Type': 'text/plain'
    }
  });
}

Basically, we don't assume that 4xx/5xx should go to the ErrorBoundary - we let return/throw distinguish that.
So, if you get a 5xx response from somewhere other than the loader (i.e., CDN, Express, etc.), we can't distinguish that from a user-returned 500 at the moment.
My initial gut reaction is to add an X-Remix-Response header responses coming from loader/action calls on the server, and then if we don't see that client side we can automatically treat 4xx/5xx status code as errors since we know they they did not come from user-defined application loaders.
Somewhat related, I have seen folks ask for all 4xx/5xx to be automatically categorized as errors.  I don't love this idea because it breaks from fetch behavior (and is closer to how axios handles it), but I can understand where the request is coming from.  This would obviously be a big breaking change so would require a future flag if we went this route (I'm not confident this would win out).

This should be resolved by #6783 and available once 1.18.2 is released


Fix dark mode

BUILD: https://builds.gradle.org/repository/download/Gradle_Release_Check_BuildDistributions/76984767:id/distributions/gradle-8.6-docs.zip!/gradle-8.6-20240117193041%2B0000/docs/userguide/kotlin_dsl.html

@bot-gradle test BD

I've triggered the following builds for you. Click here to see all build failures.

BuildDistributions build

@bot-gradle test BD

I've triggered the following builds for you. Click here to see all build failures.

BuildDistributions build

@bot-gradle merge

The merge queue build has started. Click here to see all failures if any.

@bot-gradle clean

@bot-gradle test BD

I've triggered the following builds for you. Click here to see all build failures.

BuildDistributions build

@bot-gradle merge

The merge queue build has started. Click here to see all failures if any.


Permissions on user's roles not being respected

"Admin" only users should not have the same privileges as "Editor" and couldn't create and sync layers

Changes in contents methods:

Create: only by users with editor role;
Update/patch: disabled to all users;
Remove: only by the user who synced the content, identified by syncedBy field.