Proposal: New Architecture for sqlx::query!() Macros

Status Quo
Currently, the default mode of operation for the sqlx::query!() family of macros is to connect to a running database server (or open that database in-process for SQLite) as specified by the DATABASE_URL environment variable, and then use database-specific commands to parse and analyze the passed query, to get information on bind parameters and output columns and generate code based on that information.
As an option, the macros can also operate in an "offline" mode where instead of connecting to a database server, they read a sqlx-data.json file created by cargo sqlx prepare which contains the same data and use that to generate code.
The implementation is relatively straightforward and functions well enough, but has some drawbacks:
Compiling a project with a lot of query!() macro invocations is painfully slow.
Since each macro invocation involves opening and working with a TCP connection to a database server from within an asynchronous runtime, the query macros execute a lot more code than your typical procedural macro. This is compounded by the fact that procedural macros are exclusively compiled in debug mode which means no optimizations (which async code relies pretty heavily on to be fast).
I have tried playing around with setting rustflags in a config.toml to force sqlx-macros to compile in release mode but it didn't really help and seemed to actually worsen compile times if it did anything at all; the most noticeable thing that it did was clobber all the cached compiler artifacts for the whole dependency tree, requiring a full recompile.
Macro invocations are currently executed serially which further compounds the problem.
Currently as a mitigation we only start a single runtime and then cache it in a lazy_static!(); since proc macros are executed in a separate process that is kept running for the duration of the compiler, this saves some overhead per invocation. I've been meaning to see if caching a Pool of connections saves any overhead as well. I imagine it would, at least somewhat. We could cache analysis results as well, but that would only help for queries that are copy-pasted a lot.
Many people find the idea of procedural macros opening arbitrary TCP connections to be highly unpalatable.
Oftentimes when someone comes across SQLx and starts reading about the query macros, their first reaction is something along the lines of, "wow, the compiler actually lets you do that? That's disgusting!"
Fair enough, the current procedural macro environment is extremely laissez-faire and there have been calls to lock down what procedural macros (and build scripts) can actually do, thanks to sensationalized proof-of-concept projects showing how a proc macro executed by a transitive dependency could steal sensitive information like SSH keys and the like.
Personally, I think the threat is kind of overblown, or at least rather moot, considering a malicious crate in your dependency tree could accomplish all kinds of nastiness already, just at runtime instead of compile time. I fear that a big push to sandbox procedural macros and build scripts could lure people into a false sense of security and make them feel like they don't need to audit their dependencies. But, I digress.
The query!() macros don't play that well with IDEs.
This situation has improved drastically compared to when we first introduced SQLx, to the point where IntelliJ-Rust and Rust Analyzer can both provide code completion for sqlx::query!(). However, it can still take some time to get results, in part thanks to point 1 here, and the macros aren't "pure" in the sense that their output could change from run to run based on schema changes, without the compiler or IDEs being able to pick up on it. They should be working from cached output where possible, and be able to invalidate that cache in a way that IDE plugins can learn about.
Offline mode is finicky to use and it's easy to forget to run cargo sqlx prepare before committing.
Offline mode isn't easily discoverable (in part thanks to just how dense the documentation on sqlx::query!() has gotten... we really need to move that stuff to an MDBook or something). It's also easy to forget to run cargo sqlx prepare before committing, which often leads to really confusing CI errors.
In the past, what I've done is add cargo sqlx prepare && git add sqlx-data.json to .git/hooks/precommit to make sure I don't forget, but this makes committing take forever thanks to point 1.
The macros don't have a way to ensure that the database is up-to-date with migrations.
This is somewhat on purpose as we don't want the macros automatically executing migrations which may end up breaking the user's local development database and/or clobbering manually generated test data. Still, it does get annoying.
Proposal
Instead of having the macros connect directly to the database server at compile time, they could communicate with a daemon process that the user starts and leaves running while working on their project.
I'm imagining this daemon process as another subcommand of sqlx-cli, perhaps cargo sqlx watch or something like that. It would watch target/sqlx which the macros would write their query strings to as files. I considered using a crate like interprocess to let the macros and daemon talk directly, but I think communicating with the filesystem is better because that's a medium that the compiler natively understands as I'll elaborate on in a second.
Since a cargo installed binary is built in release mode by default, the latency of the actual request to parse and analyze the query should improve quite a bit.
The macros could submit their request to the daemon and then return some dummy code (or even a temporary error, like Waiting for query analysis. Is `cargo sqlx watch` running?), telling the compiler to watch target/sqlx for the daemon to write the result of processing the query. Then the compiler could move on to the next macro invocation instead of blocking on the completion of the first one. This would let the macros essentially run in parallel, which should significantly reduce overall compile times.
Assuming #570 is implemented as part of this rearchitecting, the daemon could also automatically update entries in .sqlx/ for successfully parsed queries so that cargo sqlx prepare doesn't have to be manually run before committing.
This would also be more amenable to sandboxing as we would only require write access to target/ and read access to the project, which wouldn't be doing anything more than what build scripts are already expected to do, and any sandboxing scheme that is implemented would be required to support. Support for Windows named pipes or Unix domain sockets would probably be right out in this case unless we specifically asked for it, and that would be an uphill battle.
The daemon could also periodically re-run the analysis of queries to pick up changes to the database schema.
@mehcode suggested that the daemon could automatically migrate the database if necessary, so no other manual setup would be needed. It could even spin up a temporary database so migrations could be automatically tested and synchronized with the query macros without breaking a local development database.

I'm wondering if there's a way that we can support incremental adoption of cargo sqlx watch into people's workflows. It is yet another process that needs to be running to compile your code, and some might find that annoying regardless of the potential benefits.
Perhaps cargo sqlx watch could create target/sqlx/watch.lock or something as a temporary file and the macros could just check if that file exists or not to know if cargo sqlx watch is running. If the file doesn't exist, they can fall back to their current behavior.

If #1183 could be finished and the workspace situation improved (see discussion in #1223), I think that would solve my own pain points. I'm not really convinced a daemon would make working with SQLx easier at all.
For context: The database part of the service I work on is updated infrequently enough that an offline-first workflow works well (.env contains SQLX_OFFLINE=true and the macros only connect to the DB as part of cargo sqlx prepare).

@jplatte hmm, then as a compromise, maybe the macros could use the cached data first if it exists and the migrations haven't changed. Then only the modified macro invocation will actually have to talk to the database.
It's certainly worth trying as a first step at least.

I have thought a lot about how to handle these compile-time query checking features (not in rust, but also in other languages). Needing access to a database of the same kind as the production base during builds is almost always hard and leads to complex build system. This is why I want to suggest an alternate approach:

during compile time, only record assertions that need to hold on the DB for queries to be valid
provide a simple method to execute these assertions against any database after compilation

So in the end, you generate a check_compat(connection) method that the compiled program can use to check whether the DB satisfies all the things it expects directly during startup and fail fast if it does not. You could also provide a CLI command that does the same. To me, this has the following advantages:

no need to access the DB during compile time
also catches deployment issues, such as migrations that haven't been applied, since the compat check can be run against the real DB
still fails fast: after application startup, no query should fail due to trivial syntactic/query issues
you can test that old versions of apps are still compatible with the DB after running certain migrations (necessary if you do rolling deploys where different versions of the code are accessing the same DB), simply by running the check_compat function of the old version. This does not require a rebuilt of the old version.

Is there any use case where getting the error directly at application startup (or by running the checker function directly via cli after build or even as an automatic post-build step) is significantly worse than getting them during build? The main motivation for compile-time query checking from my experience is that you want to avoid situations where you first have to trigger a certain code path to see the error, which leads to long cycle times and is inconvenient. But if you can check all queries directly at application startup or with a CLI command, that solves the same issue without the disadvantages of compile-time network access.

during compile time, only record assertions that need to hold on the DB for queries to be valid

This is quite simply not feasible. SQLx checks the queries directly against the database (or against cached metadata previously obtained from the database) because the alternative is parsing / interpreting the query string, which could be using many, many different syntax constructs and features depending on the DB used or potentially even depending on DB plugins.
As a little thought experiment, consider this simple query: select my_function(). It's super basic, but at the same time it's impossible to know what type it will return for software other than the actual database you would use it with.

Not to derail the conversation topic to badly but i see a use case in the mechanics that @bennofs proposed with check_compat(connection) but i also agree with @jplatte that this is not really the way to go. I just imagined (and its probably the best to create a new issue for this) a way of having both. I am not aware if the current cli-tool or any migrate-style macro handles this but it could be helpful if you can take a sqlx-data.json created in the CI-Job or on your dev machine to the deployment target and have the cli-tool use the sqlx-data.json you brought with you and "compare" it against the live database and/or have a migrate-like macro to check the brought with you sqlx-data.json against the live database before starting? This would give you more confidence if you can't build against the live database (which isn't a good idea anyway i think).

@pythoneer It sounds like you are describing cargo sqlx prepare --check, which is already a thing and useful also as a CI job to make sure any new migrations are accounted for in the code (and the query cache).

As a little thought experiment, consider this simple query: select my_function(). It's super basic, but at the same time it's impossible to know what type it will return for software other than the actual database you would use it with.

Hmm, couldn't you specify the return type explicitly in the code? Are there many other things apart from return types and argument types that you'd need to know about the query? Ideally you'd be able to record the (parametrized) query, return and argument types and then check against the database that those types are convertible to/from the the actual query parameters/return value.

Hmm, couldn't you specify the return type explicitly in the code?

In theory yes, but that would be a lot of unnecessary extra boilerplate.

Are there many other things apart from return types and argument types that you'd need to know about the query?

 And you would have to be able to fully parse the query to get that information, which may be reasonable for a simple DB, but SQLx supports multiple very complex databases. Even "just" a self-made parser for PostgreSQL's SQL flavor would be a huge project.

I definitely agree that parsing queries should be avoided. As far as I can see, with query_as!, the rust types of output/input are known (parameters must be known because they are rust values, and output type is determined by the named struct). But yes, specifying the type for the query! macro could become annoying, although personally I'd take the tradeoff of having to specify types explicitly in my code vs needing to keep files depending on db schema up to date.
Anyway, this is not core of sqlx so I may just try to implement that myself as a separate crate, and see how well it works :)

As far as I can see, with query_as!, the rust types of output/input are known (parameters must be known because they are rust values, and output type is determined by the named struct)

The processing is still driven by the metadata returned from the database. Proc macros can't introspect any code that isn't in their input, so all query_as!() does it take the name of the struct and emit a literal value for it the same way query!() does, but without generating an ad-hoc struct first.
It will happily emit fields that aren't in the struct or omit ones that are if they aren't in the query, and it's up to the compiler to sanity check that.

At least considering Postgres, couldn't we use an exported schema using pg_dump -s database or a custom JSON format that isn't dependent on specific queries? This way, you could run cargo sqlx prepare only once after all migrations are run on the DB, then the file it generates can be used to validate any query. Would that be too slow or complex? I like some of these ideas around automating testing for migrations w/o breaking current database, but I'm honestly not sure if I love running another daemon for project development.

@loafofpiecrust have a look at this FAQ answer: https://github.com/launchbadge/sqlx/blob/master/FAQ.md#why-cant-sqlx-just-look-at-my-database-schemamigrations-and-parse-the-sql-itself
It's a bit snarky but the suggestion comes up a lot and so it gets a bit annoying to respond to.

Yup @abonander sorry for the noise.

Also related, I find using query! cumbersome sometimes:

I run my migrations from my binary, to use query!() I need to run migrations with sqlx-cli first during development. It's not the end of the world, but makes me dread working on schema related things.
It locks you into using a single backend per crate.

I either need to introduce additional crates, one per backend
Misusing features make them mutually exclusive, this solves issue only partially - still can't SQLite and PostgreSQL both using query!()

I stopped using it due to that, but then I added a crate dependency that uses it, and it's even worse - it is misusing features and provides offline support only for PostgreSQL.
I think you're on the right track that macro opening any kind of network connections is pretty wild design. Is there any way to do something like:

if query can be done with sqlx-data.json do that
if not in release mode and query can't be done with offline support - just do it the same way regular query works.

emit a warning or whatever

if in release mode and query can't be done with sqlx-data.json bailout.

I don't mind checking that sqlx-data.json is up-to-date during CI

This obviously depends on sqlx-data.json to support multiple backends at the same time.

I'd rather not daemon:

On my local machine during test/check...maybe? It does complicate on-boarding a little, but oh well.
For sure don't want to run in my "lab" (Sometimes I run whole stack in container where code is rebuilt on change)

Sorry for the wall of text. sqlx is a great crate, you are all doing a great job.

This is an interesting idea, and is a much bigger idea than my thought of having a sqlx describe command to inspect the results of a query.
I do see significant potential advantages to separating the query analysis from the rust compilation:

It provides a familiar workflow for anyone used to code-gen tools (xsd, swagger, proto, etc.)

Might enable outright code-gen in the future, allowing sqlx type-checking without macros

Introduces an offline-cache of the results for all queries:

very likely to speed up compilation
partial progress towards an optional 'query sanity check' method at runtime (describe query at runtime; compare result to compile time result; return errors for differences)

Could provide better access to sqlite-explain logs (proc macro swallows stdout/stderr output, which is where the sqllite-explain logging goes for 99.9% of uses).

Possibly better formatting options (I develop sqlx sqlite analysis changes using a fork of sqlx which formats the logs as graphviz dot format, because plain text execution traces get hard to visualize fast)

The potential disadvantages I see are:

compilation instructions for projects using sqlx would diverge from typical rust instructions (you either need to run a tool, or launch the daemon)
It might be tricky for the daemon to decide when to re-describe queries in the face of schema migrations (assuming it's cherry-picking what to describe)

the built-in migration implementation might provide a trigger, but using the built-in migrations is optional
some databases might have DDL timestamps/logs which could provide a trigger

Is there a way to signal to rust-analyzer that a 'describe result has changed'?  If describe is asynchronous, will rust-analyzer react when some dummy/result file is updated with the actual results?

Relying on something resembling 'offline' mode, with an external tool to do the sql query analysis, seems like a better default than the current default.
For how I use sqlx, I would still prefer if cargo run/check was capable of handling everything, even if it was opt-in. But even if that was functionality was lost, sqlx would STILL be better than only finding out about a typo after the application tries to run the query.


As a home owner/renter, I want to be able to monitor a video feed of my property so that I can feel more secure when leaving my home.

This is the MVP for the first sprint.

We are able to do this, with port forwarding. We will leave it up to our users whether or not they want to expose a port and maybe we can have some guidelines for it. Regardless, we are currently able to do this.


SetSlotUnavailable triggers Assert

Calling SetSlotUnavailable (always?) triggers an assert when checking whether the parent module of the slot matches the module it is about to be remove from:
ASSERT(child->Parent().get() == this);
This is most likely caused by modules not being fully created yet when slots are being added (i.e. made available) to modules in the constructor. Either this has to be fixed, or the assert needs to be replaced with a more appropriate safety check.
Maybe this issue will be automatically solved by the new graph?

It seems the CoreInstance is currently not using SetSlotUnavailable() at all to manage the graph.
The function gets called by the following modules only: CinematicView, ReplacementRenderer, SphereRenderer.
The new graph will likely not bother to support this functionality. We want to get rid of child/parent stuff in modules and calls ASAP.
I would be happy to look at a MegaMol project that triggers the mentioned ASSERT, as the other calls of the function seem to work just fine (?).

Having a quick look at the modules, I am certain the problem only occurs if calling SetSlotUnavailable() in the constructor. The modules, in which it works, call the function from within Render() or create().
I already used SetSlotUnavailable() in my private repository when I do not want the parent's slots to be available in my module. Doing so in the constructor triggers the assert, but apart from this seems to have no negative effect whatsoever when running in Release (without asserts enabled). This probably means that the "sanity" check can be removed. Can someone confirm this theory?


Enhancements on hostMetrics GraphQL field

Dates (from, to) should be GraphQLDateTime
Fields should be resolved individually, to prevent computing everything when you're just interested in one field like for https://github.com/opencollective/opencollective-frontend/blob/11f51deae32db7b172a7716779f56be177ad3717/components/edit-collective/sections/HostPlan.js#L178

Alternatively, the second item could be achieved like mentioned in https://github.com/opencollective/opencollective/issues/4586#issuecomment-912416325


Feature/0004 equipment maint

Various changes to Equipment Master

I screwed up a merge / rebase so created an additional branch with changes applied https://github.com/Acumatica/CMMS-Lite/pull/30


UI for import-map-overrides

Should we build a UI for import map overrides, such as the one shown in the following gif:

Sounds good to me! :)

The next release of single-spa-inspector will also support doing overrides when using this library, if that helps too :)


README Spanish translation.

I added a README-es.md file, which is a translation to Spanish of the original README, also I added a direct link in the README file, this is for our huge Hispanic community, Thanks!

I think these translations should go into tensorflow.org website. We cannot create readme files for each language in the world and expect them to be in sync.


New PR to test SignCLA at 2019-06-28 14:15:48.00129429 -0700 PDT m=+8.831256941

Body of PR

unsigned-fail-staging: test succeeded


Added firebase-client typescript definition file

Created definition file for firebase-client (https://github.com/jpstevens/firebase-client) - a simple Firebase REST wrapper that uses promises.

@fpsscarecrow Thanks mate!


Avoid blank OAI xml elements

@jacobthill the issue with Texas Tech is that in at least one record they provide an empty publisher element. So we get something like:
<dc:publisher>Digitized by the Southwest Collection/Special Collections Library.</dc:publisher>
<dc:publisher/>

This PR checks if there are any keys once we flatten the value.

@edsu yeah, I definitely need to address the tests for this. The issue I'm seeing with get_tag is that it's not providing the full tag. For instance, it returns 035 when we want 035_a which happens in the flatten method, but originally that was after inserting into result, so we ended up with only the last result not all results when there were multiples.
I'd be happy to chat about this if you have any better ideas :-)

@edsu I think I've addressed your comments and added a bit of data to the example data to extend the test.

Thanks @aaron-collier!


Exclusive Option ArgGroup is enforced even when help.option is supplied

I just started to try out the new 4.0.0 alpha build, the new argument group feature comes in very handy for me. I came across the following issue, though:
Minimal code sample:
public class ArgGroupsTest {

    public static void main(String[] args) {
        CommandLine cmd = new CommandLine(new TestCommand());
        cmd.parseWithHandler(new CommandLine.RunAll(), args);
    }

    @Command(subcommands = { TestSubCommand.class, CommandLine.HelpCommand.class }, name = "ArgGroupsTest", mixinStandardHelpOptions = true, version = "0.xx")
    static class TestCommand implements Runnable {

        @ArgGroup(exclusive = true, multiplicity = "1")
        DataSource datasource;

        static class DataSource {
            @Option(names = "-a", required = true)
            static boolean isA;
            @Option(names = "-b", required = true)
            static File dataFile;
        }

        @Override
        public void run() {
            // nothing to do here
        }
    }

    @Command(name = "test", showDefaultValues = true)
    static class TestSubCommand implements Runnable {

        @Override
        public void run() {
            System.out.println("Subcommand called!");

        }
    }
}

Now, when requesting help:
$ ArgGroupsTest --help

the following message is presented on stderr:
Error: Missing required argument (specify one of these): -a, -b=<dataFile>
Usage: ArgGroupsTest (-a | -b=<dataFile>) [-hV] [COMMAND]
  -a
  -b=<dataFile>
  -h, --help       Show this help message and exit.
  -V, --version    Print version information and exit.
Commands:
  test
  help  Displays help information about the specified command

Once help is requested, usage instructions should be given on stdout, I don't expect any error due to a missing required mutually exclusive option here.

Ouch! Good catch! I agree this is not expected behavior. I’ll look at this as soon as I can.

I found the cause. Basically, all validation should be skipped if any help is requested. Currently only some validation is skipped but not all. The validation for the arg groups is among the ones that are not skipped.
This should fix it:
Index: src/main/java/picocli/CommandLine.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- src/main/java/picocli/CommandLine.java	(date<PHONE_NUMBER>285)
+++ src/main/java/picocli/CommandLine.java	(date<PHONE_NUMBER>285)
@@ -8852,11 +8852,13 @@
                 }
             } while (!argumentStack.isEmpty() && continueOnError);

-            validateConstraints(argumentStack, required, initialized);
+            if (!isAnyHelpRequested()) {
+                validateConstraints(argumentStack, required, initialized);
+            }
         }

         private void validateConstraints(Stack<String> argumentStack, List<ArgSpec> required, Set<ArgSpec> matched) {
-            if (!isAnyHelpRequested() && !required.isEmpty()) {
+            if (!required.isEmpty()) {
                 for (ArgSpec missing : required) {
                     Assert.assertTrue(missing.group() == null, "Arguments in a group are not necessarily required for the command");
                     if (missing.isOption()) {

I'll push this when I get a chance.


OADP v1.1.3 updates

Update lab to version v1.1.3 and address issues filed by users:

https://github.com/redhat-cop/agnosticd/issues/6265

Lab feedback by User

1.1.4 is out now.


bug: Example for useWaitForTransaction? [not working]

Is there an existing issue for this?

[X] I have searched the existing issues

Package Version
0.3.7
Current Behavior
Could you provide an example for useWaitForTransaction? I don't know why but I cannot make it work. Maybe it's the way the use-wagmi refs are interacting with eachother... or am I calling the methods incorrectly/in the wrong order? I tried lots of stuff but could not make it work. Here's an example... the watchers are never called after I executed the transaction in metamask.
Thanks
<template>
    <div>
        <button @click="createToken">
            Mint (0.0033 ETH)
        </button>
    </div>
</template>

<script lang="ts" setup>
import { useContractWrite, useNetwork, useWaitForTransaction } from 'use-wagmi'
import { parseEther } from 'viem'

const { chain } = useNetwork()
const { hashCallsControllerContract, getContractAddress } = useBlockchain()
const controllerAddress = (chain?.value !== undefined) ? getContractAddress(hashCallsControllerContract, chain.value) : null
const controllerAbi = hashCallsControllerContract.abi
const { data: mintResult, write: mint } = useContractWrite({
    address: controllerAddress,
    abi: controllerAbi,
    functionName: 'mint',
    value: parseEther('0.0033')
})
const { isLoading, isSuccess } = useWaitForTransaction({
    hash: mintResult.value?.hash
})

watch(mintResult, (newValue:any) => {
    console.log('mintResult: ' + newValue)
})
watch(isLoading, (newValue:any) => {
    console.log('isLoading: ' + newValue)
})
watch(isSuccess, (newValue:any) => {
    console.log('isSuccess: ' + newValue)
})
watchEffect((newSomething:any) => {
    console.log(newSomething)
})

const createToken = async () => {
    const { data: tokenMetadata } = await useFetch('/api/createToken', { method: 'POST' })
    if (!tokenMetadata.value) { console.error('could not create token metadata'); return }
    mint({ args: [tokenMetadata.value.tokenMetadataHash, tokenMetadata.value.metadataAuthChecksum] })
}
</script>

<style scoped>

</style>

Expected Behavior
No response
Steps To Reproduce
No response
Link to Minimal Reproducible Example (StackBlitz, CodeSandbox, GitHub repo etc.)
No response
Anything else?
No response

I think the problem lies in the fact that the method useWaitForTransaction is not updating when the mintResult is updated... But I can also not use a watcher since then I get this error:
Uncaught (in promise) Error: useConfigmust be used withinUseWagmiPlugin.
watch(mintResult, (newValue:any) => {
    console.log('mintResult hash: ' + newValue?.hash)
    const { isLoading, isSuccess } = useWaitForTransaction({
        hash: newValue?.hash
    })
    mintIsLoading = isLoading
    mintIsSuccess = isSuccess
})
watch(mintIsLoading, (newValue:any) => {
    console.log('isLoading: ' + newValue)
})
watch(mintIsSuccess, (newValue:any) => {
    console.log('isSuccess: ' + newValue)
})

I fixed the issue by using viem's waitForTransactionReceipt method. (which I can use in the mintResult watcher)
const waitForTransaction = async (hash:`0x${string}`) => {
        const transaction = await publicClient.waitForTransactionReceipt({ hash })
        return transaction
}

@dataexcess Is the hash you pass to useWaitForTransaction reactive

well.. no it expects a string? Can you give an example how to use a reactive value?
Do you mean like this?
const { data: mintResult, write: mint } = useContractWrite({
    address: controllerAddress,
    abi: controllerAbi,
    functionName: 'mint',
    value: parseEther('0.0033')
})

const transactionHash = ref<string>("")

watch(mintResult, (newValue:any) => {
    console.log('mintResult: ' + newValue)
    transactionHash.value = newValue.hash
})

const { isLoading, isSuccess } = useWaitForTransaction({
    hash: transactionHash
})

@dataexcess
const { data: mintResult, write: mint } = useContractWrite({
    address: controllerAddress,
    abi: controllerAbi,
    functionName: 'mint',
    value: parseEther('0.0033')
})

const { isLoading, isSuccess } = useWaitForTransaction({
    hash: computed(() => mintResult?.value.hash)
})


[WIP] refactor operator group cluster role name

Description of the change:
In its current implementation, OLM creates three cluster roles for and operator-group: -admin, -view, and -edit.
Motivation for the change:
OCPBUGS-14698
Architectural changes:
The cluster role name format was changed to: olm.operatorgroup.{admin|edit | view}
Testing remarks:
When this hits a running cluster, it will abandon the currently existing cluster roles in favor of new ones that respect the new format. This means that additional migration information will need to be provided in documentation.
The unit tests check that the appropriate cluster roles are created whether or not there is a currently existing. E2e tests responsible for the cluster role creation were also updated for the new format.
Reviewer Checklist

[ ] Implementation matches the proposed design, or proposal is updated to match implementation
[ ] Sufficient unit test coverage
[ ] Sufficient end-to-end test coverage
[ ] Bug fixes are accompanied by regression test(s)
[ ] e2e tests and flake fixes are accompanied evidence of flake testing, e.g. executing the test 100(0) times
[ ] tech debt/todo is accompanied by issue link(s) in comments in the surrounding code
[ ] Tests are comprehensible, e.g. Ginkgo DSL is being used appropriately
[ ] Docs updated or added to /doc
[ ] Commit messages sensible and descriptive
[ ] Tests marked as [FLAKE] are truly flaky and have an issue
[ ] Code is properly formatted

This means that additional migration information will need to be provided in documentation.

Are we taking a new major version here?

Yeah. Now we have the philosophical disccusion of whether the ClusterRole name is part of the API, or an implementation detail XD. I guess these ClusterRoles are the way to aggregate permissions across the operators in a namespace and are created for that purpose. So, I suppose the naming format would also be part of the API. So, breaking change.

So the main takeaways are that this change:

Prefix’s the k8s resources with olm.operatorgroup.
Adds a random suffix to the k8s resources based on the UID of the OG
-- Names look like olm.operatorgroup.(view|edit|admin)-######
Existing OG’s view, admin, edit clusterRoles/clusterRoleBindings will be abandoned. It would be difficult to ID these resources with Kubectl and without OLM specific knowledge

My initial thoughts:

Changing the name is fine, but basing it on the UID makes the OG’s name unpredictable. Could we sha the spec of the namespace/name which should always be unique for OGs due to the k8s api?
-- Changing the name places existing clusterRole(Binding)s into three categories:
--- Cat 1: resources that OLM owned alone.
---- OUTCOME: Abandoned resource that needs cleanup
--- Cat 2: resources that OLM was fighting another controller over.
---- OUTCOME: Other controller wins and we’re good.
---Cat 3: resources that were created through a third party but isn’t watched by a controller and OLM updated.
---- OUTCOME: Basically the same as 2, left in state that OLM update it to BUT something is probably broken.
I would like to provide customers with guidance to perform the manual migration:
-- We could have solved this earlier by keeping track of resources that OLM creates and not updating resources created by third parties. Steve recently introduced a PR that adds some labels to non-OLM api resources created by OLM. This doesn’t solve our migration issue, but we should consider using labels and not updating label-less resources as a best practice going forward.
-- The question now becomes how will users identify what resources to look at. Some options:
--- We provide a script that a customer points at a cluster, it finds existing OGs on cluster, and it prints out the clusterRole(Binding)s that need to be inspected and possibly deleted.
-- Alternatively, we could perform a slow rollout in which we introduce labels and then change the name. I don’t see much value here since we can map existing OGs -> resources

@awgreene good call out on Steve's PR. I'll rebase and add that label to the role, if he hasn't already. We've created our own "owner refs" as labels on the ClusterRole. So, it should be straightforward to identify the ClusterRoles. Using ns/name was my first thought for a hash - @kevinrizza called out that it could leak information (i.e. people could use guessing attacks). The resource ID is stable and should lead to predictable values (at least for the lifetime of the object). wdyt?

In a lot of the test code, there's odd white-space changes, which can probably be undone.

Still looking to get those unnecessary formatting changes in operator_test.go reverted.

Closing due to #3035


Update tooltip content and styling on update()

Fixes issue #4991
Another attempt at this issue, credits go to @kurkle.
The tooltip is now updated based on the new data that is visible under the mouse and hides if that point is no longer visible.
To do this the last mouse event is stored and passed to the eventhandler upon update. The data for the tooltip is updated during transition as well as the hover styling of the point.
For a working example of this PR: https://codepen.io/Evertvdw/pen/PgqbWq
Current behaviour: https://codepen.io/Evertvdw/pen/QPbdLQ

Found a mistake still that did not set lastActive to an empty initially. This resulted in #3791 breaking again, so needed to fix it.

I believe there was an objection to calling me.eventHandler in https://github.com/chartjs/Chart.js/pull/5817 and it looks like that's still being done. Is it possible to call tooltip.update instead? https://github.com/chartjs/Chart.js/pull/5817#issuecomment-471594189

I believe there was an objection to calling me.eventHandler in #5817 and it looks like that's still being done. Is it possible to call tooltip.update instead? #5817 (comment)

I don't think its possible.

@kurkle can you clarify why you don't think it's possible?

@kurkle can you clarify why you don't think it's possible?

To clarify, it certainly is possible, it just does not work correctly 😏
And the reason is: tooltip._active array might contain items that are no longer available and / or under the mouse.
So those active items need to be re-evaluated and that's what eventHandler does (for both hover and tooltip).

I guess this PR basically does what was proposed here:
https://github.com/chartjs/Chart.js/issues/4991#issuecomment-346949948

@Evertvdw will you be able to address @nagix's comments?

@Evertvdw will you be able to address @nagix's comments?

Sorry I forgot about this for a bit, but yes I think I can store the lastEvent only if that is not a click of course. About his first comment I need to look into that, but don't have the time right now.

Updated codepen: PR 6181 Updated

I'd love to see this merged. Require it for project with a constantly updating dataset.
Thanks for all your hard work @Evertvdw

@benmccann @nagix can you review this?

It looks fine to me though I'm not very familiar with this part of the code, so I'll defer to @etimberg @nagix @kurkle on this one. Thanks for having been patient on this one and for following up on it

Yes, updated the PR pen: https://codepen.io/Evertvdw/pen/PgqbWq

Hi,
can someone fix the codeclimate issue and force the merge.
I' also waiting for this update. :-)

@Evertvdw thanks so much for your patience. I think this PR is ready to be merged since it has been approved by @kurkle, @etimberg, and myself. Unfortunately there's a merge conflict. Would you be able to rebase this PR against master so we can merge it?

Rebased and merged in https://github.com/chartjs/Chart.js/pull/6635

@benmccann Sorry, I forgot about it again, but thanks for doing it!


[E2E] Torchbench hf_distil_whisper amp_bf16 training accuracy failed

🐛 Describe the bug
torchbench_amp_bf16_training
xpu  train hf_distil_whisper
Traceback (most recent call last):
File "/home/sdp/actions-runner/_work/torch-xpu-ops/pytorch/benchmarks/dynamo/common.py", line 4626, in run
) = runner.load_model(
File "/home/sdp/actions-runner/_work/torch-xpu-ops/pytorch/benchmarks/dynamo/torchbench.py", line 302, in load_model
benchmark = benchmark_cls(
File "/home/sdp/actions-runner/_work/torch-xpu-ops/benchmark/torchbenchmark/util/model.py", line 39, in call
obj = type.call(cls, *args, **kwargs)
File "/home/sdp/actions-runner/_work/torch-xpu-ops/benchmark/torchbenchmark/models/hf_distil_whisper/init.py", line 12, in init
raise NotImplementedError("Training is not implemented.")
NotImplementedError: Training is not implemented.
model_fail_to_load
loading model: 0it [00:00, ?it/s]
loading model: 0it [00:00, ?it/s]
Versions
torch-xpu-ops: https://github.com/intel/torch-xpu-ops/commit/1d70431c072db889d9a47ea4956049fe340a426d
pytorch: d224857b3af5c9d5a3c7a48401475c09d90db296
device: pvc 1100, bundle: 0.5.3, driver: 803.61

Not xpu issue


PLOTTING: color alignment with service(Discovery,SpaceClaim)

As part of the JSON information, whenever it is send back, we also get information on the color of each body/component. We should make use of it when plotting.

Partially possible via #1357 - now we have to implement the plotter logic to use this color value.


GEOMESA-2021,GEOMESA-339 Use InputStreams in converters

Deprecated convert.SimpleFeatureConverter[T], replaced with convert2.SimpleFeatureConverter
Converter instances are wrapped appropriately so old and new API are both picked up
Converters operate on input streams and not type bound instances (e.g. strings)
** Quoted newlines are supported in CSV parsing
** JSON no longer requires multi-line or single-line option
Refactored expressions and changed parsing to parboiled
AbstractConverters are based on pureconfig to allow conversion to/from config

Signed-off-by: Emilio Lahr-Vivaz<EMAIL_ADDRESS>

working on going through this again today
[ERROR] Failures:
[ERROR]   KafkaDataStoreTest.KafkaDataStore should::write/update/read/delete features There is 1 failure
Stream(LazyImmutableSimpleFeature:jo:jones|20|Mon Jan 02 00:00:00 UTC 2017|POINT (-10 -10), LazyImmutableSimpleFeature:sm:smith|30|Sun Jan 01 00:00:00 UTC 2017|POINT (0 0))
  is missing: ScalaSimpleFeature:sm:smith2|32|Sun Jan 01 00:00:02 UTC 2017|POINT (2 2)
  must not contain: LazyImmutableSimpleFeature:sm:smith|30|Sun Jan 01 00:00:00 UTC 2017|POINT (0 0)

that's an intermittent failure that has started cropping up in master :/
i kicked the build, let's see if it works this time

@jahhulbert-ccri how's this one looking?

A few questions...  1.  Do the jars that an end user would use change?

Can you point to where and how we are using Shapeless?

Do we have any options around the convert -> convert2 move?  Can you say more about that?

jars do not change
shapeless is used by pureconfig for auto-converting type classes. we don't use it directly
I wanted to re-use class names, so created a new package to hold the new traits. The old ones need to stay unchanged (and deprecated) for semver. I think for 3.0 we should move everything back into the main convert package


[BUG] DNS resolution for external domain names (Internet) not working on pods

What did you do
I created a default k3d cluster and deployed pods on it. The pods are not able to resolve external dns. I tried setting K3D_FIX_DNS to 0 and 1, both made no difference, this example used default.

How was the cluster created?

k3d cluster create dnstest

What did you do afterwards?

kubectl apply -f https://k8s.io/examples/admin/dns/dnsutils.yaml

vagrant@ubuntu-10032023:~/testcluster$ kubectl exec -it dnsutils -- nslookup google.com
Server:		<IP_ADDRESS>
Address:	<IP_ADDRESS>#53

** server can't find google.com: NXDOMAIN
command terminated with exit code 1

error message in coredns pod logs:
vagrant@ubuntu-10032023:~/testcluster$ kubectl logs coredns-76f668cf94-rxgvs -n kube-system
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.server
.:53
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.server
[INFO] <IP_ADDRESS>:38206 - 26358 "HINFO IN<PHONE_NUMBER>883247646.1506387703770981152. udp 57 false 512" NXDOMAIN qr,aa,rd 132 0.000832127s
[INFO] plugin/reload: Running configuration SHA512 = fd586de816c2c35b2d8a1c5ceb51dda557a14f33879cb49b6c6a115bc61f862d3cdad6881dd13478e0a77a6f5267199b6f43500da4783b193415947413fb64e3
CoreDNS-1.10.1
linux/amd64, go1.20, 055b2c3
[INFO] <IP_ADDRESS>:48569 - 3595 "A IN google.com.default.svc.cluster.local. udp 54 false 512" NXDOMAIN qr,aa,rd 147 0.000183376s
[INFO] <IP_ADDRESS>:44383 - 41899 "A IN google.com.svc.cluster.local. udp 46 false 512" NXDOMAIN qr,aa,rd 139 0.000093384s
[INFO] <IP_ADDRESS>:45401 - 20902 "A IN google.com.cluster.local. udp 42 false 512" NXDOMAIN qr,aa,rd 135 0.000095355s
[INFO] <IP_ADDRESS>:41790 - 244 "A IN google.com. udp 28 false 512" NXDOMAIN qr,aa,rd 103 0.000072917s
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.server

cluster dns points to coredns service:
vagrant@ubuntu-10032023:~/testcluster$ kubectl exec -it dnsutils -- cat /etc/resolv.conf
search default.svc.cluster.local svc.cluster.local cluster.local
nameserver <IP_ADDRESS>
options ndots:5

k3d cluster running on one server node container with lb
vagrant@ubuntu-10032023:~/testcluster$ docker container ls
CONTAINER ID   IMAGE                            COMMAND                  CREATED         STATUS         PORTS                             NAMES
bcee24ec17ff   ghcr.io/k3d-io/k3d-proxy:5.7.0   "/bin/sh -c nginx-pr…"   3 minutes ago   Up 3 minutes   80/tcp, <IP_ADDRESS>:38263->6443/tcp   k3d-dnstest-serverlb
b79f010fe5e5   rancher/k3s:v1.29.6-k3s1         "/bin/k3d-entrypoint…"   3 minutes ago   Up 3 minutes                                     k3d-dnstest-server-0

the coredns pod does seem to resolve correctly to the underlying docker network gateway ip <IP_ADDRESS>
vagrant@ubuntu-10032023:~/testcluster$ kubectl debug -it coredns-66c56f4556-dgm46  -n kube-system --image registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3 --target coredns -n kube-system
Targeting container "coredns". If you don't see processes from this container it may be because the container runtime doesn't support this feature.
Defaulting debug container name to debugger-v99vb.
If you don't see a command prompt, try pressing enter.
root@coredns-66c56f4556-dgm46:/#
root@coredns-66c56f4556-dgm46:/# cat /etc/resolv.conf
search eu.pg.com
nameserver <IP_ADDRESS>
options ndots:0

this IP also resolves.

vagrant@ubuntu-10032023:~/testcluster$ kubectl get nodes -owide
NAME                    STATUS   ROLES                  AGE   VERSION        INTERNAL-IP   EXTERNAL-IP   OS-IMAGE           KERNEL-VERSION      CONTAINER-RUNTIME
k3d-dnstest4-server-0   Ready    control-plane,master   20d   v1.29.6+k3s1   <IP_ADDRESS>    <none>        K3s v1.29.6+k3s1   5.15.0-69-generic   containerd://1.7.17-k3s1

the k3d docker node container resolves correctly
vagrant@ubuntu-10032023:~/testcluster$ docker exec -it k3d-dnstest-server-0 nslookup google.com
Server:		<IP_ADDRESS>
Address:	<IP_ADDRESS>:53
Non-authoritative answer:

Non-authoritative answer:
Name:	google.com
Address: <IP_ADDRESS>

`the underlying virtualbox machine (spawned by vagrant) also resolves correctly:
the machine ```ning on a company laptop
vagrant@ubuntu-10032023:~/testcluster$ nslookup google.com
Server:		<IP_ADDRESS>
Address:	<IP_ADDRESS>#53

Non-authoritative answer:
Name:	google.com
Address: <IP_ADDRESS>
Name:	google.com
Address: 2a00:1450:4001:810::200e

as a test, I tried to change the forward configuration of the coredns configmap directly to docker gatway 172.19..01 or alternativel <IP_ADDRESS> but that did not change anything as well
vagrant@ubuntu-10032023:~/testcluster$ kubectl get cm coredns -n kube-system -oyaml
apiVersion: v1
data:
  Corefile: |
    .:53 {
        errors
        health
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
          pods insecure
          fallthrough in-addr.arpa ip6.arpa
        }
        hosts /etc/coredns/NodeHosts {
          ttl 60
          reload 15s
          fallthrough
        }
        prometheus :9153
        forward . <IP_ADDRESS>
        cache 30
        loop
        reload
        loadbalance
        import /etc/coredns/custom/*.override
    }
    import /etc/coredns/custom/*.server
  Nodehosts: |
    <IP_ADDRESS> k3d-dnstest4-server-0
kind: ConfigMap
metadata:
  annotations:
.....

What did you expect to happen
I expect the nslookup command to run/resolve properly for external DNS in every cluster pod
Which OS & Architecture
vagrant@ubuntu-10032023:~/testcluster$ uname -a
Linux ubuntu-10032023 5.15.0-69-generic #76-Ubuntu SMP Fri Mar 17 17:19:29 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
vagrant@ubuntu-10032023:~/testcluster$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.2 LTS
Release:	22.04
Codename: jammy

output of k3d runtime-info

vagrant@ubuntu-10032023:~/testcluster$ k3d runtime-info

arch: x86_64
cgroupdriver: systemd
cgroupversion: "2"
endpoint: /var/run/docker.sock
filesystem: extfs
infoname: ubuntu-10032023
name: docker
os: Ubuntu 22.04.2 LTS
ostype: linux
version: 27.0.3

Which version of k3d
vagrant@ubuntu-10032023:~/testcluster$ k3d version
k3d version v5.7.0
k3s version v1.29.6-k3s1 (default) ```

## Which version of docker
 ```vagrant@ubuntu-10032023:~/testcluster$ docker version
Client: Docker Engine - Community
 Version:           27.0.3
 API version:       1.46
 Go version:        go1.21.11
 Git commit:        7d4bcd8
 Built:             Sat Jun 29 00:02:33 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          27.0.3
  API version:      1.46 (minimum version 1.24)
  Go version:       go1.21.11
  Git commit:       662f78c
  Built:            Sat Jun 29 00:02:33 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.18
  GitCommit:        ae71819c4f5e67bb4d5ae76a6b735f29cc25774e
 runc:
  Version:          1.7.18
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

potentially related
https://github.com/k3d-io/k3d/issues/1515

there is a change which happened on coredns switching to usage of coredns-custom which seems to create the problem. Clusters spawned with k3d >=5.7.0 are using coredns-custom configmap which shoots the coredns based cluster pod external FQDN resolutions
https://github.com/k3d-io/k3d/releases/tag/v5.7.0
https://github.com/k3d-io/k3d/compare/v5.6.3...v5.7.0
https://github.com/k3d-io/k3d/commit/71b5755ebdb3c02e7b82665e88f50e316adec311
when you remove the import entry from the coredns cm, it is working. The new coredns-custom configuration seems to have a hiccup somewhere, can someone pls take a look?
When I downgrade on the same cluster to 5.6.3 and run the same cluster creation, I have no challenges.

I have seen that issues have already been reverted in > 5.7.0, hence closing.


[rlgl] Custom shaders called with rlLoadDrawQuad() cannot fetch data from texture uniforms.

Issue description
TL;DR: Shaders made with rlLoadDrawQuad() can't seem to reliably access sampler2D uniforms via texture or texelFetch.
In my program I render a 3D scene into a RenderTexture. I then call rlLoadDrawQuad with a custom shader to load the previous texture, filter it and render to another RenderTexture, which I then output to the screen. However, it seems rlLoadDrawQuad cannot access uniform texture data when it is being called after BeginTextureMode(); removing that and outputting directly into the screen fixes this, but then I can't do further operations with that RenderTexture.
However, when attempting to create a small code sample to share here, I couldn't get rlLoadDrawQuad() to load a texture at all, even when outputting directly to the screen. Everything else in the shaders work correctly, the uniform location is found and the texture coordinates display fine onscreen, but all shader data is black no matter what texture it tries to access despite the image being loaded. I'm not sure where the problem is at this point to be honest.
Environment
Desktop (GLFW)
Windows 10 64-bit
GPU NVIDIA GeForce RTX 3050 Laptop GPU/PCIe/SSE2
OpenGL:  3.3.0 NVIDIA 537.13
GLSL: 3.30 NVIDIA via Cg compiler
Code Example
The code below should output whatever image you load up as example.png, but instead draws blank.
main.c:
#include "raylib.h"
#include "rlgl.h"

int main()
{
    InitWindow(1600, 900, "Test");

    Texture testTexture = LoadTexture("example.png");
    Shader testShader = LoadShader("rendertexture_VP.glsl", "rendertexture_FP.glsl");

    SetTargetFPS(60);
    while (!WindowShouldClose())
    {
        BeginDrawing();
            ClearBackground(BLACK);
            int loc = GetShaderLocation(testShader, "source");
            SetShaderValueTexture(testShader, loc, testTexture);
            rlEnableShader(testShader.id);
            rlLoadDrawQuad();
            rlDisableShader();
        EndDrawing();
    }

    CloseWindow();
}

rendertexture_VP.glsl:
#version 330

layout (location = 0) in vec3 vertexPosition;
layout (location = 1) in vec2 vertexTexCoord;

out vec2 fragTexCoord;

void main()
{
    fragTexCoord = vertexTexCoord;
    gl_Position = vec4(vertexPosition, 1.0f);
}

rendertexture_FP.glsl:
#version 330

in vec2 fragTexCoord;

uniform sampler2D source;

out vec4 finalColor;

void main()
{
    vec3 texColor = texture(source, fragTexCoord).xyz;
    finalColor = vec4(texColor, 1.0f);
}

@mikeemm Considering you are using rlgl, you should use rlSetShader(unsigned int id, int *locs) to enable shader locations, or do the process manually.
Note that rlgl is a lower level API, just a thin layer over OpenGL, but still has some custom functionality.

@raysan5 I tried that before, and neither rlSetShader() nor BeginShaderMode() (which just calls rlSetShader() internally) worked. Adding either of these in the example I provided does not fix the issue for me, as the shader still refuses to access the texture. Please don't close this issue quite yet.

@mikeemm Ok, reopening it.

Turns out before you set the sampler uniform you have to bind the texture using rlEnableTexture(); I assume this worked at first in my personal project because at some point during the rendering phase something in my code already bound the texture and left it bound.
However, the other reason this doesn't work is because SetShaderValueTexture() calls rlSetUniformSampler(), which calls glUniform1i(locIndex, 1 + i). This assumes that there was already one texture bound at texture unit 0 before binding the one you'd actually want to use; again, I assume this worked for me at first because a previous operation left a texture bound on unit 0 without adding its id to the RLGL state. Using SetShaderValue() instead with an explicit texture unit solves this.
So this doesn't work:
BeginDrawing();
    rlEnableTexture(testTexture.id);
    int texLoc = GetShaderLocation(testShader, "source");
    SetShaderValueTexture(testShader, texLoc, testTexture);
    rlLoadDrawQuad();
    rlDisableShader();
EndDrawing();

But this does:
BeginDrawing();
    rlEnableTexture(testTexture.id);
    int texUnit = 0;
    int texLoc = GetShaderLocation(testShader, "source");
    SetShaderValue(testShader, texLoc, &texUnit, SHADER_UNIFORM_SAMPLER2D);
    rlLoadDrawQuad();
    rlDisableShader();
EndDrawing();

This behavior must be intentional so that it works together with other drawing functions, which seem to set a default texture to texture unit 0 - my question is, where and why is this texture added? It seems incoherent that SetShaderValueTexture() would work or not depending on where in the program it was being used, with no way to know whether it would work until runtime. Is the reasoning for this behavior already documented somewhere perhaps?


📝 Fix typo in docs/en/docs/advanced/behind-a-proxy.md

Thanks @Leommjr! :mag: :cake:
And thanks for the review and comment @Ryandaydev :bow:


update(Strings): Organize all strings - UL-46

Description

Organize the strings in one file in Main App and one file in UI_Library

Strings in the UI Library that don't need to be translated, were left in their file, like:

Type of Change

[ ] ✨ New feature (non-breaking change which adds functionality)
[ ] 🛠 Bug fix (non-breaking change which fixes an issue)
[ ] ❌ Breaking change (fix or feature that would cause existing functionality to change)
[x] 🧹 Code refactor
[ ] ✅ Build configuration change
[ ] 📝 Documentation
[ ] 🧪 Tests
[ ] 🗑 Chore

neat 🎉


DTXClient standalone

Awesome presentation and repo... is it possible to change the DTX Client standalone code so it doesnt have to require IDA and the sdk? I'd just like to communicate with the Instruments server without IDA. Any help is appreciated!

It is very possible, though I don't know if I will have time to do it.
It seems the client code only relies on the IDA SDK for the qstring and bytevec_t classes. STL has basically the equivalent of those, with std::string and std::vector<uint8_t>. Try copying dtxmsg_common.(cpp,h) and dtxmsg_client.(cpp,h) to somewhere else, and replace all instances of qstring with std::string and all instances of bytevec_t with std::vector<uint8_t> (of course, they don't have the exact same behavior so you may have to massage it a little).
I bet it's easier than you think!

Thanks Troy. I appreciate the help. I’ve got everything converted over but I’m having issues with make channel. I’ve got preform handshake working great but make channel is always coming back with a -1 / err being shown as broken pipe. I’ve checked the code so many times now but I’m out of ideas. Hoping you might have some insight

Actually I take that back... my send message seems to be wrong because the instruments server seems to always send the capabilities and channels regardless even if I don’t send any message and just call receive... the issue is probably in my send message then but still don’t know what the issue is.
Do you know how many bytes the messages you typically send are?

Actually never mind, got it working... had to stay completely c++
Something must be different with how objective C is packing bytes or order. Thanks Troy

Glad you got it working! Let me know if you spot any issues :)

@neildufva ll you be able to share your working ?
coz i m also looking for DTX Client standalone code.
To make it work without IDA
Thanks

@neildufva @troybowman hi i need help in some areas , i have converted project but stuck at some places


[BugFix] Fix rank as reserved keyword bug

What type of PR is this：

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes #https://github.com/StarRocks/starrocks/issues/17839
Problem Summary(Required) ：

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[ ] I have checked the version labels which the pr will be auto backported to target branch

[x] 2.5
[ ] 2.4
[ ] 2.3
[ ] 2.2

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.27 out of 28 committers have signed the CLA.:white_check_mark: evelynzhaojie:white_check_mark: EsoragotoSpirit:white_check_mark: amber-create:white_check_mark: xiaoyong-z:white_check_mark: stdpain:white_check_mark: GavinMar:white_check_mark: ZiheLiu:white_check_mark: sparklezzz:white_check_mark: nshangyiming:white_check_mark: ABingHuang:white_check_mark: satanson:white_check_mark: Youngwb:white_check_mark: kevincai:white_check_mark: decster:white_check_mark: silverbullet233:white_check_mark: zuyu:white_check_mark: Seaven:white_check_mark: Astralidea:white_check_mark: stephen-shelby:white_check_mark: miomiocat:white_check_mark: packy92:white_check_mark: meegoo:white_check_mark: mofeiatwork:white_check_mark: shshenhua:white_check_mark: bestolap:white_check_mark: kangkaisen:white_check_mark: hellolilyliuyi:x: mergify[bot]You have signed the CLA already but the status is still pending? Let us recheck it.

@Mergifyio backport branch-2.5


Icon for light/dark theme

Icon is orange that works with light/dark theme. We can do better:

Provide a icon for light/Dark theme
Orange icon can be used to signal an active aws console tab

Orange (FC9700) icon looks pretty good on light and dark background: https://webaim.org/resources/contrastchecker/?fcolor=FC9700&bcolor=828282