Amplify-Swift does not update local data on mutations, or creation

Describe the bug
I have been working on a simple implementation of Appsync/GraphQL/DataStore in my app, and have successfully got it pulling in initial data from the database on app launch, but when data is mutated or new items are created, I get this error in the Xcode console in the App (iOS Simulator)
2023-06-08 22:05:19.673116-0500 Airboss[13135:4618954] [IncomingAsyncSubscriptionEventToAnyModelMapper] The operation couldn’t be completed. (Amplify.GraphQLResponseError<AWSPluginsCore.MutationSync<AWSPluginsCore.AnyModel>> error 0.)

If I close the simulator and re-launch it, all of the items, including the expected additions are there.  It's just not updating things in realtime.
The mutations are being triggered through a direct HTTP POST request to the Graphql api endpoint.
Here is a sample of the mutation for a "createFlight"
mutation createFlight {
  createFlight(input: {
    _version: 0,
    fltId: "644809278df1c179f926e07e",
    customerId: "123",
    origin: "ABCD",
    destination: "ABDE",
    flightNumber: "418",
    receivedArrivalMessage: false,
    receivedDepartureMessage: false,
    scheduledOut: "1686229200000",
    scheduledOff: "1686229500000",
    scheduledOn: "1686241500000",
    scheduledIn: "1686241800000",
    actualOut: "1686229200000",
    actualOff: "1686229500000",
    actualOn: "1686241500000",
    actualIn: "1686241800000",

  }) {
    fltId
    customerId
    origin
    destination
    flightNumber
    receivedArrivalMessage
    receivedDepartureMessage
    scheduledOut
    scheduledOff
    scheduledOn
    scheduledIn
    actualOut
    actualOff
    actualOn
    actualIn
    _deleted
    _version
    _lastChangedAt
  }
}

It should be noted that this mutation does complete successfully, as in when I look at the data in the DynamoDB Table, it is in there.  However, in the XCode Console, it is just returning this error, while listening in real time.  A simulator re-launch, and it pulls in all of the correct data.
Here is the schema:
type Flight @model {
  fltId: String! @primaryKey( sortKeyFields:["customerId"])
  customerId:String! @index(name: "byCustomerAndOutTime", sortKeyFields:["scheduledOut"],queryField: "flightsByCustomerAndOutTime")
  origin: String!
  destination: String!
  flightNumber: String!
  scheduledOut: String!
  scheduledOff: String!
  scheduledOn: String!
  scheduledIn: String!
  actualOut: String!
  actualOff: String!
  actualOn: String!
  actualIn: String!
  receivedArrivalMessage: Boolean!
  receivedDepartureMessage: Boolean!
}

Steps To Reproduce
I have the following function in xcode to implement this:

class FlightWatcher: ObservableObject {
    @Published var flights: [Flight] = []
    @Published var flightsSynced: Bool = false
    var flightSubscription: AnyCancellable?

    func subscribeFlights() {
        self.flightSubscription = Amplify.Publisher.create(
            Amplify.DataStore.observeQuery(for: Flight.self)
        )
        .receive(on: DispatchQueue.main)
        .sink {
            if case .failure(let error) = $0 {

                print("Error \(error)")
            }
        } receiveValue: { [self]
            querySnapshot in
            print(querySnapshot)

            print("[Snapshot] item count: \(querySnapshot.items.count), isSynced: \(querySnapshot.isSynced)")
            if querySnapshot.isSynced {
                flightsSynced = true
                print("[Snapshot] item count: \(querySnapshot.items.count), isSynced: \(querySnapshot.isSynced)")
            }
            flights = querySnapshot.items
            print("Flights Synced: \(flightsSynced)")

        }

    }

    func unsubscribeFromTodos() {
        flightSubscription?.cancel()
    }
}

Expected behavior
I would expect that on a mutation event, or a create event, that the data would update in the app, in near realtime.
Amplify Framework Version
2.11.5
Amplify Categories
API, DataStore
Dependency manager
Swift PM
Swift version
5.8.1
CLI version
12.0.3
Xcode version
14.3.1.  14E300c
Relevant log output
<details>
<summary>Log Messages</summary>

DataStoreQuerySnapshot<Flight>(items: [], isSynced: false)
[Snapshot] item count: 0, isSynced: false
Flights Synced: false
DataStoreQuerySnapshot<Flight>(items: [Airboss.Flight(fltId: "644809278df1c179f926e07d", customerId: "pvl", origin: "CYYT", destination: "CVB2", flightNumber: "418", scheduledOut: "1686229200000", scheduledOff: "1686229500000", scheduledOn: "1686241500000", scheduledIn: "1686241800000", actualOut: "1686229200000", actualOff: "1686229500000", actualOn: "1686241500000", actualIn: "1686241800000", receivedArrivalMessage: false, receivedDepartureMessage: false, createdAt: Optional(Amplify.Temporal.DateTime(foundationDate: 2023-06-09 03:04:31 +0000)), updatedAt: Optional(Amplify.Temporal.DateTime(foundationDate: 2023-06-09 03:04:31 +0000)))], isSynced: true)
[Snapshot] item count: 1, isSynced: true
[Snapshot] item count: 1, isSynced: true
Flights Synced: true
2023-06-08 22:05:19.673116-0500 Airboss[13135:4618954] [IncomingAsyncSubscriptionEventToAnyModelMapper] The operation couldn’t be completed. (Amplify.GraphQLResponseError<AWSPluginsCore.MutationSync<AWSPluginsCore.AnyModel>> error 0.)

```
Is this a regression?
No
Regression additional context
No response
Device
iPhone 14 Pro Simulator
iOS Version
16.4
Specific to simulators
No response
Additional context
No response

I have further information to share.  In an effort to better understand the error, I enabled 'verbose' logging in the iOS app for Amplify.  Here are my further troubleshooting steps:

I Removed ALL entries from the DynamoDB manually.
I removed the "app" from the iOS Simulator.
Re-Built/Re-launched app on iOS Simulator, 0 items retrieved, as expected.
Using Postman, I simulated a "create" event to the graphql endpoint.  Item was successfully created, and when checking the DynamoDB Table, it does exist as expected.
The iOS Simulator logged multiple errors when this "createItem" event happened, and are detailed below.
Closing and reloading the simulator, now results in 1 item being pulled via appsync/graphql subscription as expected.

Here is the new, verbose logging from xcode, for subscription during a graphql "create" item event:
2023-06-09 10:50:16.201237-0500 Airboss[78139:6469504] [StarscreamAdapter] websocketDidReceiveMessage: - {"id":"2E39FE8E-DEEE-454F-B982-01FAAB5EE25F","type":"data","payload":{"data":{"onCreateFlight":null},"errors":[{"message":"Cannot return null for non-nullable type: 'AWSDateTime' within parent 'Flight' (/onCreateFlight/createdAt)","path":["onCreateFlight","createdAt"]},{"message":"Cannot return null for non-nullable type: 'AWSDateTime' within parent 'Flight' (/onCreateFlight/updatedAt)","path":["onCreateFlight","updatedAt"]}]}}
2023-06-09 10:50:16.202659-0500 Airboss[78139:6469504] [RealtimeConnectionProvider] Resetting stale connection timer
2023-06-09 10:50:16.210806-0500 Airboss[78139:6469509] [IncomingAsyncSubscriptionEventPublisher] onCreateValueListener: data(Swift.Result<AWSPluginsCore.MutationSync<AWSPluginsCore.AnyModel>, Amplify.GraphQLResponseError<AWSPluginsCore.MutationSync<AWSPluginsCore.AnyModel>>>.failure(GraphQLResponseError<MutationSync<AnyModel>>: GraphQL service returned a successful response containing errors: [Amplify.GraphQLError(message: "Cannot return null for non-nullable type: \'AWSDateTime\' within parent \'Flight\' (/onCreateFlight/createdAt)", locations: nil, path: Optional([Amplify.JSONValue.string("onCreateFlight"), Amplify.JSONValue.string("createdAt")]), extensions: nil), Amplify.GraphQLError(message: "Cannot return null for non-nullable type: \'AWSDateTime\' within parent \'Flight\' (/onCreateFlight/updatedAt)", locations: nil, path: Optional([Amplify.JSONValue.string("onCreateFlight"), Amplify.JSONValue.string("updatedAt")]), extensions: nil)]
Recovery suggestion: The list of `GraphQLError` contains service-specific messages))
2023-06-09 10:50:16.210981-0500 Airboss[78139:6469509] [IncomingAsyncSubscriptionEventToAnyModelMapper] receive(_:): data(Swift.Result<AWSPluginsCore.MutationSync<AWSPluginsCore.AnyModel>, Amplify.GraphQLResponseError<AWSPluginsCore.MutationSync<AWSPluginsCore.AnyModel>>>.failure(GraphQLResponseError<MutationSync<AnyModel>>: GraphQL service returned a successful response containing errors: [Amplify.GraphQLError(message: "Cannot return null for non-nullable type: \'AWSDateTime\' within parent \'Flight\' (/onCreateFlight/createdAt)", locations: nil, path: Optional([Amplify.JSONValue.string("onCreateFlight"), Amplify.JSONValue.string("createdAt")]), extensions: nil), Amplify.GraphQLError(message: "Cannot return null for non-nullable type: \'AWSDateTime\' within parent \'Flight\' (/onCreateFlight/updatedAt)", locations: nil, path: Optional([Amplify.JSONValue.string("onCreateFlight"), Amplify.JSONValue.string("updatedAt")]), extensions: nil)]
Recovery suggestion: The list of `GraphQLError` contains service-specific messages))
2023-06-09 10:50:16.211195-0500 Airboss[78139:6469509] [IncomingAsyncSubscriptionEventToAnyModelMapper] dispose(of subscriptionEvent): data(Swift.Result<AWSPluginsCore.MutationSync<AWSPluginsCore.AnyModel>, Amplify.GraphQLResponseError<AWSPluginsCore.MutationSync<AWSPluginsCore.AnyModel>>>.failure(GraphQLResponseError<MutationSync<AnyModel>>: GraphQL service returned a successful response containing errors: [Amplify.GraphQLError(message: "Cannot return null for non-nullable type: \'AWSDateTime\' within parent \'Flight\' (/onCreateFlight/createdAt)", locations: nil, path: Optional([Amplify.JSONValue.string("onCreateFlight"), Amplify.JSONValue.string("createdAt")]), extensions: nil), Amplify.GraphQLError(message: "Cannot return null for non-nullable type: \'AWSDateTime\' within parent \'Flight\' (/onCreateFlight/updatedAt)", locations: nil, path: Optional([Amplify.JSONValue.string("onCreateFlight"), Amplify.JSONValue.string("updatedAt")]), extensions: nil)]
Recovery suggestion: The list of `GraphQLError` contains service-specific messages))
2023-06-09 10:50:16.211306-0500 Airboss[78139:6469509] [IncomingAsyncSubscriptionEventToAnyModelMapper] dispose(of graphQLResponse): failure(GraphQLResponseError<MutationSync<AnyModel>>: GraphQL service returned a successful response containing errors: [Amplify.GraphQLError(message: "Cannot return null for non-nullable type: \'AWSDateTime\' within parent \'Flight\' (/onCreateFlight/createdAt)", locations: nil, path: Optional([Amplify.JSONValue.string("onCreateFlight"), Amplify.JSONValue.string("createdAt")]), extensions: nil), Amplify.GraphQLError(message: "Cannot return null for non-nullable type: \'AWSDateTime\' within parent \'Flight\' (/onCreateFlight/updatedAt)", locations: nil, path: Optional([Amplify.JSONValue.string("onCreateFlight"), Amplify.JSONValue.string("updatedAt")]), extensions: nil)]
Recovery suggestion: The list of `GraphQLError` contains service-specific messages)
2023-06-09 10:50:16.213770-0500 Airboss[78139:6469509] [IncomingAsyncSubscriptionEventToAnyModelMapper] The operation couldn’t be completed. (Amplify.GraphQLResponseError<AWSPluginsCore.MutationSync<AWSPluginsCore.AnyModel>> error 0.)

Apparently I needed to include "createdAt" and "modifiedAt" fields in the return fields of the mutation, and suddenly it's all working.  Curious I couldn't find that documented anywhere...I probably just missed it.    Oh well, moving on.


'search' state not getting params from url (createHashHistory)

I'm building a form/wizard which uses hashes to make it possible to go to different steps of the form (using url example.com/form#step-2 navigates to step 2 of the form for example). I can navigate between the steps via url and links without any issues.
However, in some cases we also want to to send params with the url to preload information in the form. When using createHashHistory the 'search' part of my state stays empty. I can get the params in 'search' when I use createBrowserHistory, but I can't use createBrowserHistory because I need the hashes for the navigation between steps.
For example when I use example.com/form?user=123456#step-2 I would expect 'search' to be '?user=123456'
const reducer = history => combineReducers({
    router: connectRouter(history),
    ...
})

export const connectHistory = createHashHistory({hashType: 'noslash'})

const store = createStore(
    reducer(connectHistory),
    {},
    composeEnhancers(
        applyMiddleware(
            routerMiddleware(connectHistory),
            thunk
        )
    )
)

export default store

ReactDOM.render(
    <Provider store={store}>
        <ConnectedRouter basename='/form' history={connectHistory}>
            <Application />
        </ConnectedRouter>
    </Provider>,
    document.getElementById("App")
);

I believe the docs explicitly state not to reference hash history, you should have no issue using hashes with browser history also. I think you just need a bit of help making it work, are you still stuck?


CMS Issues

[ ]  Pages created appear as 404s hours after they were published. E.g. https://fossilfuelregistry-v2-kb3l6.ondigitalocean.app/api https://fossilfuelregistry-v2-kb3l6.ondigitalocean.app/analysis https://fossilfuelregistry-v2-kb3l6.ondigitalocean.app/data-documentation - possible permissioning issue? I can edit pages which have already been published. NB no problem creating new articles.
[x]  Adapt design to allow Power Bi embeds without the white space, and preferably without exposing the horizontal slider at the bottom. E.g. https://fossilfuelregistry-v2-kb3l6.ondigitalocean.app/article/urgewald.
[ ] Is there a way to standardise the colour of headers as the same colour in the PDFs of the screens (see attached doc)?
[ ] Add following items to Category list: Oil, Coal, Gas, Methane, Company Report, Methodology
[ ] Expose the category list so that Editors can mantain it.

@OpenOilUG regarding second point:

Whitespace you can control by giving the iframe less height in the CMS.
We have no power over what PowerBI renders inside iframe. Bottom slide can possibly be removed from PowerBI side if they have options for it?

Regarding "color of headers", if you are referring to color of PowerBI headers the answer is no.

@JesperWe

the height function only controls the white space _to the sides' of the graphic by making it either bigger or smaller - as it gets bigger naturally it stretches proportionately across more of the page. BUT... there is also a vertical white space between the iframe and the last element... we need to get rid of this.
colour of headers does not refer to PowerBI headers but to the fonts in the flow of text in the PDF originally attached. if there is somewhere to set the default colour of h1 h2 h3 headlines system wide?

@Jesper

Add URL to feedback form from which a user came in to make a comment (as per convo)


Generate config.cmake files for generate user projects using CMake

Please generate config.cmake files for generate user projects using CMake.
Many modern libraries provide cmake/***Config.cmake file for users to find library and configure to project using CMakeLists.
Currently, CMake files of RealSense ID SDK does not generate it.

@gabime Do RealSense ID team have a plan to fix this issue yourself? (I'm concerned because related pull request #24 send from community has been closed by you.)

@UnaNancyOwen Yes, we plan to add this in one of the next releases.

@gabime It's good! Thanks, 👍

Released in v0.15.0. Use cmake -DRSID_INSTALL=ON to enable.


Feat/eature/add elasticdb tests

Fixes # .
Changes proposed in this PR:

tests

we can close this, fixed the tests in the PR itself


Add image thumbnail to the confirm bid screen

finishes https://artsyproduct.atlassian.net/browse/AUCT-473

Adds image { url(...) } to the GraphQL query to retrieve a URL to the small image of the artwork
Replaces the <Containers> with a <ScrollView> so smaller screens can host all these elements without getting them squashed
Updates tests, snapshots and storybooks to reflect the design change
The “A valid credit card is required for bidding.” message has already been added as part of https://github.com/artsy/emission/pull/1730 so we get that for free everywhere else

Screenshots
iPhone XS

iPhone SE

If so, what's the motivation for not implementing such a test.

Passing a prop to a component is very basic even though the source is GraphQL and there is no conditional logic here. Typos will be caught by yarn type-check which automatically gets triggered on every git push, so it should be good.

Merge on green


Linux: autocompletions should not be added in /usr/bin for deb and rpm

Is there an existing issue for this?

[X] I have searched the existing issues

Output of ddev debug test
Expand `ddev debug test` diagnostic information
[COPY-PASTE HERE THE OUTPUT OF `ddev debug test`]

Expected Behavior
The autocomplete scripts are already installed in /usr/share, where they belong.
/usr/bin for executables only.
Actual Behavior
ddev_bash_completion.sh, ddev_fish_completion.sh and ddev_zsh_completion.sh are added to /usr/bin folder for deb and rpm packages, and they are all executable, which adds to the confusion when trying to autocomplete ddev in your console.
See:

https://pkg.ddev.com/deb:ddev/-/content/usr/bin
https://pkg.ddev.com/rpm:ddev/-/content/usr/bin

Steps To Reproduce

Use Linux (or WSL2).
Type ddev<TAB><TAB> in console
See

$ ddev
ddev                     ddev_bash_completion.sh  ddev_zsh_completion.sh
ddev.link                ddev_fish_completion.sh

Anything else?
Noticed this in:

https://github.com/ddev/ddev/pull/5701#discussion_r1463720177

Is this a problem with goreleaser?

Yes, goreleaser builds all binaries for apt and rpm without filtering.
I believe #5759 might fix it, but I don't know how to test it.


Migrating from RNCryptor-objc to RNCryptor-swift

I am working on an application that is using the older objc version of RNCryptor.
The current objc release of RNCryptor is able to decrypt the data that has been encrypted.
When I attempt to switch to the SPM Swift and decrypt previously encrypted data using that, I get a UnknownHeader error.
Is there anything I need to do to ensure that the newer swift version can decrypt data that was encrypted using the older objc version?
It was originally encrypted like this:
NSData *data = [value dataUsingEncoding:NSUTF8StringEncoding];
NSData *encryptedData = [RNEncryptor encryptData:data
                                    withSettings:kRNCryptorAES256Settings
                                        password:encryptionKey
                                           error:&error];

return [encryptedData base64EncodedStringWithOptions:0];
and previously decrypted successfully like this:
NSData *encryptedData = [[NSData alloc] initWithBase64EncodedString:value options:0];
NSData *decryptedData = [RNDecryptor decryptData:encryptedData
                                    withPassword:encryptionKey
                                           error:&error];

return [[NSString alloc] initWithData:decryptedData encoding:NSUTF8StringEncoding];
Swift decryption which is failing with unknown header:
        let data: Data = Data(base64Encoded: encrypted)!
        let decryptedData = try RNCryptor.decrypt(data: data, withPassword: password)
        let decryptedString = String(data: decryptedData, encoding: .utf8)

What am I missing?  Thanks for any help!

What version of the ObjC library is being used here? What is first several bytes of ‘data’?I expect you’re either using a very very old version of the ObjC library (which I can help you with), or you’re incorrectly decoding your data in Swift earlier in the process. On Feb 6, 2024, at 8:04 PM, abush12345 @.***> wrote:
I am working on an application that is using the older objc version of RNCryptor.
The current objc release of RNCryptor is able to decrypt the data that has been encrypted.
When I attempt to switch to the SPM Swift and decrypt previously encrypted data using that, I get a UnknownHeader error.
Is there anything I need to do to ensure that the newer swift version can decrypt data that was encrypted using the older objc version?
It was originally encrypted like this:
NSData *data = [value dataUsingEncoding:NSUTF8StringEncoding];
NSData *encryptedData = [RNEncryptor encryptData:data
withSettings:kRNCryptorAES256Settings
password:encryptionKey
error:&error];
return [encryptedData base64EncodedStringWithOptions:0];
and previously decrypted like this:
NSData *encryptedData = [[NSData alloc] initWithBase64EncodedString:value options:0];
NSData *decryptedData = [RNDecryptor decryptData:encryptedData
withPassword:encryptionKey
error:&error];
return [[NSString alloc] initWithData:decryptedData encoding:NSUTF8StringEncoding];

Swift decryption which is failing with unknown header:
let data: Data = Data(base64Encoded: encrypted)!
let decryptedData = try RNCryptor.decrypt(data: data, withPassword: password)
let decryptedString = String(data: decryptedData, encoding: .utf8)
What am I missing?  Thanks for any help!
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

Thanks for your reply Rob.
"What version of the ObjC library is being used here? What is first several bytes of ‘data’?I expect you’re either using a very very old version of the ObjC library (which I can help you with),"
It is very, very old.   The original developers took the approach of copying the base file set into the codebase and its been ignored ever since.
It works with your current version of the objc version, and Ive confirmed that the Swift decoder is being passed exactly the same data as we are sending to the objc version.
We have a lot of existing users, so If we have to do a slow migration, that is ok, but if there is something you could suggest to speed that up and allow an immediate update, I would love to hear it.
Thanks again for your help.
Trying to find something that will help you guess which version its from, there is nothing explicit about version that Ive found.
In RNCryptor.m:
NSString *const kRNCryptorErrorDomain = @"net.robnapier.RNCryptManager";
const uint8_t kRNCryptorFileVersion = 2;
In RNDecryptor.m:
static const NSUInteger kPreambleSize = 2;

(BOOL)updateOptionsForPreamble:(NSData *)preamble
{
const uint8_t *bytes = [preamble bytes];
// See http://robnapier.net/blog/rncryptor-hmac-vulnerability-827 for information on the v1 bad HMAC
#ifdef RNCRYPTOR_ALLOW_V1_BAD_HMAC
if (bytes[0] == 1) {
self.options = bytes[1];
self.hasV1HMAC = YES;
return YES;
}
#endif
if (bytes[0] == kRNCryptorFileVersion) {
self.options = bytes[1];
return YES;
}
return NO;
}

Yeah, that's what I expected. This is v2 of the format. It was replaced in 2013 and was never ported to Swift.
The only difference between v2 and v3 is how multi-byte passwords are handled. v2 had a significant bug that truncated them. If you know that your passwords are always in ASCII, then you can just adjust the code to accept 2 as well as 3. (An easier solution may be to just fix the leading byte from 0x02 to 0x03 before passing it to the decryptor.)
If you need to deal with arbitrary passwords, then the fix will be similar to https://github.com/RNCryptor/RNCryptor/commit/2c3cae0e677c1aa4d841b655c82bcb0d4086bd60.
Depending on your needs, I can discuss making the changes for you, but my open source time is a bit limited right now. I can definitely work in a small consulting job, but it may take a while before I made this change. I would accept a PR, however, to support v2 decryption in Swift.

I will double check, but Im pretty confident that in the context it is being used, the passwords are provably all ASCII, so I will experiment with changing the leading byte and see how that plays out.
Thanks for your support Rob, and for the great software, its worked very well for us for years (too long, arguably :)
I will poke around a little.


Docker is not built

Hi,
ERROR: Service 'php' failed to build: The command '/bin/sh -c apt-get update     && apt-get install -y zlib1g-dev wget git-core python-pip     && docker-php-ext-install zip     && pip install Sphinx' returned a non-zero code: 100
I am constantly receiving this error.
What do you think it could be?

Hi @Daisuke-sama, could you please post the full output of docker-compose up --build?

Ok, found it myself. Please try again with current master.


Roman Numerals Encoder

Create a function taking a positive integer between 1 and 3999 (both included) as its parameter and returning a string containing the Roman Numeral representation of that integer.
Modern Roman numerals are written by expressing each digit separately starting with the left most digit and skipping any digit with a value of zero. In Roman numerals 1990 is rendered: 1000=M, 900=CM, 90=XC; resulting in MCMXC. 2008 is written as 2000=MM, 8=VIII; or MMVIII. 1666 uses each Roman symbol in descending order: MDCLXVI.
Example:
solution(1000); // should return 'M'

Help:
Symbol    Value
I          1
V          5
X          10
L          50
C          100
D          500
M          1,000

Remember that there can't be more than 3 identical symbols in a row.
More about roman numerals - http://en.wikipedia.org/wiki/Roman_numerals

Add in #180eaa6


Old Version of RDKit Installed when omnia above conda-forge

When installing the openforcefield package from omnia in an environment which prioritises the omnia channel over conda-forge (whether this was done manually using conda config or within an env.yml file) using
conda install openforcefield
RDKit is by default is installed  from omnia at version 2015. This leads to a
Fatal Python error: PyThreadState_Get: no current thread

Abort trap: 6

error when importing the openforcefield package, such as by:
from openforcefield.typing.engines import smirnoff
This issue can be fixed by either swapping conda-forge to be the top channel, or upgrading the RDKit package manually to 2018 or later.
Maybe this should be resolved by pinning RDKit to be greater than or equal to V2018 or later in the released version?

I tried to find why Travis wasn't having trouble with this setup, but was unsuccessful. I couldn't find in the travis build logs where it listed which rdkit version was being pulled in. Note also that this issue might relate to #237, since installation of these bad versions might be masked by the redundancy between the RDKIT variable in our testing matrix, and the fact that rdkit is now included in our testing environment YAML.
Regardless, a fix for this behavior would be to pin rdkit > 2016, since the newest version on omnia is 2015.

We can always delete the omnia rdkit if it's causing problems. I don't think anything used it except for an ancient version of deepchem.

If it won't have any bad side effects, removing rdkit from omnia would be really nice. As long as it's there, I think folks with custom environments will keep running into this issue.

I've moved the omnia builds of rdkit to the legacy label:
https://anaconda.org/omnia/rdkit/files

I've moved the omnia builds of rdkit to the legacy label:
https://anaconda.org/omnia/rdkit/files

This seems to have fixed things. Unless any one wants to do any further testing, I think this issue can be closed?

Thanks for checking, @SimonBoothroyd !


Question about the frequency of updating momentum network

Thanks for sharing the code.
Have you investigate the influence of T (frequency of updating momentum network)? There's not corresponding ablation study in paper.
Counld you please report the result of setting with different  T ?

Hi @super233,
we did not run an exhausitive sensitivity analysis of our model, but only a selected few which were of concern due to computational constraints. With regard to T, we only looked into the extreme setting without the momentum net (T = 1, \gamma_\psi = 0).
However, it is straightforward to modify T in the code (relevant settings are MODEL.NET_MOMENTUM_ITER and MODEL.NET_MOMENTUM), if your use case requires a more detailed study.
Best,
Nikita

Thanks, I got it.


GenesisConfig in a native runtime free world

The GenesisConfig is currently one of the only things for that we still require the native runtime. It isn't compiled for wasm right now. We also require the native runtime to load a GenesisConfig from a JSON file and as the GenesisConfig can change with newer native runtimes we got the raw chain spec that only contains the key/value storage pairs of the genesis block. When we remove the native runtime we need a new way to handle the GenesisConfig.
I propose that we put the GenesisConfig into the genesis wasm. This means that we start compiling the GenesisConfig for wasm, but I would put this behind some rust feature. So, the normal production builds should not include the GenesisConfig anymore. This is done to reduce the size of the wasm binary. I would also keep supporting the serialization of the GenesisConfig into JSON. If the GenesisConfig is part of the genesis wasm, we have some nice advantages. The first one being that we could keep the JSON serialized GenesisConfig in the chain spec and even distribute this chain spec as we would call into the genesis wasm to parse and generate the genesis block. This would require that we move out the genesis wasm from the actual genesis field in the chain spec to some dedicated field from where we can load the wasm binary. Another advantage would be that we could finally make the chain-spec-builder work properly. Currently this binary isn't really working for chain specs that aren't build from the kitchensink-runtime. The chain-spec-builder could be passed some crate name that belongs to some runtime and then it would generate the chain spec for this runtime. We could also enable certain features in the runtime etc.
To build what I outlined above, we would need the following:

Make GensisConfig build in wasm. This will require supporting serde in the wasm build to serialize/deserialize the GenesisConfig to/from JSON.
Add a new runtime api GenesisBuilder (better name should be found) that can be used to build the genesis config, load it etc.
Change the chain spec to have a dedicated field for the code.
Conditionally compile GenesisConfig into the wasm binary. By default the GenesisConfig should probably be compiled into the wasm binary. When doing a on-chain build there should exist a feature to disable the GenesisConfig.

Part of: https://github.com/paritytech/substrate/issues/10579

Required support for serde addressed in: https://github.dev/paritytech/substrate/pull/13027

Is anyone really using runtime-formatted JSON specs? All we really need is raw spec which does not require serde.

I think zombienet relies on meddling with the JSON chain spec and other testing tools probably as well.
Also it is very convenient for debugging…

Yeah and this will open a lot of possibilities. We will be able to ship json chain specs without them breaking, as the wasm file that loads them stays the same. We will be able to write tools that can modify chain specs very easily etc. It improves the user experience quite a lot.

I see it is a nice to have not essential. We can always load metadata from wasm and use it to encode/decode the raw values.
If this feature doesn't need to be part of Substrate, maybe it doesn't need to be part of Substrate?

I see it is a nice to have but not essential. We can always load metadata from wasm and use it to convert between the raw values and JSON. polkadot.js makes this relatively easy.

Good point, if that works good enough we don't need serde/json support.

Chopsticks already implemented this feature and it works well.
Custom storage override:
https://github.com/AcalaNetwork/chopsticks/blob/master/configs/kusama.yml
Code that convert yaml/json to hex:
https://github.com/AcalaNetwork/chopsticks/blob/master/packages/chopsticks/src/utils/set-storage.ts

Yeah to hex is easy, the other way around is more the problem. GenesisConfig fields are also not directly the storage items and can be from a different layout and then use custom logic to fill the storage. All in all still something that sounds useful to me. Especially as it is really just much easier for people to interact with it.
We would also then "disable" the GenesisConfig for builds that are going on chain as runtime upgrade to remove serde etc. So, there isn't really any downside.

@bkchr ,
maybe dumb question, but I am not sure what is good decision here:
When drafting GenesisBuilder Runtime API I've put there some functions which work on json.
IMO it would be good to return serde_json::error (Result here).
But in order to use this type while crossing runtime border, it must implement TypeInfo.
I see following options:

do some wrapper for serde_json::error, that copies the fields (and assert sizes) and implements the TypeInfo + From(serde_json::error).
the other approach could be String, but from what I see format is rather not used in runtime.

What do you think?

Conditionally compile GenesisConfig into the wasm binary. By default the GenesisConfig should probably be compiled into the wasm binary. When doing a on-chain build there should exist a feature to disable the GenesisConfig

Without this, I don't really want to enable Serialize/Deserialize.
This is a big regression. We only enable Serialize/Deserialize for good reasons and now we are forced to have then included wasm.

But the linker will remove it when not used, so serialize/deserialize will not be part of the wasm. But yes, you need it at compile time.

But the linker will remove it when not used

It is used if the wasm is able to build genesis? So without a feature flag to disable the ability to build genesis, it will be included on the onchain wasm.

Our parachain v2 is built upon the previous solo chain and parachain. Its genesis state includes all the previous states. Will this make the genesis WASM hundreds of megabytes?

@AurevoirXavier the code to generate genesis is in wasm but the genesis state is not in wasm.

But the linker will remove it when not used

It is used if the wasm is able to build genesis? So without a feature flag to disable the ability to build genesis, it will be included on the onchain wasm.

Yes, but this already explained in the issue that the GenesisConfig needs to be build conditionally. There needs to be a guide, but this is something that downstream needs to do on its own as we can not control the feature in your runtime. I think up to this day no one besides us uses a on-chain-build feature to disable logging for example.

Support for genesis building in runtime can be implemented as follows (see this PR https://github.com/paritytech/substrate/pull/14310 for full code):
use frame_support::genesis_builder_helper::{build_config, create_default_config};
sp_api::impl_runtime_apis! {
        ...
        #[cfg(feature = "genesis-builder")]
        impl sp_genesis_builder::GenesisBuilder<Block> for Runtime {
                fn create_default_config() -> Vec<u8> {
                        create_default_config::<RuntimeGenesisConfig>()
                }

                fn build_config(config: Vec<u8>) -> sp_genesis_builder::Result {
                        build_config::<RuntimeGenesisConfig>(config)
                }
        }

The implementation that actually uses all serde staff can be hidden behind dedicated feature.
When the feature is enabled depends on use-case. As Basti said, for on-chain builds the feature can be disabled and the resulting wasm blob will not contain any serialization code.
Having this approach implemented for polkadot runtime (PR is under construction), the numbers are as follows:

release profile:

type
size
num of funcs

with genesis-builder
7868841
906

no genesis-builder
8530347
0

release profile:

type
size
num of funcs

with genesis-builder
6436520
519

no genesis-builder
5896270
0

where num of funcs is the number of *serde* symbols in wasm binary:
wasm-objdump -x polkadot_runtime.wasm | grep -e "- func.*serde" | wc -l
Just a way to check if serde was actually optimized by LTO.
As you can see all the serde-related symbols were stripped away.

Once this whole thing is done, please consider reflecting your changes to https://docs.substrate.io/build/genesis-configuration/ cc @wentelteefje @Rashmirreddy

Yes I already talked with @michalkucharczyk to help updating the guides after these changes are merged!


[PINOT-6845] Provide a controller override for storage quota checks

Changes include:

add a controller level config to disable storage quota checks. Disabled by default. Will flip this around when we are ready to reenable the storage checks.
Use the conf to determine if quota needs to be checked during upload.

Testing done:
Pushed changes to integration test controller and confirmed that quota tests are not done.
Currently there is no test coverage for the APIs. Will look into adding them in subsequent PRs.

Codecov Report

Merging #3060 into master will increase coverage by 23.97%.
The diff coverage is 70%.

@@              Coverage Diff              @@
##             master    #3060       +/-   ##
=============================================
+ Coverage     46.31%   70.28%   +23.97%
- Complexity        0        4        +4
=============================================
  Files           934      934
  Lines         43496    43501        +5
  Branches       5907     5907
=============================================
+ Hits          20145    30575    +10430
+ Misses        21663    10869    -10794
- Partials       1688     2057      +369

Impacted Files
Coverage Δ
Complexity Δ

.../com/linkedin/pinot/controller/ControllerConf.java
40.9% <100%> (+5.03%)
0 <0> (ø)
:arrow_down:

...i/resources/PinotSegmentUploadRestletResource.java
56.09% <50%> (-1.45%)
0 <0> (ø)

...not/controller/validation/StorageQuotaChecker.java
78.26% <71.42%> (+46.91%)
0 <0> (ø)
:arrow_down:

...in/pinot/core/query/pruner/ValidSegmentPruner.java
57.14% <0%> (-28.58%)
0% <0%> (ø)

...e/impl/dictionary/LongOnHeapMutableDictionary.java
82.22% <0%> (-11.12%)
0% <0%> (ø)

...impl/dictionary/DoubleOnHeapMutableDictionary.java
68.88% <0%> (-6.67%)
0% <0%> (ø)

...roller/helix/core/PinotTableIdealStateBuilder.java
72.56% <0%> (-4.43%)
0% <0%> (ø)

...mpl/dictionary/DoubleOffHeapMutableDictionary.java
74.54% <0%> (-3.64%)
0% <0%> (ø)

...impl/dictionary/FloatOffHeapMutableDictionary.java
81.81% <0%> (-3.64%)
0% <0%> (ø)

...manager/realtime/HLRealtimeSegmentDataManager.java
74.28% <0%> (-2.39%)
0% <0%> (ø)

... and 492 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8e2ca1b...c732172. Read the comment docs.


feat: option to use c# 11 required keyword for required properties

This PR adds a UseRequiredKeyword option to the C# code generator. This will use the required modifier, added in C#11, for any properties that are required in the json schema.

@RicoSuter ping!

@amis92 Isn't this already the case, given that a property is only treated as nullable if GenerateOptionalPropertiesAsNullable is enabled and the property is not required?
https://github.com/RicoSuter/NJsonSchema/blob/9bf8f695b373410e8b51e1363270c08dda7b8127/src/NJsonSchema.CodeGeneration.CSharp/Models/PropertyModel.cs#L54-L55

That, or the null is allowed by type JSON Schema keyword.


Change GitHub action from pointing to master, to pointing to main

This fixes an issue from the template-repository to point the code-formatter action from master to main.

happy with the PR


Revise consolidated Readme document

@edrob999 - please add a detailed description here...

I discovered that I do not understand what the TITLE description is trying to define
What REPO is this referring to?
Which README is thig referring to?

:thumbsup:
Signed-off-by: Ed Robinson<EMAIL_ADDRESS>


Enable docker image for RISC-V on 21, 22, 23

We verified it works on jdk21u with https://ci.adoptium.net/job/build-scripts/job/jobs/job/evaluation/job/jobs/job/jdk21u/job/jdk21u-evaluation-linux-riscv64-temurin/92
Relates to https://github.com/adoptium/temurin-build/issues/3591

Using this configuration works on jdk21u as verified with https://ci.adoptium.net/job/build-scripts/job/jobs/job/evaluation/job/jobs/job/jdk21u/job/jdk21u-evaluation-linux-riscv64-temurin_SmokeTests/10

@smlambert @sxa @andrew-m-leonard could I please get a review? This is to match what's done for jdk17u, and jdk19u. I've verified it works for jdk22 at https://ci.adoptium.net/job/build-scripts/job/jobs/job/evaluation/job/jobs/job/jdk22/job/jdk22-evaluation-linux-riscv64-temurin/11/

Note sure if related but groovy tests fail:
Execution failed for task ':test'.

Could not resolve all files for configuration ':testRuntimeClasspath'.
Could not find org.testng:testng:7.2.0.
Searched in the following locations:
- https://repo.maven.apache.org/maven2/org/testng/testng/7.2.0/testng-7.2.0.pom
- https://repo.jenkins-ci.org/releases/org/testng/testng/7.2.0/testng-7.2.0.pom
- https://repo.jenkins-ci.org/public/org/testng/testng/7.2.0/testng-7.2.0.pom
Required by:
project : > org.codehaus.groovy:groovy-all:3.0.4 > org.codehaus.groovy:groovy-testng:3.0.4

re: https://github.com/adoptium/ci-jenkins-pipelines/pull/869#issuecomment-1868128612 - seen prior to this PR, reported in https://github.com/adoptium/ci-jenkins-pipelines/issues/866

@sxa @smlambert does that look good to you?

I've triggered a regen of the pipeline jobs so they should pick up this new parameter for the next runs


Invalid IBAN number from Ivory Coast

We have a customer with an IBAN number from Ivory Coast, this number is a valid number and renderd as valid by www.iban.com
However this libary (which we use in our software) returns that the specific IBAN number is not valid.
Note that for privacy reasons we cannot post the actual bank account number which has the issue.
On https://www.iban.com/structure.html there is a test number with the same behaviour: CI93CI0080111301134291200589

Hi @MartijnJong thanks for the report.
Côté d'Ivoire is not in the IBAN registry so I have trouble finding an official specification for their number. Can you provide any?
The contributor who originally added this one probably based himself on https://bank.codes/iban/structure/cote-divoire/ or some similar source.
But indeed, from the real-life examples I can gather online, it seems to be wrong.
Will fix and release a patch soon.
Laurent

Hey @MartijnJong , this is now fixed and published to npm as version 0.0.11
Thanks for your help 🙏


New End User Interface

This patch provides a new end user interface for creating extensions
that is more "C" like in nature. The big change that was needed to
support this patch was the removal of the gsl::not_null from delegate
types which is not needed since the APIs can ensure that the vcpus are
always valid making the APIs more accessible to C programmers.
We also removed dead code in the entry.cpp file that was added when the
RSDP code was added but never removed, and we added two additional
init functions for the new C interface

OBE


OpenVX on arm

Hello all,
Is there any OpenVX implementation for ARM? I can find this https://gpuopen.com/compute-product/amd-openvx/ but not for ARM devices.
Thanks,
Peter

Hi,
No there isn't but you can use the Khronos Openvx 1.0.1 Sample Implementation


fast-check in typescript

fast-check

Currently constructing sample sets via new Distribution(...), but otherwise I want to focus on interpreting squiggle strings for these tests (there are many examples of this)
Learning typescript and how to unpack our interface's notion of result (without marking stuff any) took up a lot of time, and I'm not necessarily happy with the results so far.
You'll notice that I gave up on not having any type, happy to be shown a good way of gaining back the property of any-lessness.
I'm discovering interesting things that should be rigorously documented-- what sorts of beta distribution parameters lead us to expecting it to NaN out (rendering a blank chart), questions like this fast-check has given me insight into. I guess boundaries on Arbitrary generators are a form of documentation, right now.
Threw in material for getting typescript coverage onto codecov, we'll see how that works.

Todo

[x] Modestly factor out some repeated code into a helpers file.
[x] Clean up commented out tests: be very clear if we think that the math is right but the code is wrong, or the other way around.

Codecov Report

Merging #316 (3052c22) into develop (4418f5b) will increase coverage by 0.53%.
The diff coverage is n/a.

:exclamation: Current head 3052c22 differs from pull request most recent head 1395a36. Consider uploading reports for the commit 1395a36 to get more accurate results

@@             Coverage Diff             @@
##           develop     #316      +/-   ##
===========================================
+ Coverage    61.14%   61.68%   +0.53%
===========================================
  Files           33       33
  Lines         2041     2041
===========================================
+ Hits          1248     1259      +11
+ Misses         793      782      -11

Impacted Files
Coverage Δ

...script/Distributions/SymbolicDist/SymbolicDist.res
75.90% <0.00%> (+1.20%)
:arrow_up:

...Interface/ReducerInterface_GenericDistribution.res
67.91% <0.00%> (+5.97%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4418f5b...1395a36. Read the comment docs.

Pinging @NunoSempere re the files Jstat_test.ts and SampleSet_test.ts

@OAGr wrote:

I don't understand your last point on error paths/objects.

A squiggle error is a javascript object. As we put more effort into the UX around error messages, we may want to validate or enforce in tests that certain computations are following a path to certain errors. @Hazelfire mentioned this as a positive aspect of Symbolic_test.ts in a comment above.

Pinging @NunoSempere re the files Jstat_test.ts and SampleSet_test.ts

Hey, Sam mentioned that it wasn't particularly urgent me to look into this, but feel free to nudge me in the other direction


Deprecate github.com/pkg/errors in favor of stdlib

The https://github.com/pkg/errors project has been archived for some time in favor of the proposals that landed in the stdlib. Just taking a quick look, seems like errors and fmt should be able to handle most of the use cases that Helm uses. I'm actively doing this for a tool that depends on the helm libraries which drew my attention to its use here.
Happy to help contribute swapping those out, but I want to make sure the project would be interested.

@komish PR is welcome.

@komish @yxxhero Do you mind if I create a PR for this?

Hm, now that I've looked into this a bit more, it seems like the main benefit of using github.com/pkg/errors are the stack traces which are returned by Errorf, Wrap, and Wrapf. For instance, it's possible to replace errors.Errorf with fmt.Errorf without trouble from the type system, but you lose the stack trace. I'm guessing these stack traces are pretty important @yxxhero? If so, then the PR I drafted might not be of much use, but if it's fine to lose the stack traces from some of these calls, then we could move forward.

@jgsheppa apologies, I totally missed notification for this issue. Thanks for taking care of it!

Anyone working on this issue?

@jgsheppa Are you working on this issue?

@jgsheppa Are you working on this issue?

As far as I know, it is still being discussed by the team.

Shameless plug: you can also switch it to drop-in replacement gitlab.com/tozd/go/errors. It fixes many issues, is maintained, and supports modern Go's error patterns (sentinel errors, %w formatting, joined errors, etc.). It also provides some nice utility functions and structured details so that it is easy to extract dynamic data out of errors (instead of trying to get them out of formatted strings). Has improved error formatting and JSON marshaling of errors. It is interoperable with other errors packages and does not require a dependency on itself to extract data (e.g., stack trace) from errors.


"No results" message should have additional explanation why that might be

:+1: Just to add to this, if a site is offline when it's crawled, the results page should probably show that visually at a glance.  E.g.  http://lmgtdfy.usopendata.org/ : Cityofchicago.gov is not a real site afaik.


implement s2grid

Spherical harmonics evaluated on spherical grid, including transformations to/from the grid.
quadrature weights are still in terms of beta

Side note: install black and format the code with it.


The blank option should not be present there in Calendar dropdown.

CRM version : V1.1.3
Steps To reproduce--
1. log in as Admin/User.
2. Click on Google icon on side bar .
3.  Reach on  Select the calendar you want to sync section.
Actual result: When we are trying to select the calendar for sync. then a blank option is showing there. Please fine the below screenshot.
Expected result: The blank option should not be present there in Calendar dropdown.

@prateek-webkul
Now, this issue is fixed. so I am closing this issue.
And I have attached a screenshot for future reference.


Update README.md

This is to improve the fun/work ratio. I'm open to other suggesstions but it will be a tough discussion, @jakoschiko.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>7
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 83.757%

Totals

Change from base Build<PHONE_NUMBER>5:
0.0%

Covered Lines:
1021

Relevant Lines:
1219

💛  - Coveralls


fix errors in the test;

skip test_get_prob_and_cov_cond1 as it does not seem to work.

Yes, it is my next to do list.


Create Engineer

Thanks

If you're interested in contributing an Earth Engine tutorial, please see these instructions.
If you're interested in reporting an issue, please use this template.


Transparency of docker images and scoring programs

It is shown in #723 that the starting kit and public data buttons do not exist anymore in Codabench.
More generally, we would like to add a "transparency" feature. Organizers could have option to decide what they want to make public: the competition docker image, the scoring program, ingestion program, input data, etc.
Each piece made public would have a download button from the competition web pages.
The docker image could simply be shown using its DockerHub tag.
The sample submission (solution) should also be subject to this transparency, not only by being downloadable, but by being automatically run and evaluated.
The fact that CodaLab Competitions and Codabench are open-source is a crucial point in making these platforms great tools for the scientific community. This way, everybody can review the code and the benchmarks and competitions are more valuable.
However, the code and data used by the organizers is not necessarily open. Even if some organizers provide the code via Github with a link in the documentation, one can not ensure that it is really the code used to evaluate the submissions, making the final results less transparent. The goal of this feature is to have a real transparency, when enabled, between the organizers and the participants.

Similar ideas:

Organizers contact email

It would be good to have a place, typically a field in the editor, for the organizer's email address. This is important because it is very common that participants contact the administrators of the platform instead of the organizers for competition specific questions because they can't find their contact.
Inside parenthesis next to "organized by":

A sentence in the forum:

Link to the report

It would be interesting to have a way to show the link to a competition report. The link could be accessible from an icon benchmark listing, and also well visible in the competition home page.

Reward (prize) of the competition

As stated in #954, we should be able to set a reward (e.g. 4000$) from the editor. I don't think we want it to be doable from the competition.yaml file because that should be done only one the "production" version of the competition, after the tests. It could then be shown in the interface, in the home page of the challenge and in the public challenges list.
In the list:

Below "participants" and "submissions" using the same styling:

TODOs:
Editor / YAML fields

[x] Display docker image name in competition
Solved by: #944
[x] Add a YAML and editor field Organizers contact where the organizers put an email address that will be shown in the interface of the competition
Solved by: #1063
[x] Editor field Reward (free-form string?)
Solved by: #1063
[x] Editor field Challenge report
Solved by: #1063

Files tab
First PR #970:

[x] For non-approved participants: Not accessible
[x] For approved participants: Showing objects made available by organizers
[x] For organizers: Showing all object and a column showing what is available or not to participants

Next TODOs:

[x] In editor / YAML, two options: make programs available, make input data available (#1136)
[x] Add documentation about the fields to add starting kit and public data in the file competition.yaml
[x] Solutions appear even when not registered to the competition. Should be treated in the same way as public data and starting kit (show only to registered participants and organizers, and always in available state)
[x] The No Files Available Yet text is always showing (#1041)
[x] Fix the column Available that always displays "Yes" (see #970) (#1136)
[x] Green checks instead of yes (see screenshot below, like the resources interface). These ticks are just markers and should not be clickable. (#1136)

See the discussion below for better understanding

Similar ideas:

Email organizers

It would be good to have a place (typically a field in the editor) for the organizer's email address.

Link to the report

It would be interesting to have a way to show the link to a competition report. The link could be accessible from the benchmark listing.

@Didayolo we should discuss this to make a plan for these features

@ihsaan-ullah Yes, I think some details should be discussed to agree on what we want to do. Also, it is important to discuss with Benjamin who is working on adding the "Files" tab to download data and starting kit.

@Didayolo Benjamin is working on download download buttons but the decision is not finalized yet about where to inject starting kit and sample submission

A good solution to how organizers can chose what is available for participants or not is to use the "files" tab for this:

This tab should be:

Not accessible by non-approved participants
Showing available objects to approved participants
Showing all objects to organizers and having a column showing what is available or not to participants (and an option to chose to make available/unavailable each object).

For non-approved, we could have the same thing as the submissions tab:

(check the behavior when revoked, etc.)

@Didayolo

Add a yaml / editor field Organizers contact where the organizers put an email address that will be shown in the interface of the competition

Is this required? Normally we put contact info in the html/markdown pages

Think about the possibility of having a Challenge report field or something

What is challenge report?

Is this required? Normally we put contact info in the html/markdown pages

We need to think about it. The problem is that many organizers forget to put their contact in the documentation. Also, as it is free form, the contact info can be hard to find when it is somewhere.

What is challenge report?

I was thinking about after the competition end: an analysis paper, or a blog post sharing the results, etc.

We need to think about it. The problem is that many organizers forget to put their contact in the documentation. Also, as it is free form, the contact info can be hard to find when it is somewhere.

In this case this is useful. Some questions:

where will this be shown in the UI? Maybe a new tab about contact info?
do we just need email address or something else too?
In the editor: if we just need an email, this can be added in the main page where can add/edit Queue and docker. If we need more than just an email, then a separate tab

I was thinking about after the competition end: an analysis paper, or a blog post sharing the results, etc.

should this be an html page or just one field where organizer can put a link. I think we should give multiple options if we are not providing html/markdown page. For example: link to

dataset
paper
blog

Contact email

where will this be shown in the UI? Maybe a new tab about contact info?

A new tab seems to be too much just for an email address, but it is a possibility.

where will this be shown in the UI? Maybe a new tab about contact info?

I think a single email address is sufficient. If organizers want to provide more, they can still use the documentation.

In the editor

Yes, it should be a simple field in the main page of the editor. Maybe there should be a warning or something showing that it is important to add a contact email.
Report

should this be an html page or just one field where organizer can put a link

I think a link is sufficient and gives enough freedom, it is good to keep it simple. Once again, organizers can still edit the documentation pages if they want to give more information. Here, the goal is to have an interface to redirect to the analysis. Typically, if the link to the report could be displayed in the competition list interface.

@Didayolo  What should be the name fo the report?
External link, Related paper, Related report or something else

Very good question.
Analysis paper, Challenge report ?

Challenge report seems reasonable. Lets use this for now and we can change it later if this is not what we want

@Didayolo

Field Reward
Just one field for reward? What will happen when there are prizes for 1st 2nd 3rd positions?

@ihsaan-ullah We can discuss about it. In CodaLab we have just one field for the reward, that is usually used to show the total reward, so the addition of all prizes.
Note that this interface is just used to inform participants about the total prize. No money transfer is made from the platform.

🎉


naive_stack_allocator double delete

Describe the bug
during use of naive_string or std::basic_string<char, std::char_traits, naive_stack_allocator<char, 64>> a double delete can occur when string s("") style construction is used because the ALTERNATE_TYPE constructor is called after 1 byte is allocated
This causes naive_stack_allocator to think the item was allocated by the chained allocated which in turn makes it call deallocator there which fails because it wasn't allocated by new
To Reproduce
Steps to reproduce the behavior:

declare string type using naive_stack_allocator
instantiate a string using constructor taking a string such as char const*
let the object go out of scope

Expected behavior
string should be deallocated and only memory allocated by the chained allocated should be freed there
Desktop (please complete the following information):

OS: Windows 10 2004

closing as native stack allocator has been removed


Go to definition unnecessary shows export along true definition

VSCode Version: 1.33.1 and 1.34.0-insiders
OS Version: Arch Linux

Steps to Reproduce:

create foo.js

function foo() {}

module.exports = { foo };

create bar.js
const foo = require('./foo');

foo.foo();

In bar.js Ctrl+click on foo() to go to definition.
It shows both function definition and export from modules.exports

Does this issue occur when all extensions are disabled?: Yes

Confirmed on<EMAIL_ADDRESS>


New reptile sounds

About the PR

Reptilians and Kobolds can now thump their tail, huff, and hiss
Why / Balance

Additional sounds for roleplay purposes. Tail thump also included in SS13 Shiptest
Technical details

Added new type ReptilianBodyEmotes to speech_emote_sounds.yml
Added audio paths in speech_emote_sounds.yml
Added relevant emotes to emotes.ftl
Gave allowedEmotes component to Reptilians and Kobolds
Added new Emote icons
Media

https://github.com/user-attachments/assets/83c9993f-ca48-4005-aeee-9a19d64c1255
Requirements

[X] I have read and am following the Pull Request and Changelog Guidelines.
[X] I have added media to this PR or it does not require an ingame showcase.

Breaking changes

Changelog

:cl: Minemoder, ArtisticRoomba, Sarahon

add: Reptiles and Kobolds can now thump their tail, huff, and hiss

Here's what the sounds sound like.
https://github.com/user-attachments/assets/ec14a248-0016-43fb-95c4-d68b5e032ae7

undraft when you are ready!

Sure, I'll see if I can edit the current huff or find a different one that fits better

I suggest you read through https://github.com/space-wizards/space-station-14/pull/29448 since there was a lot of feedback on lizard noises there. I'd also like to add my own comments here but I already discussed it in the PR I linked

Sure, I'll see if I can edit the current huff or find a different one that fits better

Any progress on that?

No, not really. I've been struggling to find some sounds that work and fit with current reptilian sounds. What parts of the hiss and huff were too guttural? I think I have a good idea of why the huff won't fit but I just want a bit more info that I can work with to find or make some good sounds.

No, not really. I've been struggling to find some sounds that work and fit with current reptilian sounds. What parts of the hiss and huff were too guttural? I think I have a good idea of why the huff won't fit but I just want a bit more info that I can work with to find or make some good sounds.

It's like, too low pitched with deep vibrations if you sorta get what i mean.

Does that also apply to the hiss? Because the only low pitched rumbly part at the start of the sound isn't really that low pitched. I could trim that part of the hiss if it'd make the sound work.

Alright so instead of letting this sit here forever, I've decided to just trim this PR down to the tail thump and not add the huff or hiss. How should I go about doing that? Should I remove the yaml changes, or comment them out so someone in the future can take it over? Should I remove or keep the huff and hiss emote icons for future use?

It'd be accepted then yeah i think

Random, unrelated test fail.
Code looks good to me. Thank you for your contribution!


getLatestStratumJobEx fail

Hello!
First of all thanks for creating such mining pool =)
But during a testing we cannot connect to stratum server

Serverside logs have many of this:
E0524 19:03:46.582401 22602 StratumSession.cc:328] decode line fail, not a json string
I0524 19:03:46.769662 22602 StratumServer.cc:1139] got an error on the socket: Connection reset by peer
I0524 19:03:46.948123 22602 StratumSession.cc:246] client connect, ip: <IP_ADDRESS>
W0524 19:03:47.953395 22602 StratumServer.cc:128] getLatestStratumJobEx fail
I0524 19:03:49.422773 22602 StratumServer.cc:1139] got an error on the socket: Connection reset by peer
I0524 19:06:18.259013 22602 StratumSession.cc:246] client connect, ip: <IP_ADDRESS>
E0524 19:06:18.262748 22602 StratumSession.cc:328] decode line fail, not a json string
I0524 19:06:18.440995 22602 StratumServer.cc:1139] got an error on the socket: Connection reset by peer
I0524 19:06:18.647907 22602 StratumSession.cc:246] client connect, ip: <IP_ADDRESS>
I0524 19:06:20.646862 22602 StratumServer.cc:1136] socket closed
I0524 19:06:34.655304 22602 StratumSession.cc:246] client connect, ip: <IP_ADDRESS>
E0524 19:06:34.659281 22602 StratumSession.cc:328] decode line fail, not a json string
I0524 19:06:34.840478 22602 StratumServer.cc:1139] got an error on the socket: Connection reset by peer
I0524 19:06:35.034485 22602 StratumSession.cc:246] client connect, ip: <IP_ADDRESS>
I0524 19:06:37.049279 22602 StratumServer.cc:1136] socket closed

May be its a known issue and there is an easy answer?

Please show me the log of jobmaker.
It seems no job sending to kafka so getLatestStratumJobEx fail.

cat ./run_jobmaker/log_jobmaker/jobmaker.INFO
Log file created at: 2018/05/23 06:36:11
Running on machine: ip-172-31-2-78
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0523 06:36:11.976052 11081 Zookeeper.cc:63] Zookeeper: connecting to zookeeper brokers: 0s
I0523 06:36:16.986544 11081 Zookeeper.cc:133] Zookeeper: fight for lock with 1 clients
I0523 06:36:16.986585 11081 Zookeeper.cc:139]   * 0. node0000000001
I0523 06:36:16.986591 11081 Zookeeper.cc:147] Zookeeper: got the lock!
W0523 06:36:16.986665 11081 JobMakerMain.cc:142] using bitcoin testnet3
I0523 06:36:16.986871 11081 Kafka.cc:347] producer librdkafka version: 0.9.1
I0523 06:36:16.986886 11081 Kafka.cc:82] consumer librdkafka version: 0.9.1
I0523 06:36:16.986896 11081 Kafka.cc:82] consumer librdkafka version: 0.9.1
I0523 06:36:16.986905 11081 Kafka.cc:82] consumer librdkafka version: 0.9.1
I0523 06:36:16.986912 11081 JobMaker.cc:68] Block Version: 20000000
I0523 06:36:16.986919 11081 JobMaker.cc:69] Coinbase Info: region1/Project BTCPool/
I0523 06:36:16.986923 11081 JobMaker.cc:70] Payout Address: my2dxGb5jz43ktwGxg2doUaEb9WhZ9PQ7K
I0523 06:36:16.987118 11081 Kafka.cc:417] add brokers: ec2-34-219-18-149.us-west-2.compute.amazonaws.com:9092
I0523 06:36:16.987582 11081 Kafka.cc:424] create topic handle: StratumJob
I0523 06:36:17.018752 11081 Kafka.cc:160] add brokers: ec2-34-219-18-149.us-west-2.compute.amazonaws.com:9092
I0523 06:36:17.018831 11081 Kafka.cc:167] create topic handle: RawGbt
I0523 06:36:20.050447 11081 Kafka.cc:160] add brokers: ec2-34-219-18-149.us-west-2.compute.amazonaws.com:9092
I0523 06:36:20.050526 11081 Kafka.cc:167] create topic handle: NMCAuxBlock
I0523 06:36:22.081872 11081 Kafka.cc:160] add brokers: ec2-34-219-18-149.us-west-2.compute.amazonaws.com:9092
I0523 06:36:22.081943 11081 Kafka.cc:167] create topic handle: RawGw
I0523 06:36:24.113158 11081 JobMaker.cc:194] consume latest rawgbt message from kafka...
I0523 06:36:29.113328 11081 JobMaker.cc:204] consume latest rawgbt messages done
W0523 06:36:29.113373 11081 JobMaker.cc:610] RawGbt Map is empty

It seems bad is gbtmaker
cat ./run_gbtmaker/log_gbtmaker/gbtmaker.ERROR
Log file created at: 2018/05/25 08:21:38
Running on machine: ip-172-31-2-78
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0525 08:21:38.315428 11077 GbtMaker.cc:105] bitcoind zmq recv exception: Interrupted system call
F0525 08:21:38.315748 11077 GbtMakerMain.cc:127] gbtmaker init failure

cat ./run_gbtmaker/log_gbtmaker/gbtmaker.INFO
Log file created at: 2018/05/25 08:21:38
Running on machine: ip-172-31-2-78
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0525 08:21:38.539988 32160 Kafka.cc:347] producer librdkafka version: 0.9.1
I0525 08:21:38.540321 32160 Kafka.cc:417] add brokers: ec2-34-219-18-149.us-west-2.compute.amazonaws.com:9092
I0525 08:21:38.540370 32160 Kafka.cc:424] create topic handle: RawGbt
I0525 08:21:38.582880 32160 Utils.cc:362] getnetworkinfo: {"result":{"version":150100,"subversion":"/Satoshi:0.15.1/","protocolversion":70015,"localservices":"000000000000000d","localrelay":true,"timeoffset":0,"networkactive":true,"connections":10,"networks":[{"name":"ipv4","limited":false,"reachable":true,"proxy":"","proxy_randomize_credentials":false},{"name":"ipv6","limited":false,"reachable":true,"proxy":"","proxy_randomize_credentials":false},{"name":"onion","limited":true,"reachable":false,"proxy":"","proxy_randomize_credentials":false}],"relayfee":0.00001000,"incrementalfee":0.00001000,"localaddresses":[],"warnings":"Warning: unknown new rules activated (versionbit 28)"},"error":null,"id":"1"}
I0525 08:21:38.583097 32160 GbtMaker.cc:100] check bitcoind zmq, waiting for zmq message 'hashtx'...


Stuck with the inputs

Request: Help with Obtaining STAR Alleles from VCF Files (Illumina Short Read Sequencing, Dragen Variant Caller)
Hi,
I’m looking to get STAR alleles for a set of VCF files generated from Illumina short-read sequencing using the Dragen platform (variant caller). The VCF files have been obtained from sequencing, and I need to identify the corresponding STAR alleles, typically used in pharmacogenomics or for specific gene variants like CYP2D6.
Could you please help me with the following:
Verifying VCF files:
Are there any specific steps or checks I should perform to ensure my VCF files are correctly formatted for STAR allele identification? The files are from Dragen, so they should already have the standard variant calling format.

Hi Dr. Lee,
UPDATE: I ran the following command using the pypgx tool to analyze CYP2D6 variants for a single sample:
pypgx run-ngs-pipeline CYP2D6 CYP2D6-pipeline --assembly GRCh38 --variants sample1.vcf.gz --platform Targeted
This successfully produced results for the sample. However, I now have a cohort of 2000 WES samples and need to run the same analysis across all of them.
Merging these 2000 VCF files will likely take days, and I’m looking for a more efficient way to carry out this analysis at scale. Can you suggest a strategy or workflow to process these files more quickly?

Hi @rajdbs32,
Thanks for your interest in using PyPGx!

Merging these 2000 VCF files will likely take days, and I’m looking for a more efficient way to carry out this analysis at scale. Can you suggest a strategy or workflow to process these files more quickly?

Please take a look at #108. Honestly, 2,000 samples is not that many ;) If you really want to speed things up, I would suggest you look into parallel computing. Basically, you could divide your samples into 10 batches, each with 100 samples, and then fire up 10 separate instances in AWS or Google Cloud. Also, please note that you do NOT need to merge 2,000 VCF files into one.

Thank you, Dr. Lee. I have tried what you suggested and the tool works great.


goss_file vs tests is confusing

so, as far as I understand, tests only copies the given files to the machine, but the tests that are actually execute is whatever is defined on goss_file, right?
so, even if tests is an array, I can only run 1 suite.
maybe we can remove goss_file and run all the suites given on tests?
if you agree, I can work on a patch for it...
thanks for the project BTW :)

I actually have a version of this working on my fork...

@fishnix This is actually quite useful to have this. Would you accept a PR for this?

closing old issues, reopen if still interested 👍


return score with named queries

Description
Opensearch already support labelling the queries, that returns as a list in the returned results, of which query it matched. However one of the use case while doing hybrid search with query text and dense vector is to determine individual scores for each query type. This is very useful in further analysis and building offline model to generate better weights for ranking score. Hence adding this feature that sends the client to add the score for each matched query.
Related Issues
Resolves #[Issue number to be closed when this PR is merged]

Check List

[x] New functionality includes testing.

[x] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
[x] Commits are signed per the DCO using --signoff
[ ] Commit changes are listed out in CHANGELOG.md file (See: Changelog)
[ ] Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

❕ Gradle check result for e323df4: UNSTABLE

TEST FAILURES:

      1 org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

This seems to be a flaky test, not related to my changes

https://github.com/opensearch-project/OpenSearch/issues/9191
Seems to be an open issue for the flaky test @dblock

https://github.com/opensearch-project/OpenSearch/pull/11626
Replacing by above PR and closing this one due to multiple unsigned commits and rebase issues ^


Toolchain: AdoptOpenJDK vendor rebranded to "Eclipse Foundation"

Expected Behavior
When using toolchain with vendor spec JvmVendorSpec.ADOPTOPENJDK, gradle should use local JDK or download and install it.
Current Behavior
Gradle does not use my local JDK and untar downloaded JDK for each module
Context
It looks like AdoptOpenJDK vendor is now "Eclipse Foundation", gradle compares vendor for my local JDK and for previously downloaded one and unpack again downloaded version.
Steps to Reproduce
I just added the toolchain declaration to an project generated with gradle init
I pushed the project there https://github.com/MOOM-Planet/gradle-toolchain-vendor
On second build, we can see that downloaded JDK is unpacked again
2021-08-13T14:13:34.215+0200 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Toolchain detection' started
2021-08-13T14:13:34.216+0200 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Completing Build operation 'Toolchain detection'
2021-08-13T14:13:34.216+0200 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Toolchain detection' completed
2021-08-13T14:13:34.217+0200 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire exclusive lock on adoptopenjdk-16-x64-linux.tar.gz.
2021-08-13T14:13:34.217+0200 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on adoptopenjdk-16-x64-linux.tar.gz.
2021-08-13T14:13:34.217+0200 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Provisioning toolchain adoptopenjdk-16-x64-linux.tar.gz' started
2021-08-13T14:13:34.217+0200 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Unpacking toolchain archive' started
2021-08-13T14:13:38.701+0200 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Completing Build operation 'Unpacking toolchain archive'
2021-08-13T14:13:38.701+0200 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Unpacking toolchain archive' completed
2021-08-13T14:13:38.701+0200 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Completing Build operation 'Provisioning toolchain adoptopenjdk-16-x64-linux.tar.gz'
2021-08-13T14:13:38.701+0200 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Provisioning toolchain adoptopenjdk-16-x64-linux.tar.gz' completed
2021-08-13T14:13:38.701+0200 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on adoptopenjdk-16-x64-linux.tar.gz.```

### Your Environment
Build scan URL: https://scans.gradle.com/s/32j43spna2sj4

There's a workaround that might work for you: https://github.com/gradle/gradle/issues/18004#issuecomment-898569966

JFYI, constant re-downloading of the JDK was fixed in 7.2 (see #17509).

I was wrong, the patch I mentioned will not help with this particular case because we only check that the requested spec has AdoptOpenJdk as a vendor; we don't check if the downloaded toolchain actually is from AdoptOpenJdk.


fix: refined error message

Signed-off-by: zeppp<EMAIL_ADDRESS>
1.Describe what this PR did
refined error message
2.Does this pull request fix one issue?
fix #310
3.Describe how you did it
4.Describe how to verify it
5.Special notes for reviews

Thanks a lot.
Could you add more details in 4.Describe how to verify it?
How about adding a test command output to show us you are right.
Otherwise, it is not so direct, reviewing code directly is not a easy way without any evidence that this bugfix is reasonable.
@zeppp

I found that this is a duplicate of #324
If we are determined to fix some issue, we can leave a comment of #dibs to assign yourself to this issue. Then issue is on you.

Sorry about that,i will #dibs next time.


[Feature Request] Supoort PopupRoute routes

1⃣
Is your feature request related to a problem? Please describe.
The problem is that BeamPage.pageRouteBuilder returns PageRoute where it could be Route.

2⃣
Describe the solution you'd like (This is an actual solution)
The solution is to make it returns Route so the new signature would be:
class BeamPage extends Page {
    // ...
    final Route Function(RouteSettings settings, Widget child)? pageRouteBuilder;
    // ...
}

3⃣
Additional context
1- This should not require any changes because the PageRoute class is a child class of the Route class.
2- This will allow other route types to be used such as the PopupRoute which is another child class of the ModalRoute class.

Hi @ihadabs
Thanks for creating an issue!
This is a good idea and is already implemented on master:
https://github.com/slovnicki/beamer/blob/9e5bc1099e124fac0a1bf618ec0bfbe346fa4d6b/package/lib/src/beam_page.dart#L146
You can try it out by depending on beamer like this:
# in your pubspec.yaml
beamer:
  git:
    url: https://github.com/slovnicki/beamer.git
    path: package

There are a number of breaking changes between v0.14.1 and master (future v0.15.0), so take a look at CHANGELOG on how to migrate. Let me know if everything works as expected for you


DATABASE_URI is hardcoded somewhere

Link to reproduction
No response
Describe the Bug
Changing DATABASE_URI to PAYLOAD_DATABASE_URI
Then providing it to db: mongooseAdapter({ url: process.env.PAYLOAD_DATABASE_URI }),.
Results in the string not being used but instead DATABASE_URI MUST be provided under that name.
To Reproduce
Set PAYLOAD_DATABASE_URI  and pass to mongooseAdapter.
Payload Version
2
Adapters and Plugins
No response

I was able to swap these values in the config and .env without issue. Could you provide a repo that reproduces this issue?

Maybe it's when using the dev version of the admin UI. Seeing I was unable to run build:payload.
Perhaps in that situation  it uses a payload which has DATABASE_URI coded somewhere?


Enable Oracle multiple insert to handle CURRENT_TIMESTAMP correctly

This fixes the case on Oracle where if you perform a multiple insert query with an object that contains knex.fn.now(), the resulting sql query will have misaligned bindings in the sub-query's values. I didn't want to change parameterize, so I just added a regex switch in Oracle's positionBindings.
Before:
begin
  execute immediate 'insert into "users" ("deleted_at", "email") values (CURRENT_TIMESTAMP, :1)'
  using CURRENT_TIMESTAMP, ?;

  execute immediate 'insert into "users" ("deleted_at", "email") values (CURRENT_TIMESTAMP, :1)'
  using CURRENT_TIMESTAMP, ?;
end;

After:
begin
  execute immediate 'insert into "users" ("deleted_at", "email") values (:1, :2)'
  using CURRENT_TIMESTAMP, ?;

  execute immediate 'insert into "users" ("deleted_at", "email") values (:1, :2)'
  using CURRENT_TIMESTAMP, ?;
end;

@rhys-vdw Any chance this can go into 0.11.0? :wink:

@wubzz Could you add the Oracle SQL label to this?

Thanks @wubzz ! Would it also be possible to add the 0.11.0 label? This is pretty important for my bulkInserts in Oracle

@AiNoKame Hey sorry I just released but I'm happy to get this problem fixed and release a patch.

the resulting sql query will have misaligned bindings in the sub-query's values. I didn't want to change parameterize, so I just added a regex switch in Oracle's positionBindings

positionBindings seems like a strange place for this change, but I don't have a good idea of the bigger picture. What is causing the incorrect placement?


Implement Microservices prototype

[ X] Get flask project from Corey to work

[ ] Separate logic from route files

[ ] Get project to run on a docker container

[ ] Separate logic files into their own containers


1D FFT on multiple lines

Hello is there a way to get the 1D fft on multiple lines at the same time?
I have a bunch of signals, all having the same length, and I would like to have the 1D fft for each of them.
I was going to loop over all the signals and run the 1D clfft for each of them. Do you think that running them all together would gain better performance? Is there a way to do that with clfft?
In other words I would like to have an intermediate result of a 2D fft, before transposing and transforming in the other direction.
Thanks a lot!
Fabio

Yes, when you create an FFT plan, it has a default batch value of 1, but you can increase the batch value with clfftSetPlanBatchSize().

Thanks a lot for your answer!
Is there any example that you could point me to please?
Or could you tell me which are the parameters to change besides batchsize?
If concatenate 2 complex signals (each with real/imaginary interleaved) and I set the batchsize to 2, the 1D fft looks right for the first signal, but wrong for the second.
How do I use clfftSetPlanDistance? in the documentation it refers to clfftSetPlanDimension() but I can't find this method.
Thank you so much!

I figured it out myself: I was setting the wrong sizes!
The size of the fft should be set to the size of each single fft line; whereas I was setting it to the size of the whole buffer.

Correct, our kernels need the stride to jump between each individual FFT.  Glad you figured it out.

Hello again
I have a row-major array and I would like to run the 1D fft for each Column.
I just set the strideX equal to the batch size (i.e. the number of columns) using clfftSetPlanInStride. Is there anything else I have to set? The plane dimension is the number of rows right?
It looks like the first signal is correctly transformed, but the following ones seem truncated, like shifted... do I need to set some padding anywhere?
Thank you so much for the help!

This is my test code for 3 arrays columns of 64 samples each
   int NumSamples = 64;
   int NumBatches = 3;
   clfftPlanHandle m_planHandle;
size_t sizes[1] = { NumSamples };
size_t strides[1] = { NumBatches };

clfftCreateDefaultPlan(&m_planHandle, m_context, CLFFT_1D, sizes);
clfftSetResultLocation(m_planHandle, CLFFT_OUTOFPLACE);
clfftSetPlanBatchSize(m_planHandle, NumBatches);
clfftSetPlanInStride(m_planHandle, CLFFT_1D, strides);
clfftBakePlan(m_planHandle, 1, &m_queue, NULL, NULL);

clfftEnqueueTransform(m_planHandle, CLFFT_FORWARD, 1, &m_queue, 0, NULL, NULL, &clBufIN, &clBufOUT, NULL);

The first signal spectrum is fine, but the other two look like the input signal was cropped.
Do you have any suggestion?

To go from the first column to second, how much is the offset? I am assuming it is 1 from what you are saying. So you to need set the distances to 1. Look into clfftSetPlanDistance.

Yes that was it! :)
Thank you so much!


How to finetune and serve the finetuned model using VLLM like tools

Hi I want to finetune it on my custom OCR tasks, can you share finetuning script and also help in telling the best way to serve it very very fast.

同问

SGlang support would be excellent, as it doesn't seem to work out of the box.


Release tools wishlist....

A tool which will bump the version number in the module/target.json, commit as "vX.Y.Z", and generate the tag.
A tool which will generate the default and docker manifest files based on the latest tag available for each project
A tool which generates, tests and uploads the new container, from a central server

Replaced by https://kubostech.atlassian.net/browse/KUBOS-71, https://kubostech.atlassian.net/browse/KUBOS-72 and https://kubostech.atlassian.net/browse/KUBOS-70


Unexpected results from qiskit.aqua.algorithms.NumpyMinimumEigensolver

Information

Qiskit Aqua version: 0.7.3
Python version: 3.7.7
Operating system: MacOS Catalina, Version 10.15.5, Macbook Pro, 13 inch, 16GB

What is the current behavior?
The NumPyMinimumEigensolver in qiskit.aqua.algorithms is giving wrong results.
Steps to reproduce the problem
import numpy as np
import pylab
from qiskit.aqua import QuantumInstance
from qiskit.aqua.algorithms import NumPyMinimumEigensolver, VQE
from qiskit.aqua.components.optimizers import SLSQP
from qiskit.chemistry.drivers import PySCFDriver
from qiskit.chemistry.core import Hamiltonian, QubitMappingType

molecule = 'H .0 .0 -{0}; Na .0 .0 {0}'

pts  = [x * 0.1  for x in range(10, 25)]
pts += [x * 0.25 for x in range(10, 18)]
pts += [4.5]

energies = np.empty(len(pts))
distances = np.empty(len(pts))

for i, d in enumerate(pts):
    driver = PySCFDriver(atom=molecule.format(d/2), basis='sto3g')
    qmolecule = driver.run()
    operator =  Hamiltonian(qubit_mapping=QubitMappingType.JORDAN_WIGNER,
                                two_qubit_reduction=False,
                                freeze_core=True, orbital_reduction=[])
    qubit_op, aux_ops = operator.run(qmolecule)
    result = NumPyMinimumEigensolver(qubit_op, aux_operators=aux_ops).run()
    result = operator.process_algorithm_result(result)
    energies[i] = result.energy
    distances[i] = d

pylab.plot(distances, energies, label="NumpyMinimumEigensolver")
pylab.xlabel('Interatomic distance')
pylab.ylabel('Energy')
pylab.title('NaH Ground State Energy')
pylab.legend(loc='upper right');

Run this code and the results is show in the following figure. The results after distance > 2.3 are not correct

What is the expected behavior?

This picture shows the expected results (the green line).
Suggested solutions
Look into the implementation of NumpyMinimumEigensolver.
``

@ajavadia Hi Ali, I opened this issue for the bug I reported to you today.

I confirm that if I run your code from the command line I get a plot like you show. Yet if I modify the nah_uccsd notebook to just run the NumpyMinimumEigensolver then the plot is as expected. I will need to investigate what is different when running that it behaves like this.

Hi, thanks for your attention.

Hi, some more information for you. If you set two_qubit_reduction to False when using PARITY it shows the same plot as JW mapping. If with JW mapping we set z2symmetry_reduction to 'auto' in the Hamiltonian (new capability in the latest 0.7 Aqua release) then the plot is again as expected.
Two qubit reduction is a type of reduction done by exploiting known symmetry. Z2Symmetry reduction reduces by finding symmetries too. I think what is happening is that, as the molecule is stretched, states are crossing over one another. If you print the result at each step i.e. the one returned process_algorithm_result (has some nice readable format) you will see the number of particles (electrons) measured changes from 2 to 1. When the operators are created, by 2nd quantization, they will end up with states ranging from all orbitals occupied, to all unoccupied. Of course NaH, with the core frozen, and only two particles, we are really wanting the state with two particles (2 orbitals occupied of the right spin too), but the NumpyMinimumEigensolver has no constraints over it and just picks off the lowest eigenvalue. When the symmetry reduction is done, which logically splits the operator up in multiple smaller ones, then the one we use (we know which one to select) seems to no longer contain the other state(s) that result in this crossover (presumably these states are in one of the other smaller operators we discard).
What I also did was to take the first 4 states from the matrix diagonalization that NumpyMinimumEigensolver uses. Here you can see
the ground state in orange and another, that is labelled excited state 2 in red. At that distance of 2.3 you can see the states still exist but after the ordering has changed and what we consider the ground state is now what we started off with as Excited State 2.

When run with VQE+UCCSD the outcome of the above is correct since the wavefunction of UCCSD constrains the final state to have 2 particles in it. As NumpyMinimumEigensolver is a generic solver it is just picking the lowest eigenvalue no matter what and we see the behavior you observe.
I think this behavior may just be the 'expected' result with that molecule in this case. I will however try to check with one of our chemistry experts around this molecule's behavior to see if this can be explained too from a chemistry perspective.

Thank you for the explaination. So I guess that means we need to be careful when using the generic NumpyMinimumEigensolver.

Yes, it advisable to check the result. This is why we added the number of particles, spin etc as extra operators which we measure at  for the eigenstate of the algorithm result. Checking is advised for VQE too, especially where a heuristic ansatz is used for the wavefunction where it is unconstrained by aspects such as number of particles to keep it in the proper solution space.
Given the above explanation etc I will close this off now.