One-to-many relationship - Failed compiling GraphQL schema

Note: If your issue/bug is regarding the AWS Amplify Console service, please log it in the
Amplify Console GitHub Issue Tracker

Describe the bug
A clear and concise description of what the bug is.
Running amplify add api and selecting One-to-many relationship from What best describes your project throws an error:
Failed compiling GraphQL schema: id field is not of type String ? Correct the errors in schema.graphql and press Enter to re-compile.
Amplify CLI Version
You can use amplify -v to check the amplify cli version on your system
4.18.0
To Reproduce
Steps to reproduce the behavior or terminal output if applicable
? Please select from one of the below mentioned services: GraphQL
? Provide API name: myapi
? Choose the default authorization type for the API API key
? Enter a description for the API key: demo
? After how many days from now the API key should expire (1-365): 7
? Do you want to configure advanced settings for the GraphQL API No, I am done.
? Do you have an annotated GraphQL schema? No
? Do you want a guided schema creation? Yes
? What best describes your project: One-to-many relationship (e.g., “Blogs” with “Posts” and “Comments”)
? Do you want to edit the schema now? Yes
Expected behavior
A clear and concise description of what you expected to happen.
No error
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. Mac/Windows/Ubuntu]
Node Version. You can use node -v to check the node version on your system

Additional context
Add any other context about the problem here.

Hi @atgillette
We have already made the change for this issue in this PR:https://github.com/aws-amplify/amplify-cli/pull/4070.
For the workaround you can use the following code
type Post @model @key(name: "byBlog", fields: ["blogID"]) {
  id: ID!
  title: String!
  blogID: ID!
  blog: Blog @connection(fields: ["blogID"])
  comments: [Comment] @connection(keyName: "byPost", fields: ["id"])
}

You can the BlogID from String! to ID! as explained in above code.
Let me know if are still stuck on this issue.

Thx @akshbhu.


OpenDAL python binding is much slower than ossfs

Discussed in https://github.com/apache/opendal/discussions/4900

Originally posted by sysu-yunz July 15, 2024
I'm working on a data analysis project in Python that uses Alibaba Cloud OSS. I'm looking for a universal method to read/write parquet files and have been experimenting with OpenDAL and fsspec/ossfs.
During my tests, I found that OpenDAL is about 4-5 times slower than ossfs. While I really appreciate OpenDAL's vision, this performance gap is a significant obstacle to adoption for my use case.
Here's the code I used for testing:
import opendal
import polars as pl
import ossfs
import time
from io import BytesIO
import pyarrow.dataset as ds

oss_config = {
    "endpoint": "my-endpoint",
    "key": "my-key",
    "secret": "my-secret",
    "bucket": "my-bucket",
}

class myOperator:
    def __init__(self) -> None:
        self.op = opendal.Operator(
            "oss",
            bucket=oss_config["bucket"],
            endpoint=oss_config["endpoint"],
            access_key_id=oss_config["key"],
            access_key_secret=oss_config["secret"],
        )

    def write(
        self,
        df: pl.DataFrame,
        file_name: str,
    ) -> None:
        # Open and write the DataFrame to the file.
        with self.op.open(file_name, mode="wb") as file:
            df.write_parquet(file)  # raise warning: Polars found a filename. Ensure you pass a path to the file instead of a python file object when possible for best performance.
            # buffer = BytesIO()
            # df.write_parquet(buffer)
            # file.write(buffer.getvalue())

    def read(
        self,
        file_name: str,
    ) -> None:
        # Open and read the DataFrame from the file.
        with self.op.open(file_name, mode="rb") as file:
            read_df = pl.read_parquet(file) # raise error: TypeError: 'opendal.File' object is not iterable
            print(f"read_df: {read_df.head(5)}")

class myOSSFileSystem:
    def __init__(self) -> None:
        self.fs = ossfs.OSSFileSystem(
            bucket=oss_config["bucket"],
            endpoint=oss_config["endpoint"],
            key=oss_config["key"],
            secret=oss_config["secret"],
        )

    def write(
        self,
        df: pl.DataFrame,
        file_name: str,
    ) -> None:
        # Open and write the DataFrame to the file.
        with self.fs.open(f"{oss_config['bucket']}/{file_name}", mode="wb") as file:
            buffer = BytesIO()
            df.write_parquet(buffer)
            file.write(buffer.getvalue())

    def read(
        self,
        file_name: str,
    ) -> None:
        # Open and read the DataFrame from the file.
        with self.fs.open(f"{oss_config['bucket']}/{file_name}", mode="rb") as file:
            read_df = pl.read_parquet(file)
            print(f"read_df: {read_df.tail(5)}")

    def read_ds(
        self,
        file_name: str,
    ) -> None:
        dataset = ds.dataset(
            source=f"{oss_config['bucket']}/{file_name}", format="parquet", filesystem=self.fs
        )
        pl.scan_pyarrow_dataset(dataset).collect()

df = pl.DataFrame(
    {
        "a": [1, 2, 3, 4, 5],
        "b": [5, 4, 3, 2, 1],
    }
)

# compare time cost of writing with opendal and ossfs
start = time.time()
op = myOperator()
op.write(df, "file_op.parquet")
print(f"opendal write cost: {time.time() - start}")

start2 = time.time()
# 🏆
fs = myOSSFileSystem()
fs.write(df, "file_fs.parquet")
print(f"ossfs write cost: {time.time() - start2}")

# # compare time cost of reading with opendal and ossfs
# start = time.time()
# op = myOperator()
# op.read("file_op.parquet")
# print(f"opendal read cost: {time.time() - start}")

# start2 = time.time()
# fs = myOSSFileSystem()
# fs.read("file_fs.parquet")
# print(f"ossfs read cost: {time.time() - start2}")

# start3 = time.time()
# # 🏆
# fs2 = myOSSFileSystem()
# fs2.read_ds("file_fs.parquet")
# print(f"ossfs-ds read cost: {time.time() - start3}")

The output:
Small dataframe( df in code ):

opendal write cost: 0.4947168827056885
ossfs write cost: 0.21863603591918945

Large dataframe( from a parquet file ~7M) :

opendal write cost: 2.2169010639190674
ossfs write cost: 0.4991908073425293

Has anyone else encountered similar performance issues? Are there any optimizations or best practices I might be missing? I'd love to use OpenDAL, but the current speed difference is too significant for my project's needs.

I've investigated further based on @Xuanwo 's suggestions:

ossfs caching: I didn't enable any specific cache feature for ossfs, nor did I find evidence of it writing to the local filesystem first in my analysis of the ossfs and fsspec code.

BytesIO vs direct writing: I tested writing to BytesIO versus direct writing, and it didn't significantly change the performance results.

Local file operations: To check for any local file operations, I implemented an audit hook to monitor file access:

import sys

def audit_hook(event, args):
    if event.startswith(('open', 'os.mkdir', 'os.remove')):
        print(f"File operation: {event}, args: {args}")

# write parquet here

The only file operation detected was:
File operation: open, args: ('/etc/apache2/mime.types', 'r', 16777216)
This seems unrelated to the core writing process.

@sysu-yunz @Xuanwo I tested the above code on my computer(a m1 macbook) and got the following results for a 7.5M file.
opendal write cost: 5.381440877914429
opendal write cost: 4.242851734161377
opendal write cost: 5.645648241043091
opendal write cost: 5.148315191268921
opendal write cost: 6.76938796043396
opendal write cost: 6.574304103851318
opendal write cost: 4.361173152923584
opendal write cost: 4.172515153884888
opendal write cost: 4.181843996047974
opendal write cost: 4.608723878860474
opendal write average cost: 5.108707523345947
ossfs write cost: 3.7259011268615723
ossfs write cost: 3.256885051727295
ossfs write cost: 4.137396812438965
ossfs write cost: 3.9885058403015137
ossfs write cost: 3.753488063812256
ossfs write cost: 3.5728158950805664
ossfs write cost: 3.87477707862854
ossfs write cost: 3.67903208732605
ossfs write cost: 3.600888729095459
ossfs write cost: 3.447010040283203
ossfs write average cost: 3.7037297248840333

I didn't see a 4x difference, but opendal was definitely slower.
if i change opendal write method to this
    def write(
        self,
        df: pl.DataFrame,
        file_name: str,
    ) -> None:
        # Open and write the DataFrame to the file.
        with self.op.open(file_name, mode="wb") as file:
            # df.write_parquet(file)  # raise warning: Polars found a filename. Ensure you pass a path to the file instead of a python file object when possible for best performance.
            buffer = BytesIO()
            df.write_parquet(buffer)
            self.op.write(file_name, buffer.getvalue())
            # file.write(buffer.getvalue())

They are pretty much the same.
opendal write cost: 3.345073938369751
opendal write cost: 3.335000991821289
opendal write cost: 3.3251380920410156
opendal write cost: 3.4893510341644287
opendal write cost: 3.895937919616699
opendal write cost: 3.3452987670898438
opendal write cost: 3.181452989578247
opendal write cost: 3.3563389778137207
opendal write cost: 3.221903085708618
opendal write cost: 3.04779314994812
opendal write average cost: 3.3544297933578493
ossfs write cost: 3.1847658157348633
ossfs write cost: 4.221267223358154
ossfs write cost: 3.1877329349517822
ossfs write cost: 4.032901048660278
ossfs write cost: 5.51796293258667
ossfs write cost: 3.189411163330078
ossfs write cost: 3.8925600051879883
ossfs write cost: 3.137454032897949
ossfs write cost: 3.0315730571746826
ossfs write cost: 4.050448179244995
ossfs write average cost: 3.744664716720581

This is a bit strange, but here's a comparison of the results I wrote with BytesIO, and I ran it 10 times like you, and I calculated the average
opendal write cost: 2.4278807640075684
opendal write cost: 2.4647626876831055
opendal write cost: 2.138329029083252
opendal write cost: 2.2295989990234375
opendal write cost: 2.1915431022644043
opendal write cost: 2.1873180866241455
opendal write cost: 2.011554718017578
opendal write cost: 2.2732584476470947
opendal write cost: 2.0787739753723145
opendal write cost: 2.518566131591797
average opendal write cost: 2.252260422706604
ossfs write cost: 0.5420899391174316
ossfs write cost: 0.5580389499664307
ossfs write cost: 0.5613429546356201
ossfs write cost: 0.5493559837341309
ossfs write cost: 0.5359418392181396
ossfs write cost: 0.5070497989654541
ossfs write cost: 1.1258749961853027
ossfs write cost: 0.5559501647949219
ossfs write cost: 0.5436689853668213
ossfs write cost: 0.5498058795928955
average ossfs write cost: 0.6029793739318847

The network in my office is in the same area as OSS, I wonder if the same is true on your side?
Here's my library version and python version, I don't know if it's the same as yours?
opendal==0.45.6
ossfs==2023.12.0

Python 3.11.9

@sysu-yunz I am in the same region as OSS, this my machine info:
> uname -a
Darwin LGNB-2023010001.local 22.3.0 Darwin Kernel Version 22.3.0: Thu Jan  5 20:50:21 PST 2023; root:xnu-8792.81.2~2/RELEASE_ARM64_T8112 arm64

and library version:
opendal==0.45.6
ossfs==2023.12.0

Python 3.12.3

this is small dataframe result:
opendal write cost: 0.12370681762695312
opendal write cost: 0.131605863571167
opendal write cost: 0.08657598495483398
opendal write cost: 0.10484504699707031
opendal write cost: 0.06839585304260254
opendal write cost: 0.08764481544494629
opendal write cost: 0.0695180892944336
opendal write cost: 0.06910419464111328
opendal write cost: 0.07377099990844727
opendal write cost: 0.07526993751525879
opendal write average cost: 0.08909075260162354
ossfs write cost: 0.14001989364624023
ossfs write cost: 0.1259160041809082
ossfs write cost: 0.11330318450927734
ossfs write cost: 0.12091207504272461
ossfs write cost: 0.12328791618347168
ossfs write cost: 0.13866591453552246
ossfs write cost: 0.1255178451538086
ossfs write cost: 0.13857603073120117
ossfs write cost: 0.11663579940795898
ossfs write cost: 0.10755586624145508
ossfs write average cost: 0.12507236003875732

I noticed that the Python version is different. Will this be an issue?

I changed my python version to Python 3.12.4, the result stay same.

I noticed that the Python version is different. Will this be an issue?

I used 3.11.9 and got the same result, it could be other reason.
@sysu-yunz If you use this code
    def write(
        self,
        df: pl.DataFrame,
        file_name: str,
    ) -> None:
        # Open and write the DataFrame to the file.
        #with self.op.open(file_name, mode="wb") as file:
            # df.write_parquet(file)  # raise warning: Polars found a filename. Ensure you pass a path to the file instead of a python file object when possible for best performance.
          buffer = BytesIO()
          df.write_parquet(buffer)
          self.op.write(file_name, buffer.getvalue())
            # file.write(buffer.getvalue())

Are you getting the same?

Really thank you guys @Xuanwo @Zheaoli @hoslo , I should first run test on Aliyun ECS internal network, sorry to bother you. I think we can close this issue because my service run on ECS and the result shows I can use OpenDAL confidently.

Thanks a lot to @hoslo and @Zheaoli for debugging! Also, welcome @sysu-yunz; feel free to report any more issues with opendal. Hope you're enjoying using opendal!

Hi @Zheaoli, since you can reproduce the same result, could you open a new issue to debug it further? It seems there are some unusual cases causing opendal to be slower than ossfs. I'm not sure if that's related to public endpoint resolving or credentail loading logic.

Hi @Zheaoli, since you can reproduce the same result, could you open a new issue to debug it further? It seems there are some unusual cases causing opendal to be slower than ossfs. I'm not sure if that's related to public endpoint resolving or credentail loading logic.

Yes, in my plan. I will try to find some cues first

Thanks a lot in advance.

Hi @Zheaoli, since you can reproduce the same result, could you open a new issue to debug it further? It seems there are some unusual cases causing opendal to be slower than ossfs. I'm not sure if that's related to public endpoint resolving or credentail loading logic.
Also, @sysu-yunz, feel free to join in this debug journey if you're interested.

I want to give a try, but I haven't even started learning the Rust language yet, learning by doing I think🤣


Detectron2 support

I used label-studio a year ago and since then lots of features have been added. I am trying to integrate the detectron2 support for the active learning model but unable to do it. I have tried making some changes using mmdetection script but there is a compatibility issue for Flask, pillow, and some others.  I upgrade/downgrade any of them either detectron2 gives an error or label-studio.
Kindly add support for this detectron2 framework with updated dependencies.

Have you any logs with compatibility issues?


feat(update): add ng update as alias of ng init

Fix #4007

Having ng update probably fixes #2959 #2930

@beeman for now ng update does exactly the same as ng init, it's just an alias.


Better way to list kernels from jupyter and find a matching kernel

For #8612

Use json to list kernel specs from jupyter
New method to find a specific kernel spec from jupyter

Codecov Report

Merging #8785 into master will decrease coverage by 0.63%.
The diff coverage is 65.27%.

@@            Coverage Diff             @@
##           master    #8785      +/-   ##
==========================================
- Coverage   58.66%   58.03%   -0.64%
==========================================
  Files         526      526
  Lines       27180    26870     -310
  Branches     4044     4058      +14
==========================================
- Hits        15946    15594     -352
- Misses      10363    10398      +35
- Partials      871      878       +7

Impacted Files
Coverage Δ

src/client/datascience/jupyter/notebookStarter.ts
70.43% <ø> (ø)
:arrow_up:

src/client/common/installer/types.ts
100% <ø> (ø)
:arrow_up:

src/client/common/installer/productPath.ts
87.93% <0%> (-3.15%)
:arrow_down:

src/client/common/installer/condaInstaller.ts
90.24% <100%> (+0.24%)
:arrow_up:

...lient/datascience/jupyter/kernels/kernelService.ts
55.1% <100%> (ø)

src/client/common/types.ts
100% <100%> (ø)
:arrow_up:

src/client/common/installer/productService.ts
100% <100%> (ø)
:arrow_up:

src/client/common/installer/productNames.ts
100% <100%> (ø)
:arrow_up:

src/client/datascience/jupyter/jupyterExecution.ts
55.26% <100%> (ø)
:arrow_up:

src/client/common/installer/pipInstaller.ts
59.37% <16.66%> (-2.7%)
:arrow_down:

... and 50 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 17b7dbe...312587c. Read the comment docs.


SEGV during cargo or trunk compile

Problem

Steps To Reproduce
Steps to reproduce the behavior:

include yew-router 0.16.0 along with git latest version of yew

Expected behavior
It either works or tells me a feature that's missing
Screenshots
Instead I get
Compiling yew-router v0.16.0
/home/andy/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x4e54e3)[0x7f1aa5ce54e3]
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f1aa521a520]
/home/andy/projects/csci/target/debug/deps/libyew_router_macro-86287919bed15f71.so(+0x218000)[0x7f1a963aa000]
error: could not compile yew-router
Caused by:
process didn't exit successfully: rustc --crate-name yew_router --edition=2018 /home/andy/.cargo/registry/src/github.com-1ecc6299db9ec823/yew-router-0.16.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 -C metadata=a645282592cce5a9 -C extra-filename=-a645282592cce5a9 --out-dir /home/andy/projects/csci/target/wasm32-unknown-unknown/debug/deps --target wasm32-unknown-unknown -L dependency=/home/andy/projects/csci/target/wasm32-unknown-unknown/debug/deps -L dependency=/home/andy/projects/csci/target/debug/deps --extern gloo=/home/andy/projects/csci/target/wasm32-unknown-unknown/debug/deps/libgloo-08eda4883ee39409.rmeta --extern gloo_utils=/home/andy/projects/csci/target/wasm32-unknown-unknown/debug/deps/libgloo_utils-21e8612dfe536b06.rmeta --extern js_sys=/home/andy/projects/csci/target/wasm32-unknown-unknown/debug/deps/libjs_sys-b7f634e88a85fa9a.rmeta --extern route_recognizer=/home/andy/projects/csci/target/wasm32-unknown-unknown/debug/deps/libroute_recognizer-b7b8a87a51d4d334.rmeta --extern serde=/home/andy/projects/csci/target/wasm32-unknown-unknown/debug/deps/libserde-8eedd5536afcd6dd.rmeta --extern serde_wasm_bindgen=/home/andy/projects/csci/target/wasm32-unknown-unknown/debug/deps/libserde_wasm_bindgen-8c7d90f5d29ea460.rmeta --extern serde_urlencoded=/home/andy/projects/csci/target/wasm32-unknown-unknown/debug/deps/libserde_urlencoded-bd13462978b7960c.rmeta --extern thiserror=/home/andy/projects/csci/target/wasm32-unknown-unknown/debug/deps/libthiserror-04823a643ccfd1ce.rmeta --extern wasm_bindgen=/home/andy/projects/csci/target/wasm32-unknown-unknown/debug/deps/libwasm_bindgen-a5a2fb49d1378357.rmeta --extern web_sys=/home/andy/projects/csci/target/wasm32-unknown-unknown/debug/deps/libweb_sys-31c1e755250b2631.rmeta --extern yew=/home/andy/projects/csci/target/wasm32-unknown-unknown/debug/deps/libyew-847835cad7faeaf3.rmeta --extern yew_router_macro=/home/andy/projects/csci/target/debug/deps/libyew_router_macro-86287919bed15f71.so --cap-lints allow (signal: 11, SIGSEGV: invalid memory reference)
warning: build failed, waiting for other jobs to finish...
during the compile (i.e. trunk serve or cargo build)
Environment:

Yew version: 0.19.3 (git master)
Rust version: rustc 1.62.1 (e092d0b6b 2022-07-16)
Target, if relevant: wasm32-unknown-unknoen
Build tool, if relevant: trunk
OS, if relevant: Ubuntu 22.04.1 LTS
Browser and version, if relevant: Chrome but not relevant

Questionnaire

[x] I'm interested in fixing this myself but don't know where to start
[ ] I would like to fix and I have a solution
[ ] I don't have time to fix this right now, but maybe later

You can't have multiple versions of the same crate and pass an object from one crate version to another one. The error message states this as mismatched types.
If you want to use Yew from git, you need to make sure that all your dependencies also use Yew from git.
yew-router = { git = "https://github.com/yewstack/yew" }

Makes sense but a SEGV to show the user they have mismatching crates? I switched both to git and it works

Weird, If it happens during cargo build then I think this is worthy of a bug report on cargo itself. If only with trunk build then it most likely belongs there, there's not much we can do to fix this here.

Ok cool we can close this but the error happens on cargo or trunk builds


Updates links for most recent PCIe fw

Updates links and docs in the load_pcie_fw folder so that it loads most recent binaries.

Where are the changes to the firmware discussed/documented? I think maybe we should raise this to @dpdutcher to disseminate the updates to all the systems?


Why is there no ResultAsync.fromThrowable?

Result.fromThrowable takes a function as an argument, and returns a new function that returns a Result. There is no corollary for ResultAsync, which really diminishes the utility of the library. There have been a lot of issues in here mentioning how people want to have an async function that returns a ResultAsync. This is just fundamentally wrong. It defeats the entire purpose of using a library like this, because the whole point is that the errors are moved from thrown, untyped values into the return type of the function.
For example:
// doesn't compile because it's not returning a `Promise`, and thats a GOOD THING
const getOrders = async (username: string): ResultAsync<Order[], AxiosError> => {
  // This is bad, because it can throw
  const { data } = await axios.get(`/users/username/${username}`)
  const { id } = data
  return ResultAsync.fromPromise(axios.get(`orders/customer-id/${id}`).then(({ data }) => data), (cause) => cause as AxiosError)
}

To use such a function, you would have to handle the potential error in the result, but you would also still need to wrap it in a try/catch
try {
  await getOrders('someUser').match(
    (orders: Order[]) => console.log(orders),
    // typesafe error type
    (error: AxiosError) => res.send(500, { message: error.message }),
  )
} catch (error: unknown) {
  // no type safety
  res.send(500, { message: error instanceof Error ? error.message : 'unknown error' })
}

fromPromise does not provide sufficient type safety, because it's possible to have synchronous error thrown while creating said promise:
const updateUser = (user: Partial<User>): Promise<User> => {
  const { id, ...updates } = user
  if (!id || Object.keys(updates).length === 0) {
    // since this is not an `async` function, this throws in a synchronous context
    throw new TypeError(`invalid user value: ${user}`)
  }
  return axios.put(`/users/$id`, updates).then(({ data }) => data)
}
// `TypeError` will be thrown here, not captured in the result
const userResult = ResultAsync.fromPromise(updateUser({}), (err: unknown) => err as TypeError | AxiosError)

For ResultAsync to actually provide safety, it's rather critical IMO to be able to convert an arbitrary function that returns a Promise to a wrapper enclosed with ResultAsync that is able to handle both types of errors, not just the rejected Promise
const updateUser = ResultAsync.fromThrowable(
  (user: Partial<User>): Promise<User> => {
    const { id, ...updates } = user
    if (!id || Object.keys(updates).length === 0) {
      // wrapper could capture sync errors
      throw new TypeError(`invalid user value: ${user}`)
    }
    // and also async errors from a rejected `Promise`
    return axios.put(`/users/$id`, updates).then(({ data }) => data)
  },
  (err: unknown) => err as AxiosError | TypeError,
)

An implementation might look like this, which would "hoist" any synchronous errors into the async context.
export const fromThrowable = <Fn extends (...args: readonly any[]) => Promise<any>, E>(
  fn: Fn,
  errorFn: (err: unknown) => E,
): ((...args: Parameters<Fn>) => ResultAsync<ReturnType<Fn> extends Promise<infer R> ? R : never, E>) => {
  return (...args) => {
    try {
      return ResultAsync.fromPromise(fn(...args), errorFn)
    } catch (error) {
      return ResultAsync.fromPromise(Promise.reject(error), errorFn)
    }
  }
}

Better late than never. I wasn't using this library for a while, but I added a PR for this.

It doesn't seem ResultAsync.fromThrowable actually exists in v6.1.0 (latest) package, am I missing something?

Just published now under v6.2.0:
https://github.com/supermacro/neverthrow/releases/tag/v6.2.0

Broken build on npm. There is no dist directory.

Fixed in v6.2.1 .. sorry about that!