Feat request: fromAsync to interop with async functions more easily

Hi! I've been using neverthrow and it's been great! Really enjoying it. I would like to propose 'one more API' 😆 :
The main thing is that we have a bunch of async functions that have known, discrete error types. For example, something that queries the database and if it doesn't find that thing, it returns an error state for the 404. But this means that technically there are two error states - the database query fails (unexpectedly, a 'defect' in Effect terminology), or the thing isn't found (a good ol' error state).
I'd love an API like this, which would let you wrap an async function that returns a Result:
import { ResultAsync, type Result, Err, ok, err } from "neverthrow";

export function fromAsync<A extends readonly any[], IO, IE, E>(
  fn: (...args: A) => Promise<Result<IO, IE | E>>,
  errorFn: (err: unknown) => E
): (...args: A) => ResultAsync<IO, IE | E> {
  return (...args) => {
    return new ResultAsync(
      (async () => {
        try {
          return await fn(...args);
        } catch (error) {
          return new Err(errorFn(error));
        }
      })()
    );
  };
}

// () => ResultAsync<boolean, RangeError | Error>
const a = fromAsync(
  async () => {
    if (Math.random() > 0.5) {
      return err(new RangeError("oh no"));
    }
    // It would be nice to flatten ResultAsync
    // values returned here, but I can't figure this out right now.
    //
    // if (Math.random() > 0.5) {
    //   return ResultAsync.fromPromise(
    //     Promise.resolve(10),
    //     () => new SyntaxError("hi")
    //   );
    // }
    return ok(true);
  },
  () => new Error("hi")
);

It'd also be super nice for this to automatically handle a ResultAsync returned by the async function. This'd give pretty good ergonomics for the common usecase of having an async function with some known error types that you'd like to preserve.

Hello @tmcw ! Thank you for this proposal.
Could you provide an example of what you resort to doing with the current API ?

Sure, here's a (genericized) example from our codebase - in this example and most others, we use Promise<Result> instead of ResultAsync because it's simpler than the alternatives, as far as I can tell.
export async function getItem({
  currentUser,
}: {
  currentUser: CurrentUser;
}): Promise<
  Result<
    {
      intervalId: string | null;
    },
    HttpError
  >
> {
  if (!currentUser) {
    return err(httpErrors.unauthorized("Unauthorized"));
  }

  // Check if the currentUser has access to this evaluation Id
  const [item] = await db
    .select({
      id: items.id,
    })
    .from(items);

  if (!item) {
    return err(httpErrors.notFound("Item not found"));
  }

  return ok({
    intervalId: item.intervalId,
  });
}

but doesn't this approach ruin the whole concept of using Result and ResultAsync? what I mean is that when I (as an external person, contributor, another developer) look at the code and see return type Promise<PutAnyTypeHere> I read it as "potentially can throw an exception", so I need to go inside the method definition and check if I was right or wrong.
on the other hand when I see the return type is Restult or ResultAsync I am sure that the function/method can be called without worries to get an unhandled exception or promise rejection. This is how I'd do it with your provided code sample:
function getItem({
  currentUser,
}: {
  currentUser: CurrentUser | null
}): ResultAsync<{ id: string }, HttpError> {
  // Check if the currentUser has access to this evaluation Id
  if (!currentUser) {
    return errAsync(httpErrors.unauthorized('Unauthorized'))
  }

  return ResultAsync.fromThrowable(
    async () => db.select({ id: items.id }).from(items),
    (err) => httpErrors.internal(err),
  )().andThen((foundItems) => {
    if (foundItems.length === 0) {
      return errAsync(httpErrors.notFound('Item not found'))
    }

    const [item] = foundItems
    return okAsync({ id: item.id })
  })
}

async function main(): Promise<void> {
  await getItem({ currentUser: { name: 'John Doe' } }).match(
    ({ id }) => console.log(`Item found with id: ${id}`),
    (error) => console.log(`got an HttpError: ${error.message}`),
  )
}

main()

hey @tmcw !
but doesn't this approach ruin the whole concept of using Result and ResultAsync? what I mean is that when I (as an external person, contributor, another developer) look at the code and see return type Promise<PutAnyTypeHere> I read it as "potentially can throw an exception", so I need to go inside the method definition and check if I was right or wrong.
on the other hand when I see the return type is Result or ResultAsync I am sure that the function/method can be called without worries to get an unhandled exception or promise rejection. This is how I'd do it with your provided code sample:
function getItem({
  currentUser,
}: {
  currentUser: CurrentUser | null
}): ResultAsync<{ id: string }, HttpError> {
  // Check if the currentUser has access to this evaluation Id
  if (!currentUser) {
    return errAsync(httpErrors.unauthorized('Unauthorized'))
  }

  return ResultAsync.fromThrowable(
    async () => db.select({ id: items.id }).from(items),
    (err) => httpErrors.internal(err),
  )().andThen((foundItems) => {
    if (foundItems.length === 0) {
      return errAsync(httpErrors.notFound('Item not found'))
    }

    const [item] = foundItems
    return okAsync({ id: item.id })
  })
}

async function main(): Promise<void> {
  await getItem({ currentUser: { name: 'John Doe' } }).match(
    ({ id }) => console.log(`Item found with id: ${id}`),
    (error) => console.log(`got an HttpError: ${error.message}`),
  )
}

main()

Maybe my example wasn't clear enough - the fromAsync definition that I laid out will return ResultAsync and would not be able to throw an exception. It calls the given wrapped function from inside of the ResultAsync constructor, and maps any potential errors to a known type.
The main thing that differs here is that it would let me use await. The current pattern of using a non-async function like in your example doesn't work with await - once you call anything asynchronous, you need to switch to chaining with .andThen. In some cases, this means that control flow and variable scopes are going to be trickier to work with than the await syntax, which doesn't require callbacks and chaining. You could shoot yourself in the foot still with this system, but if you do, that rejection is mapped to a known value with the second argument in fromAsync.

oh, now I see what you mean :) yeah, it's a pity that typescript does not allow us to return custom promise implementation in async functions...
what I did for such cases is wrote a helper module that looks like this
import { Err, Ok, Result, ResultAsync, err, ok } from "neverthrow";

type ResultOrOkValue<T, E> =
  | T
  | Ok<T, E>
  | Err<T, E>
  | Ok<ResultOrOkValue<T, E>, E>;

type EitherResult<T, E> = Result<T, E> | ResultAsync<T, E>;
type EitherResultOrOkValue<T, E> = ResultAsync<T, E> | ResultOrOkValue<T, E>;
type NestedEitherResult<T, E> = EitherResult<EitherResultOrOkValue<T, E>, E>;

type Next = [1, 2, 3, 4, 5, ...never[]];
type InferNestedOkType<T, E, Depth extends number = 0> = Depth extends never
  ? never
  : T extends EitherResult<infer U, E>
  ? InferNestedOkType<U, E, Next[Depth]>
  : T;

export function isResult<T, E>(
  maybeResult: EitherResult<T, E> | T | E
): maybeResult is Result<T, E> {
  return maybeResult instanceof Ok || maybeResult instanceof Err;
}

export function isResultAsync<T, E>(
  maybeResult: EitherResult<T, E> | T | E
): maybeResult is ResultAsync<T, E> {
  return maybeResult instanceof ResultAsync;
}

export function isEitherResult<T, E>(
  maybeResult: EitherResult<T, E> | T | E
): maybeResult is EitherResult<T, E> {
  return isResult(maybeResult) || isResultAsync(maybeResult);
}

function flattenAsync<T, E>(
  result: NestedEitherResult<T, E>
): ResultAsync<InferNestedOkType<T, E>, E> {
  const flattenResults = async (): Promise<
    Result<InferNestedOkType<T, E>, E>
  > => {
    const outerResult = await result;

    if (outerResult.isErr()) {
      return err(outerResult.error);
    }

    let outerValue = outerResult.value;

    while (isEitherResult(outerValue)) {
      // eslint-disable-next-line no-await-in-loop
      const inner = await outerValue;

      if (inner.isErr()) {
        return err(inner.error);
      }

      outerValue = inner.value;
    }

    return ok(outerValue as InferNestedOkType<T, E>);
  };

  return new ResultAsync(flattenResults());
}

export function fromActionAsync<T, E>(
  performAction: () => Promise<NestedEitherResult<T, E>>,
  makeError: (error: unknown) => E
): ResultAsync<InferNestedOkType<T, E>, E> {
  return flattenAsync(ResultAsync.fromPromise(performAction(), makeError));
}

fromActionAsync helps to perform whatever async action you want with defaulting to the E error that can also unwrap nested results, if any (as long as the error type as the same, didn't look into how to handle different kinds of errors yet...)
async function main(): Promise<void> {
  const getResult = (): Result<number, string> => ok(1);
  const getResultAsync = (): ResultAsync<number, string> => okAsync(2);
  const asyncAction = async (): Promise<
    ResultAsync<Result<number, Error>, Error>
  > => {
    const first = getResult();

    if (first.isErr()) {
      return errAsync(new Error(first.error));
    }

    const second = await getResultAsync();

    if (second.isErr()) {
      return errAsync(new Error(second.error));
    }

    return okAsync(ok(1));
  };

  const res = await fromActionAsync(asyncAction, Error);

  if (res.isErr()) {
    console.log(`Error: ${res.error}`);
  } else {
    // typeof res.value  === 'number'
    console.log(`Value: ${res.value}`); // 1
  }
}

main();

Yep! Roughly the same idea. I've been updating and using our function, and figured out how to make the second argument, the error mapper, option, so that by default you'll get something like an UnexpectedError or a 'defect' in Effect talk:
export function resultFromAsync<A extends readonly any[], IO, IE>(
  fn: (...args: A) => Promise<Result<IO, IE>>
): (...args: A) => ResultAsync<IO, IE | UnexpectedError>;

export function resultFromAsync<A extends readonly any[], IO, IE, E>(
  fn: (...args: A) => Promise<Result<IO, IE>>,
  errorFn: (err: unknown) => E
): (...args: A) => ResultAsync<IO, IE | E>;

export function resultFromAsync<A extends readonly any[], IO, IE, E>(
  fn: (...args: A) => Promise<Result<IO, IE>>,
  errorFn?: (err: unknown) => E
): (...args: A) => ResultAsync<IO, IE | E | UnexpectedError> {
  return (...args: A) =>
    new ResultAsync<IO, IE | E | UnexpectedError>(
      fn(...args).catch((error) =>
        errorFn
          ? new Err(errorFn(error))
          : new Err(
              new UnexpectedError("Unexpected error (generic)", {
                cause: error,
              })
            )
      )
    );
}


Sync to simple file

It would be a nice feature if we can choose a simple file to sync destination+source without/instead of Github.
This file can be easily shared with GoogleDirve, Dropbox, Nextcloud, FTP, or via e-mail.
Best Regards,
Eper

This can be really good feature!

Duplicate of #256


ui-sref-opts allowedOptions error

https://github.com/angular-ui/ui-router/blob/4da7e5913d504e975a91fbb3d7efd57cbe472077/release/angular-ui-router.js#L4125
Something is wrong here. Why do we need these allowedOptions? It breaks {notify: false} for the directive ui-sref-opts

https://github.com/angular-ui/ui-router/issues/2457#issuecomment-173008225


Neomorphism Accessibility Concerns

Hey @CuriousNikhil! First off, thanks for your public, exploratory work on implementing this relatively new online aesthetic. If nothing else, it's been exciting reading through the codebase as a form of reference for other UI patterns.
With that said, I’m curious how you see someone implementing this with accessibility in mind.  There are some issues when it comes to the neuromorphic design patterns and accessibility. Among the issues, there are:
• Lack of proper contrast
• Lack of distinct contrasting component states
• Increased cognitive load in app usage
I’d love to see more focus on those concerns as you continue building this out! I think it would be a great way to further explore the viability of something like this. If you have any thoughts or questions I’d love to discuss them!

I've updated the tone of the comment above (see also the previous title). I shouldn't have been as harsh as I was, and I apologize. I saw something I felt strongly about, hadn't slept in far too long, and should have waited until the morning to draft this/hit send. I hope we can have a positive discourse about accessibility - my tone won't dip back to that again. Apologies again

Hi @crutchcorn,
I know and I'm aware that there are issues regarding Accessibility when it comes to Neumorphism UI and why that is not being completely adopted piece of tech. I am glad that you have raised this issue. You don't have to apologise for it 😄.
Of course, I would not recommend to use this experimental Neumorphism UI library in production. This is just an experimentation as I am learning and playing with jetpack compose.
I would be happy to discuss related questions that I have with you and would like your inputs on this.

Totally understandable! You're making great progress with it, as well! 😊
Do you think it might be reasonable to include a warning in the README highlighting the accessibility concerns?

Yes. Will add warning in the README for the same.
Thanks.


X11 compilation in Debian 9 Stable ("Stretch") fails

undefined reference
to symbol 'pthread_create@@GLIBC_2.1'
And there's also something missing with a command called ACX or something like that (I had to shut down the PC: will update with more logs soon)
I installed more or less 5 libraries that menction pthread, including libevent and dev libs. I saw that there were debugging symbols that could help, but they are in Buster (current Debian Testing) and in Sid (Debian Unstable).
SDL2 compilation does work, but the twitchiness of clicking in menus (you go to the menu and the button flashes a ton of times, like if it was selected and then released constantly, two or three times per second? also happens with mac) is a bit unwieldy.
You are free to not provide support for this particular failure in compilation, as Debian Stable is known for shipping with very old software that is barely updated (if at all, except for security fixes) until the next release, which happens in 3 year periods (approximately). For work reasons, I am unable to change the PC's repositories to Debian Testing.

Thanks to your report!
I confirmed the problem with Raspbian.
I think that the macro of autoconf is insufficient.
Please try installing 'autoconf-archive'.
$ sudo apt install autoconf-archive

Indeed, the installation of autoconf-archive has fixed the X11 compilation errors. Thank you.


WaitWhileMainHandleIsMissing is slow

WaitWhileMainHandleIsMissing is slow when launching application calling Application.Launch method
private static void Launch()
        {
            var appProcInfo = new ProcessStartInfo
            {
                WindowStyle = ProcessWindowStyle.Maximized,
                FileName = Config.ApplicationPath,
                WorkingDirectory = Directory.GetParent(Config.ApplicationPath).FullName,
                CreateNoWindow = false,
                UseShellExecute = false
            };
            App = Application.Launch(appProcInfo);
            App.WaitWhileMainHandleIsMissing(TimeSpan.FromSeconds(30));
            using var automation = new UIA3Automation();
            var window = App.GetAllTopLevelWindows(automation).First();
            Session = window;
        }

It waits until a window handle (main handle) is available. In the latest release, there is a small fix that correctly refreshes the process in the loop as there is a bug in the .NET Framework that the handle is not being refreshed. Can you try with the latest release? Also you are not using the MainWindowHandle at all as you get the window afterwards via GetAllTopLevelWindows, so you could also just skip it.


Adding "Rewind" and "Fast Forward" (5 and 10s) as Footer Actions

useful on mobile device when no srt file on audiobook (no such keybindings possible)

closed in 1.0.9


Javascript colour

Hello! I love this theme. Thanks for your work.
I've noticed that javascript has a lot of red colour:
https://www.dropbox.com/s/5va22ky6opoc7xj/Screenshot 2018-06-28 10.41.58.png?dl=0
python looks a lot nicer:
https://www.dropbox.com/s/2m3pioskuu84lcm/Screenshot 2018-06-28 10.43.32.png?dl=0
Is this intentional, or is it something that can be changed?

Hello @nrempel !
Happy you like the theme :)
The JS syntax color scheme is intentional, however not bound to be the same forever ;)
I'm open for suggestions! Make changes to the source code and make a PR.
Regards

Thanks @HasseNasse, I'll take a look soon.


Fix NodeNotHealthy alert with correct NodeNotReady taint

How to categorize this PR?

/area monitoring
/kind bug
What this PR does / why we need it:
This PR corrects the filtering query, as the current value v1beta1constants.LabelNodeCriticalComponent is wrong. The query was thought initially to handle node.gardener.cloud/critical-components-not-ready , hence the correct value should be v1beta1constants.TaintNodeCriticalComponentsNotReady
Which issue(s) this PR fixes:
False alerts of nodes being reported as not healthy.
Special notes for your reviewer:
Release note:

Correct NodeNotHealthy filtering query to use `v1beta1constants.TaintNodeCriticalComponentsNotReady`

/retest

/retest


Draft: Tests stucks at MacOS

Often I face an issue when the test gets stuck right after the start. REP file of specified test is usually empty this time.
It happens only in MacOS (at Ubuntu all works fine).
Unfortunately, I can't reproduce this now (just yesterday it happens each 2nd run)

It start happens again (Docker project):
Pre builds steps
docker build -t dnd:latest ./clt_tests/testkit/

RUN_ARGS="-v $(pwd):/docker --privileged" ../clt/clt test -d -t ./clt_tests/tests/plain-indexes-crontab.rec dnd:latest
### It runs normally

RUN_ARGS="-v $(pwd):/docker --privileged" ../clt/clt test -d -t ./clt_tests/tests/plain-indexes-crontab2.rec dnd:latest
### This run sticks

During debugging, I discovered a magic random bytes link inside the container which seems to get stuck on OSX. There doesn't appear to be anything related to CLT, so the issue might just be with the Docker setup inside the container we use for testing or with domain resolving.
I recommend removing the external URL from the Dockerfile and using a different method, as this is also not secure.

P.S. It's done after 200s that is just huge delay but nothing wrong in CLT I found: [+] Building 201.2s (12/24)                                     docker:default

Unfortunately, the build step in any tes it's 3th step. So in case your test is stuck at that step, your REP file already will be filled by some logs. In my case rep file empty

@djklim87 As discussed, please prepare an example of a hanging test without dind.


Customize logo.png

Hi,
I can't replace the logo.png used in redoc. I have placed the logo in /web in the master branch and it didn't work, then I placed it in /web in gh-pages branch with the same result, the default logo keeps unchanged.
This is my repo: https://github.com/SmartNodeLabs/snl-appointments-api-doc
Please can you give me some clue about what I'm doing wrong?
Best regards and thank you for your work in this project!

Hi @fsanzv,
change URL to logo here: https://github.com/SmartNodeLabs/snl-appointments-api-doc/blob/master/spec/swagger.yaml#L37
This is a bug, which we will fix.

Thank you very much, solved!
Best regards


Error message when importing statblock

I get the message "type error: cannot set property "total" of undefined;" when I import a statblock

Hey, could you post a link to the statblock? Then i can take a look

Wang Chung Lee.txt

This is the statblock out of Herolabs. It is for an archtype ranger out of Pirates of the Inner Sea


Add cli flag --mermaid

Resolves L2B-1696

When running mermaid at the end of the output show the link to mermaid.live, maybe also copy markup to clipboard?

When running mermaid at the end of the output show the link to mermaid.live, maybe also copy markup to clipboard?
sindresorhus/clipboardy

I hope you like the newest commit 😃

what about adding yarn invert [project]?

That's how it works rn :)

That's how it works rn :)

Love it!


Update Spek version to 2.0.10

Release notes:
https://github.com/spekframework/spek/releases/tag/2.0.10

Codecov Report

Merging #2418 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #2418   +/-   ##
=========================================
  Coverage     82.89%   82.89%
  Complexity     2157     2157
=========================================
  Files           354      354
  Lines          6134     6134
  Branches       1120     1120
=========================================
  Hits           5085     5085
  Misses          475      475
  Partials        574      574

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b513df4...3f5ee09. Read the comment docs.


GPU

I wonder what's the size of your gpu? Thank you~

Per my experience, the code runs on a V100 (32GB memory) but not on a GTX2080 (12GB memory)


Custom CAN transport ignoring some messages in communication between STM and agent

Issue template

Hardware description: STM32F446RET6 custom board identical to 446 NUCLEO
RTOS: FreeRTOS
Installation type: static lib
Version or commit hash: humble

Steps to reproduce the issue
Im trying to implement custom transport for CAN (agent and stm part has been asseembled and works) and face an issue... Some of CAN frames ignoing by agent. Session established and some messaging starts, but no topic has been initialized.  What is curious that in candump I can see messages with corret header "7E" which are not showing in debug log of agent. Therefore the same ROS2 freertos task (example one) works well with default serial transport. Could anydoby take a look and make a suggestion of what im doing wrong.
Additional information
Code of agent:
#include <uxr/agent/transport/custom/CustomAgent.hpp>
#include <uxr/agent/transport/endpoint/CustomEndPoint.hpp>

#include <poll.h>
#include <sys/socket.h>
#include <unistd.h>
#include <signal.h>
#include <netinet/in.h>
#include <arpa/inet.h>

#include <uxr/agent/transport/can/CanAgentLinux.hpp>
#include <uxr/agent/utils/Conversion.hpp>
#include <uxr/agent/logger/Logger.hpp>

#include <net/if.h>
#include <sys/ioctl.h>

#include <linux/can.h>
#include <linux/can/raw.h>

/**
 * This custom XRCE Agent example attempts to show how easy is for the user to define a custom
 * Micro XRCE-DDS Agent behaviour, in terms of transport initialization and closing, and also
 * regarding read and write operations.
 * For this simple case, an UDP socket is opened on port 8888. Additionally, some information
 * messages are being printed to demonstrate the custom behaviour.
 * As the endpoint is already defined, we are using the provided
 * eprosima::uxr::IPv4EndPoint` by the library.
 * Other transport protocols might need to implement their own endpoint struct.
 */

int main(int argc, char** argv)
{
    eprosima::uxr::Middleware::Kind mw_kind(eprosima::uxr::Middleware::Kind::FASTDDS);

    struct pollfd poll_fd;
    struct sockaddr_can addr;
    struct ifreq ifr;

    /**
     * @brief Agent's initialization behaviour description.
     */
    eprosima::uxr::CustomAgent::InitFunction init_function = [&]() -> bool
    {
        //struct sockaddr_can addr;
        struct canfd_frame frame;
        struct can_frame rec_frame;

        bool rv = false;

        if ((poll_fd.fd = socket(PF_CAN, SOCK_RAW, CAN_RAW)) < 0)
        {
		perror("socket creation: ");
		return false;
        }

        if (-1 != poll_fd.fd)
        {
            struct sockaddr_in address{};
            strcpy(ifr.ifr_name, "can0");
            ioctl(poll_fd.fd, SIOCGIFINDEX, &ifr);
            poll_fd.events = POLLIN;
            addr.can_family  = AF_CAN;
            addr.can_ifindex = ifr.ifr_ifindex;
            if (-1 != bind(poll_fd.fd,
                (struct sockaddr *)&addr,
                sizeof(addr)))
                {
                    rv = true;
                }
            else
            {
                perror("socket bind: ");
            }
        }
            {
                rv = true;
                UXR_AGENT_LOG_INFO(
                    UXR_DECORATE_GREEN(
                        "This is an example of a custom Micro XRCE-DDS Agent INIT function"),
                    "fd: {}",
                    poll_fd.fd);
            }
        return rv;
    };

    /**
     * @brief Agent's destruction actions.
     */
    eprosima::uxr::CustomAgent::FiniFunction fini_function = [&]() -> bool
    {
        if (-1 == poll_fd.fd)
        {
            return true;
        }

        if (0 == close(poll_fd.fd))
        {
            poll_fd.fd = -1;

            UXR_AGENT_LOG_INFO(
                UXR_DECORATE_GREEN(
                    "This is an example of a custom Micro XRCE-DDS Agent FINI function"),
                "fd: {}",
                poll_fd.fd);

            return true;
        }
        else
        {
            return false;
        }
    };

    /**
     * @brief Agent's incoming data functionality.
     */
    eprosima::uxr::CustomAgent::RecvMsgFunction recv_msg_function = [&](
            eprosima::uxr::CustomEndPoint* source_endpoint,
            uint8_t* buffer,
            size_t buffer_length,
            int timeout,
            eprosima::uxr::TransportRc& transport_rc) -> ssize_t
    {
        struct canfd_frame frame;
        uint16_t rv;
        rv = 8;
        read(poll_fd.fd,&frame,sizeof(struct canfd_frame));
        memcpy(buffer,&(frame.data),rv);
        source_endpoint->set_member_value<uint32_t>("ID",frame.can_id);
        transport_rc = eprosima::uxr::TransportRc::ok;
        return rv;
    };

    /**
     * @brief Agent's outcoming data flow definition.
     */
    eprosima::uxr::CustomAgent::SendMsgFunction send_msg_function = [&](
        const eprosima::uxr::CustomEndPoint* destination_endpoint,
        uint8_t* buffer,
        size_t message_length,
        eprosima::uxr::TransportRc& transport_rc) -> ssize_t
    {
        struct can_frame frame;
        uint8_t * ptr = buffer;
        int rv;

        ssize_t bytes_sent = 0;
        frame.can_id = 12;
        frame.can_dlc = 8;

        uint16_t cycle, rest;

        cycle = message_length / frame.can_dlc;
        rest = message_length % frame.can_dlc;

        for(cycle; cycle>0; cycle--)
        {
            //TODO add poll to get free fd
            frame.can_dlc = 8;
            memcpy(&(frame.data),ptr,frame.can_dlc);
            rv = write(poll_fd.fd, &frame, sizeof(struct can_frame));
            if (rv != -1)
            {
                bytes_sent += 8;
            }
            ptr += 8;
        }
        if (rest!=0)
        {
            frame.can_dlc = rest;
            memcpy(&(frame.data),ptr,frame.can_dlc);
            rv = write(poll_fd.fd,&frame,sizeof(can_frame));
            if (rv != -1)
            {
                bytes_sent += rest;
            }
        }
        transport_rc = eprosima::uxr::TransportRc::ok;
        return bytes_sent;
    };

    /**
     * Run the main application.
     */
    try
    {
        /**
         * EndPoint definition for this transport. We define an address and a port.
         */
        eprosima::uxr::CustomEndPoint custom_endpoint;
        custom_endpoint.add_member<uint32_t>("ID");
        /**
         * Create a custom agent instance.
         */
        eprosima::uxr::CustomAgent custom_agent(
            "CAN_BUS",
            &custom_endpoint,
            mw_kind,
            true,
            init_function,
            fini_function,
            send_msg_function,
            recv_msg_function);

        /**
         * Set verbosity level
         */
        custom_agent.set_verbose_level(6);

        /**
         * Run agent and wait until receiving an stop signal.
         */
        custom_agent.start();

        int n_signal = 0;
        sigset_t signals;
        sigwait(&signals, &n_signal);

        /**
         * Stop agent, and exit.
         */
        custom_agent.stop();
        return 0;
    }
    catch (const std::exception& e)
    {
        std::cout << e.what() << std::endl;
        return 1;
    }
}

code of custom transport for STM32
#include <uxr/client/transport.h>

#include <rmw_microxrcedds_c/config.h>

#include "main.h"
#include "cmsis_os.h"

#include <unistd.h>
#include <stdio.h>
#include <string.h>
#include <stdbool.h>

#ifdef RMW_UXRCE_TRANSPORT_CUSTOM

// --- micro-ROS Transports ---
#define CAN_DMA_BUFFER_SIZE 2048
uint32_t can_head = 0;
uint32_t can_tail = 0;
uint8_t can_buffer[CAN_DMA_BUFFER_SIZE];

extern CAN_TxHeaderTypeDef TxHeader;
extern uint32_t TxMailbox;

bool cubemx_transport_open(struct uxrCustomTransport * transport){
    return true;
}

bool cubemx_transport_close(struct uxrCustomTransport * transport){
    return true;
}

size_t cubemx_transport_write(struct uxrCustomTransport* transport, uint8_t * buf, size_t len, uint8_t * err){
	CAN_HandleTypeDef * can = (CAN_HandleTypeDef*) transport->args;
    HAL_StatusTypeDef ret = HAL_ERROR;
    uint8_t rv = 0;

	ret = HAL_CAN_AddTxMessage(can, &TxHeader, buf, &TxMailbox);
	while(HAL_CAN_IsTxMessagePending(can, TxMailbox));

	rv =  (len < 8) ? len : 8;
    return (ret == HAL_OK) ? rv : 0;
   }

size_t cubemx_transport_read(struct uxrCustomTransport* transport, uint8_t* buf, size_t len, int timeout, uint8_t* err){
    size_t bytes_recv = 0;

    while ((can_head != can_tail) && (bytes_recv < len)){
        buf[bytes_recv] = can_buffer[can_head];
        can_head = (can_head + 1) % CAN_DMA_BUFFER_SIZE;
        bytes_recv++;
    }
    return bytes_recv;
	}

void HAL_CAN_RxFifo0MsgPendingCallback(CAN_HandleTypeDef *hcan){

CAN_RxHeaderTypeDef RxHeader;
uint8_t data[8];

HAL_CAN_GetRxMessage(hcan, CAN_RX_FIFO0, &RxHeader, data);
	if (RxHeader.StdId == ROUTER_ID)
	{
		for(uint16_t i=0; i<RxHeader.DLC; i++)
			{
			can_buffer[can_tail] = data[i];
			can_tail++;
			}
	}
}

#endif //RMW_UXRCE_TRANSPORT_CUSTOM

ROS2 Task
void StartDefaultTask(void *argument)
{
  /* USER CODE BEGIN StartDefaultTask */
	  // micro-ROS configuration

	  rmw_uros_set_custom_transport(
	    true,
	    (void *) &hcan1,
	    cubemx_transport_open,
	    cubemx_transport_close,
	    cubemx_transport_write,
	    cubemx_transport_read);

	  rcl_allocator_t freeRTOS_allocator = rcutils_get_zero_initialized_allocator();
	  freeRTOS_allocator.allocate = microros_allocate;
	  freeRTOS_allocator.deallocate = microros_deallocate;
	  freeRTOS_allocator.reallocate = microros_reallocate;
	  freeRTOS_allocator.zero_allocate =  microros_zero_allocate;

	  if (!rcutils_set_default_allocator(&freeRTOS_allocator)) {
	      printf("Error on default allocators (line %d)\n", __LINE__);
	  }

	  // micro-ROS app

	  rcl_publisher_t publisher;
	  std_msgs__msg__Int32 msg;
	  rclc_support_t support;
	  rcl_allocator_t allocator;
	  rcl_node_t node;

	  allocator = rcl_get_default_allocator();

	  //create init_options
	  rclc_support_init(&support, 0, NULL, &allocator);

	  // create node
	  rclc_node_init_default(&node, "cubemx_node", "", &support);

	  // create publisher
	  rclc_publisher_init_default(
	    &publisher,
	    &node,
	    ROSIDL_GET_MSG_TYPE_SUPPORT(std_msgs, msg, Int32),
	    "cubemx_publisher");

	  msg.data = 0;

	  for(;;)
	  {
	    rcl_ret_t ret = rcl_publish(&publisher, &msg, NULL);
	    if (ret != RCL_RET_OK)
	    {
	      printf("Error publishing (line %d)\n", __LINE__);
	    }

	    msg.data++;
	    osDelay(100);
	  }

candump log
00F - STM32
00C - Agent

same messages in agent debug

Closing issue. Got some unexpected behaviour in write function but managed to handle it. For anyone who will try the same transport correct code listed below. The code is not clear and should be brushed up. Also timeouts should be implemented on both sides but it works.
Agent side
#include <uxr/agent/transport/custom/CustomAgent.hpp>
#include <uxr/agent/transport/endpoint/CustomEndPoint.hpp>

#include <poll.h>
#include <sys/socket.h>
#include <unistd.h>
#include <signal.h>
#include <netinet/in.h>
#include <arpa/inet.h>

#include <uxr/agent/transport/can/CanAgentLinux.hpp>
#include <uxr/agent/utils/Conversion.hpp>
#include <uxr/agent/logger/Logger.hpp>

#include <net/if.h>
#include <sys/ioctl.h>

#include <linux/can.h>
#include <linux/can/raw.h>

/**
 * This custom XRCE Agent example attempts to show how easy is for the user to define a custom
 * Micro XRCE-DDS Agent behaviour, in terms of transport initialization and closing, and also
 * regarding read and write operations.
 * For this simple case, an UDP socket is opened on port 8888. Additionally, some information
 * messages are being printed to demonstrate the custom behaviour.
 * As the endpoint is already defined, we are using the provided
 * `eprosima::uxr::IPv4EndPoint` by the library.
 * Other transport protocols might need to implement their own endpoint struct.
 */

int main(int argc, char** argv)
{
    eprosima::uxr::Middleware::Kind mw_kind(eprosima::uxr::Middleware::Kind::FASTDDS);

    struct pollfd poll_fd;
    struct sockaddr_can addr;
    struct ifreq ifr;

    /**
     * @brief Agent's initialization behaviour description.
     */
    eprosima::uxr::CustomAgent::InitFunction init_function = [&]() -> bool
    {
        //struct sockaddr_can addr;
        struct canfd_frame frame;
        struct can_frame rec_frame;

        bool rv = false;

        if ((poll_fd.fd = socket(PF_CAN, SOCK_RAW, CAN_RAW)) < 0)
        {
		perror("socket creation: ");
		return false;
        }

        if (-1 != poll_fd.fd)
        {
            struct sockaddr_in address{};
            strcpy(ifr.ifr_name, "can0");
            ioctl(poll_fd.fd, SIOCGIFINDEX, &ifr);
            poll_fd.events = POLLIN;
            addr.can_family  = AF_CAN;
            addr.can_ifindex = ifr.ifr_ifindex;
            if (-1 != bind(poll_fd.fd,
                (struct sockaddr *)&addr,
                sizeof(addr)))
                {
                    rv = true;
                }
            else
            {
                perror("socket bind: ");
            }
        }
            {
                rv = true;
                UXR_AGENT_LOG_INFO(
                    UXR_DECORATE_GREEN(
                        "This is an example of a custom Micro XRCE-DDS Agent INIT function"),
                    "fd: {}",
                    poll_fd.fd);
            }
        return rv;
    };

    /**
     * @brief Agent's destruction actions.
     */
    eprosima::uxr::CustomAgent::FiniFunction fini_function = [&]() -> bool
    {
        if (-1 == poll_fd.fd)
        {
            return true;
        }

        if (0 == close(poll_fd.fd))
        {
            poll_fd.fd = -1;

            UXR_AGENT_LOG_INFO(
                UXR_DECORATE_GREEN(
                    "This is an example of a custom Micro XRCE-DDS Agent FINI function"),
                "fd: {}",
                poll_fd.fd);

            return true;
        }
        else
        {
            return false;
        }
    };

    /**
     * @brief Agent's incoming data functionality.
     */
    eprosima::uxr::CustomAgent::RecvMsgFunction recv_msg_function = [&](
            eprosima::uxr::CustomEndPoint* source_endpoint,
            uint8_t* buffer,
            size_t buffer_length,
            int timeout,
            eprosima::uxr::TransportRc& transport_rc) -> ssize_t
    {
        struct can_frame frame;
        read(poll_fd.fd,&frame,sizeof(struct can_frame));
        if (frame.can_id == 15)
        {
            memcpy(buffer,&(frame.data),static_cast<int>(frame.can_dlc));
            source_endpoint->set_member_value<uint32_t>("ID",frame.can_id);
            transport_rc = eprosima::uxr::TransportRc::ok;
            return static_cast<int>(frame.can_dlc);
        }
        else
        {
            perror("wrong ID");
            return -1;
        }
    };

    /**
     * @brief Agent's outcoming data flow definition.
     */
    eprosima::uxr::CustomAgent::SendMsgFunction send_msg_function = [&](
        const eprosima::uxr::CustomEndPoint* destination_endpoint,
        uint8_t* buffer,
        size_t message_length,
        eprosima::uxr::TransportRc& transport_rc) -> ssize_t
    {
        struct can_frame frame;
        uint8_t * ptr = buffer;
        int rv;

        ssize_t bytes_sent = 0;
        frame.can_id = 12;
        frame.can_dlc = 8;

        uint16_t cycle, rest;

        cycle = message_length / frame.can_dlc;
        rest = message_length % frame.can_dlc;

        for(cycle; cycle>0; cycle--)
        {
            //TODO add poll to get free fd
            frame.can_dlc = 8;
            memcpy(&(frame.data),ptr,frame.can_dlc);
            rv = write(poll_fd.fd, &frame, sizeof(struct can_frame));
            if (rv != -1)
            {
                bytes_sent += 8;
            }
            ptr += 8;
        }
        if (rest != 0)
        {
            frame.can_dlc = rest;
            memcpy(&(frame.data), ptr, rest);
            rv = write(poll_fd.fd, &frame,  sizeof(struct can_frame));
            if (rv != -1)
            {
                bytes_sent += rest;
            }
        }
        transport_rc = eprosima::uxr::TransportRc::ok;
        return bytes_sent;
    };

    /**
     * Run the main application.
     */
    try
    {
        /**
         * EndPoint definition for this transport. We define an address and a port.
         */
        eprosima::uxr::CustomEndPoint custom_endpoint;
        custom_endpoint.add_member<uint32_t>("ID");
        /**
         * Create a custom agent instance.
         */
        eprosima::uxr::CustomAgent custom_agent(
            "CAN_BUS",
            &custom_endpoint,
            mw_kind,
            true,
            init_function,
            fini_function,
            send_msg_function,
            recv_msg_function);

        /**
         * Set verbosity level
         */
        custom_agent.set_verbose_level(6);

        /**
         * Run agent and wait until receiving an stop signal.
         */
        custom_agent.start();

        int n_signal = 0;
        sigset_t signals;
        sigwait(&signals, &n_signal);

        /**
         * Stop agent, and exit.
         */
        custom_agent.stop();
        return 0;
    }
    catch (const std::exception& e)
    {
        std::cout << e.what() << std::endl;
        return 1;
    }
}

dma_transport.c - STM32 side
#include <uxr/client/transport.h>

#include <rmw_microxrcedds_c/config.h>

#include "main.h"
#include "cmsis_os.h"

#include <unistd.h>
#include <stdio.h>
#include <string.h>
#include <stdbool.h>

#ifdef RMW_UXRCE_TRANSPORT_CUSTOM

// --- micro-ROS Transports ---
#define CAN_IT_BUFFER_SIZE 2048
uint64_t can_head = 0;
uint32_t can_tail = 0;
uint8_t can_buffer[CAN_IT_BUFFER_SIZE];

extern uint8_t TxData[8];					// Data to be sent via CAN
extern uint64_t TxMailbox;					// CAN temporary mailbox. Required by HAL function
extern CAN_TxHeaderTypeDef TxHeader;		// Header for can message
extern CAN_FilterTypeDef canfilterconfig;	// Filter for receiving CAN messages

bool cubemx_transport_open(struct uxrCustomTransport * transport){
    return true;
}

bool cubemx_transport_close(struct uxrCustomTransport * transport){
    return true;
}

size_t cubemx_transport_write(struct uxrCustomTransport* transport, uint8_t * buf, size_t len, uint8_t * err){

	CAN_HandleTypeDef * can = (CAN_HandleTypeDef*) transport->args;
    HAL_StatusTypeDef ret = HAL_ERROR;
    uint8_t rv = 0;

    rv = (len < 8) ? len : 8;
    TxHeader.DLC = rv;

	ret = HAL_CAN_AddTxMessage(can, &TxHeader, buf, &TxMailbox);
	while(HAL_CAN_IsTxMessagePending(can, TxMailbox));

    return (ret == HAL_OK) ? rv : 0;
   }

size_t cubemx_transport_read(struct uxrCustomTransport* transport, uint8_t* buf, size_t len, int timeout, uint8_t* err){
    size_t rcv = 0;

    while ((can_head != can_tail) && (rcv < len)){
        buf[rcv] = can_buffer[can_head];
        can_head = (can_head + 1) % CAN_IT_BUFFER_SIZE;
        rcv++;
    }
	return rcv;
    }

void HAL_CAN_RxFifo0MsgPendingCallback(CAN_HandleTypeDef *hcan){

HAL_StatusTypeDef ret = HAL_ERROR;
CAN_RxHeaderTypeDef RxHeader;
uint8_t data[8];

ret = HAL_CAN_GetRxMessage(hcan, CAN_RX_FIFO0, &RxHeader, data);

if (ret != HAL_OK)
{
	HAL_GPIO_WritePin(GPIOD, GPIO_PIN_2, 1); //TODO ERROR HANDLER
}

if (RxHeader.StdId == ROUTER_ID)
	{
		for(uint16_t i=0; i<RxHeader.DLC; i++)
			{
			if(can_tail == CAN_IT_BUFFER_SIZE)
			{
			    can_tail = 0;
			}
			can_buffer[can_tail] = data[i];
			can_tail++;
			}
	}
}

#endif //RMW_UXRCE_TRANSPORT_CUSTOM

Got some further difficulties... I've implemented timeouts in both agent's and client's sides and everithing work pretty well except reconnect with agent. The behaveour is as follows

The client - STM32 starts upfront. It starts to send session create packets and if agent is alive everithing goes well.
If client restarts during session everithing works aswell.
But if we restarts agent during session the client would keep sending a packet:
can0  00F   [8]  7E 00 00 0D 00 81 00 00
can0  00F   [8]  00 0B 01 05 00 EB 0B EE
can0  00F   [4]  0B 80 5A 2A
which is not recognizable by client as session reconnect. Actually the agent is not responding for this packet at all even the agent reconize it as valid packet.
So the question is if client or agent has any kind procedure to restart session if it broken due to timeout?

Yes, check this approach: micro-ros_reconnection_example
The idea is to use the ping to check if the agent is reachable. If the connection is lost, entities are destroyed and created again once the agent is ready again.

Great thanks. Will follow the idea.

Hello again looks like rmw_uros_ping_agent(args) is not RTOS safe... Will try to write smthing similar on my own.

Hello again looks like rmw_uros_ping_agent(args) is not RTOS safe

Could you elaborate on this?

Seems that it takes a lot of work to wrap ping into RTOS function. And since ping is just "empty" session i've made simple workaround. It counts unsuccess write attempts and if the number of such failed writes overwhelms a value i destroy and recreate entities. Also I added a watchdog which resets the STM then writes callback stops to be called.

Seems that it takes a lot of work to wrap ping into RTOS function. And since ping is just "empty" session i've made simple workaround. It counts unsuccess write attempts and if the number of such failed writes overwhelms a value i destroy and recreate entities. Also I added a watchdog which resets the STM then writes callback stops to be called.


Configurable prompts

Follow-up of #180
The prompts should follow the options passed to the tasks runner.
@webpro Fell free to edit the README modifications, not sure how to best explain this.

Thanks @ngotchac, looks good. Could you please point this PR to the v6 branch instead of the master branch? This is quite a change, so I'd prefer to have it in the next major release (even though it's not necessarily a breaking change).

Sure !


Doclinks

Fix up the link to the guidelines.

This is no longer necessary. It was already fixed in https://github.com/Azure/azure-sdk-for-cpp/pull/1225


Add Validators FAQ

Validators FAQ initial version.
Will be integrated in website (cosmos.network/validators/faq)

By initial version, do you mean it's ready to be put on the website? Or are we waiting for a final version.

After this PR is reviewed and merged we can put the FAQ on the website. Then I expect it to be frequently updated, but at least we'll have a better process with github's features

Looks good to me - please make sure to address all of @adrianbrink comments - once this is the case I think we're good to merge - However I'll first confirm with Jae or Bucky before we actually merge in here

Adrian's comment are integrated, see commits history


try.gitea.io not being updated

Noticed that https://try.gitea.io/ does not seem to be updated any more. The version info at the bottom shows that https://github.com/go-gitea/gitea/commit/cd8cdbd9d29b2c8c49102f30a0af98378af43777 was the last commit that was built.

Resolved. Thanks for pointing this out 😄


[BYOC-DNNL] enable conv3d->bn folding

This PR contains two modifications:

support conv3d-bn folding.
add some layout in dnnl_json_runtime, and fix some typos.

Is it possible to unify 2D and 3D impls?
Also please add test cases.

Is it possible to identify conv2d or conv3d from the given callnode?

yeah, for example
https://github.com/apache/tvm/blob/4322ebc38797d8d76838db8be5918459ac3d0958/src/relay/backend/utils.h#L427-L432

yeah, for example
https://github.com/apache/tvm/blob/4322ebc38797d8d76838db8be5918459ac3d0958/src/relay/backend/utils.h#L427-L432

Thank you so much! I will try to unify 2D and 3D impls.

Hi @masahi, would you please review this PR and give me some suggestions?


Draco does not appear to compress some models, including some Kronos sample models

Hi,
I am having difficulty with Draco compression. I find that gltf-pipeline does not seem to apply any Draco compression using the -d switch (giving no error), on my own glTF file outputted from the THREE.js glTF exporter (probably my fault!), or on at least two of the Kronos glTF example files:
Avacado.glb does not seem to compress https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/Avocado/glTF-Binary
BoomBox.glb does not seem to compress https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/BoomBox/glTF-Binary
GearboxAssy.glb ** does compress ** https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/GearboxAssy/glTF-Binary
Perhaps you could use these examples as a clue as to what is preventing Draco from compressing some models? A debug message such as "Could not apply Draco because model ... " would also be really helpful! In particular I would really like to find out why my model will not compress, and fix it.
alea.zip
Thanks,
Jamie

All those models get compressed when I try them. alea.glb goes from 2.6MB to 81.3KB.
Can you post the command you're using to run gltf-pipeline? The command I'm using is:

node bin/gltf-pipeline.js -i alea.glb -o alea-draco.glb -d

Thanks for getting back to me. I have realised I made a very basic mistake with my npm upgrade and was mistakenly running an old version of gltf-pipeline (2.0). How embarrassing! Please accept my apologies & feel free to close or delete this issue. Thanks again. PS that is a great compression ratio! :)


Added more bot rules

Do we want to leave a note about how the bot can get into the super-explode state it was in yesterday? Maybe leave a note about what to do if that happens, or how to stop it?

There's not anything specific to be done there except to figure out why it's being a butt and updating its rules. So I don't have any guidance to add.

Also, I'm overriding this because it's a doc update and the CI isn't going to tell us anything about it.


Issue Generating c# files for icrc1_ledger ckbtc

I updated the c# generator to the latest (3.1.2) I tried to generate the c# files for icrc1_ledger ckbtc using its canister id . There is a bug when generating its files, The "Candid.Icrc1Ledger.Models.Value" is the only one being affected. The Value field's name is identical to the class name and t
image 1: https://github.com/edjCase/ICP.NET/assets/44788655/da79f864-c95c-4bff-a9fa-06bfe070900f
image 2: https://github.com/edjCase/ICP.NET/assets/44788655/c3630538-7730-417c-bcac-5d2f8285bc2f

https://github.com/edjCase/ICP.NET/releases/tag/3.1.3


session_login.cgi doesn't print footer

Hello,
I wanted to use the Theme Extension feature with script.js to modify the login page. Even though it was mentioned here that is should work, I didn't see any js in the login page.
As far as I can tell the problem is with this line
((!$miniserv::theme_header_captured && !$miniserv::page_capture) && return);

in theme_footer. session_login.cgi doesn't use &header so theme_header is never called and so this is never called
$miniserv::page_capture          = 1;
$miniserv::theme_header_captured = 1;

which would enable actually printing the footer including embed_js_scripts();
Just placing $miniserv::page_capture = 1; before &footer(); is called, in session_login.cgi didn't work for me, also seems like a dirty fix. I'm not sure where this should be fixed.
Greetings

@iliajie can you take a look at this one?

Well, this is expected. We don't print script.js in the login page.

@iliajie, thank you for the quick fix!


Add JUnit test for 'cancel' command

Added extra test case for the cancel command unit test

Codecov Report

Merging #88 (11b2157) into master (19b42cb) will increase coverage by 0.11%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master      #88      +/-   ##
============================================
+ Coverage     73.75%   73.87%   +0.11%
- Complexity      548      549       +1
============================================
  Files            90       90
  Lines          1772     1772
  Branches        193      193
============================================
+ Hits           1307     1309       +2
+ Misses          389      388       -1
+ Partials         76       75       -1

Impacted Files
Coverage Δ

...va/seedu/address/logic/commands/CancelCommand.java
88.23% <0.00%> (+11.76%)
:arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Configuring sourceLink per task is incompatible with configuration caching

The following snippet introduces references to objects that cause incompatibilities with Gradle's configuration caching:
tasks {
    dokkatooGenerateModuleHtml.configure {
        dokkaSourceSets.configureEach {
            sourceLink {
                // ...
            }
        }
    }
}

Writing the configuration cache fails (see also):
⌄ task:dokkatooGenerateModuleHtml of type dev.adamko.dokkatoo.tasks.DokkatooGenerateTask
  ⌄ field__dokkaSourceSets__ of dev.adamko.dokkatoo.tasks.DokkatooGenerateTask
    ⌄ bean of type org.gradle.api.internal.FactoryNamedDomainObjectContainer
      ⌄ fieldeventRegister of org.gradle.api.internal.FactoryNamedDomainObjectContainer
        ⌄ bean of type org.gradle.api.internal.collections.DefaultCollectionEventRegister
          ⌄ fieldaddActions of org.gradle.api.internal.collections.DefaultCollectionEventRegister
            ⌄ bean of type org.gradle.internal.ImmutableActionSet$SetWithFewActions
              ⌄ fieldactions of org.gradle.internal.ImmutableActionSet$SetWithFewActions
                ⌄ bean of type org.gradle.api.internal.DefaultMutationGuard$1
                  ⌄ fieldval$action of org.gradle.api.internal.DefaultMutationGuard$1
                    ⌄ bean of type org.gradle.api.internal.DefaultMutationGuard$1
                      ⌄ fieldval$action of org.gradle.api.internal.DefaultMutationGuard$1
                        ⌄ bean of type org.gradle.api.internal.DefaultCollectionCallbackActionDecorator$BuildOperationEmittingAction
                          ⌄ fielddelegate of org.gradle.api.internal.DefaultCollectionCallbackActionDecorator$BuildOperationEmittingAction
                            ⌄ bean of type org.gradle.configuration.internal.DefaultUserCodeApplicationContext$CurrentApplication$1
                              ⌄ fieldval$action of org.gradle.configuration.internal.DefaultUserCodeApplicationContext$CurrentApplication$1
                                ⌄ bean of type Build_gradle$4$5$1
                                  ⌄ fieldthis$0 of Build_gradle$4$5$1
                                    ■⚠cannot serialize Gradle script object references as these are not supported with the configuration cache.[ ?](https://docs.gradle.org/8.1/userguide/configuration_cache.html#config_cache:requirements:disallowed_types)

This issue does not occur when configuring the sourceLink via the dokkatoo extension.

Thanks for the issue, I've made a reproducer: https://github.com/adamko-dev/dokkatoo/pull/55
It seems to specifically be trigger by cross-project dependencies. I couldn't reproduce it in a single project.
I don't know what the cause might be. Unfortunately Gradle's Configuration Cache reports are essentially useless, they don't help diagnose anything.

Sorry, I forgot to talk about the project where I encountered this issue: https://github.com/GW2ToolBelt/api-generator/tree/0dae12da6a993333511227b59d7905ac515d6815. I didn't prepare a reproducer because the build is fairly straightforward. All relevant stuff is happening in build.gradle.kts.
There are no cross-project dependencies involved in my build. The relevant sections should be:
plugins {
    alias(libs.plugins.binary.compatibility.validator)
    alias(libs.plugins.dokkatoo.html) // 1.1.1
    alias(libs.plugins.dokkatoo.javadoc) // 1.1.1
    alias(libs.plugins.gradle.toolchain.switches)
    alias(libs.plugins.kotlin.jvm) // 1.8.20
    alias(libs.plugins.kotlin.plugin.serialization) // 1.8.20
    id("com.gw2tb.maven-publish-conventions")
}

(https://github.com/GW2ToolBelt/api-generator/blob/0dae12da6a993333511227b59d7905ac515d6815/build.gradle.kts#L28-L36)
tasks {
    dokkatooGenerateModuleHtml.configure {
        outputDirectory.set(layout.buildDirectory.dir("mkdocs/sources/api"))

        dokkaSourceSets.configureEach {
            reportUndocumented.set(true)
            skipEmptyPackages.set(true)
            jdkVersion.set(8)

            sourceLink {
                localDirectory.set(file("src/main/kotlin"))
                remoteUrl.set(URL("https://github.com/GW2ToolBelt/api-generator/tree/v${version}/src/main/kotlin"))
                remoteLineSuffix.set("#L")
            }
        }
    }
}

(https://github.com/GW2ToolBelt/api-generator/blob/0dae12da6a993333511227b59d7905ac515d6815/build.gradle.kts#L89-L103)
You can reproduce the issue by checking out this specific commit locally and executing the dokkatooGenerateModuleHtml task.

I think the issue is actually contained within the build script, and it isn't an issue with Dokkatoo.
This line uses the Project.file() function.
localDirectory.set(file("src/main/kotlin"))

If you refactor the script so it assigns the directory to a variable, it should work.
tasks {
    dokkatooGenerateModuleHtml.configure {

        // define a val to be compatible with CC
        val srcMainKotlinDir = layout.projectDirectory.dir("src/main/kotlin")

        outputDirectory.set(layout.buildDirectory.dir("mkdocs/sources/api"))

        dokkaSourceSets.configureEach {
            reportUndocumented.set(true)
            skipEmptyPackages.set(true)
            jdkVersion.set(8)

            sourceLink {
                localDirectory.set(srcMainKotlinDir)
                remoteUrl.set(URL("https://github.com/GW2ToolBelt/api-generator/tree/v${version}/src/main/kotlin"))
                remoteLineSuffix.set("#L")
            }
        }
    }
}

@TheMrMilchmann could you check, please?

It is not only that, but also version.
Both file and version are delegated to the build script wich then delegates to the project.
This fixes it, so yes, not a dokkatoo issue:
diff --git a/build.gradle.kts b/build.gradle.kts
index 1acf54f..be5d7f8 100644
--- a/build.gradle.kts
+++ b/build.gradle.kts
@@ -89,13 +89,15 @@ tasks {
     dokkatooGenerateModuleHtml.configure {
         outputDirectory.set(layout.buildDirectory.dir("mkdocs/sources/api"))

+        val version = project.version
+        val srcMainKotlin = layout.projectDirectory.dir("src/main/kotlin")
         dokkaSourceSets.configureEach {
             reportUndocumented.set(true)
             skipEmptyPackages.set(true)
             jdkVersion.set(8)

             sourceLink {
-                localDirectory.set(file("src/main/kotlin"))
+                localDirectory.set(srcMainKotlin)
                 remoteUrl.set(URL("https://github.com/GW2ToolBelt/api-generator/tree/v${version}/src/main/kotlin"))
                 remoteLineSuffix.set("#L")
             }

Thanks once again, @Vampire. This does the trick. I totally forgot about that and was further led in the wrong direction by the report. Unfortunately, this is really easy to miss.

Unfortunately, this is way too easy to miss.

Agreed! I wish that the HTML report provided more useful information. https://github.com/gradle/gradle/issues/24030

Well, at least it said that the problem is an action within dokkaSourceSets that access it via implicit or explicit this.


Add a new method for removeduplicates!

Which takes an array of column names rather than a single string.  Added
a deprecation warning to the old version.  This should resolve https://github.com/JuliaDB/SQLite.jl/issues/94.

I'm not too familiar with the build logs, but it looks like the failing on is due to some issues with Coveralls on 0.5 and not with this module?  Can anyone confirm this or let me know if there's some other issue, I haven't started using the 0.5 previews yet.

Thanks for the work @amellnik.
Yeah the failure looks like a coveralls issue to me, plus the tests run fine on my macbook.
The patch looks good, @quinnj I'm happy for this to merge if you are.

Thinking about it, we should probably add a test case for this as well.

Good point, I added a standalone test that uses the new method.

Gentle ping to @quinnj and @Sean1708 -- does this look ok to merge?

Travis failure is unrelated, so I'm happy for this to merge. I'll wait for @quinnj to give his input first though.

Sorry for the delay! Thanks!


kconfig: add options for maximum PSK-ID and PSK lengths

PSK-ID length was always hardcoded to maximum 64 characters. As a result
long project names with long PSK-IDs (due to
long-device-id@long-project-name PSK-ID format) would not be supported.
PSK had the same issue, but only when used with NCS.
Introduce options for both, PSK-ID and PSK, so that long credentials can be
supported without changing SDK codebase.
Alternative to: #359

FYI: @danyocom

@ChrisGammell ran into the same problem and it took him a while to figure out what the issue was. It looks like it wasn't immediately obvious to @danyocom either. @mniestroj can we also add an error message to the console if the length exceeds the configured value? That way it should be pretty straightforward for users who encounter this issue to determine what caused it and how to fix it.

@mniestroj can we also add an error message to the console if the length exceeds the configured value? That way it should be pretty straightforward for users who encounter this issue to determine what caused it and how to fix it.

I agree with this. I tested a psk-id that was 148 chars long with the limit set to 128. The credential save succeeded without error messages but the device failed to connect.


Find in files doing without case match only applies if search term is with lower case

This seem to applay only if the search term is used with capital letters.
For example if I search 0x8d the results are the following:
0x8D
0x8d
If I search 0x8D the results only:
0x8D

/needsMoreInfo
Do you have search.smartCase enabled in settings?


Fix Crypto API stuck on long streams

Description
The "Receive" was not called as the "Send" got stuck after about ~400kb of data was sent, it seems like a buffer was full, but since "Receive" wasn't called, the buffers weren't draining.
Once the "Send" task is not awaited, "Receive" is called and starts pulling data.
As long as "Send" is somehow being chained together with "Receive" (using Task.WhenAll, or .ContinueWith, etc), the "Receive" task is not starting.
The only way I was able to complete both was to disconnect them.
Ran 1000 iterations of EncryptAsync + DecryptAsync:
Using 6Mb file, total data processed ~12Gb (Encrypt 6Gb + Decrypt 6Gb)
No increase in memory was observed for daprd and tester app.
All went successful, the "Send" task was completed and cleaned.
Nevertheless, knowing that this is not a good practice when working with Tasks, would like to receive an opinion on this change or maybe an idea why this might happen with tasks.
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #[issue number]
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[ ] Code compiles correctly
[ ] Created/updated tests
[ ] Extended the documentation

Thank you for your report!
While the removal of Task.WhenAll would remove the blocking behavior you're experiencing, wouldn't you expect the operation to be blocking so that when the task returns, you can rely on an expectation that the full plaintext value was received and the full ciphertext value returned or vice versa (where in theory the encryption service should continue to operate as long as there's an inbound stream)?
By not awaiting the outbound side of the operation, exceptions wouldn't be propagated and any resources that would have been released because of the exception wouldn't release. Further, the use of Task.WhenAll means that when the provided cancellation token flags for cancellation, both operations can be cancelled. Here, if the receive operation completes (theoretically shouldn't happen) or is canceled first (could happen), the outbound side may not be properly canceled.
I think this solution needs more testing before we just merge it in. I'll get a larger file added for testing purposes and see if I can't reproduce your experience.


Published version of yew-router does not match what is in this repository

Problem
The code for the yew-router crate which is published to crates.io does not seem to match what is in this repository at all. It more closely resembles the contents of the archived yew_router repository, but it's not exact. This makes understanding how to use the code very difficult when documentation is lacking.
Steps To Reproduce
Generated docs do not match:

View the prelude documentation for yew-router on docs.rs (https://docs.rs/yew-router/0.15.0/yew_router/prelude/index.html)
View the prelude in the yew-router package in this repository (https://github.com/yewstack/yew/blob/0f92d4878e8de55a199a1740760829de6d5884d1/packages/yew-router/src/lib.rs#L73-L80)
Notice that they are completely different

Cargo.toml do not match:
Here is the Cargo.toml file that Cargo downloads for yew-router 0.15.0
# THIS FILE IS AUTOMATICALLY GENERATED BY CARGO
#
# When uploading crates to the registry Cargo will automatically
# "normalize" Cargo.toml files for maximal compatibility
# with all versions of Cargo and also rewrite `path` dependencies
# to registry (e.g., crates.io) dependencies
#
# If you believe there's an error in this file please file an
# issue against the rust-lang/cargo repository. If you're
# editing this file be aware that the upstream Cargo.toml
# will likely look very different (and much more reasonable)

[package]
edition = "2018"
name = "yew-router"
version = "0.15.0"
authors = ["Henry Zimmerman<EMAIL_ADDRESS>"Sascha Grunert<EMAIL_ADDRESS>description = "A router implementation for the Yew framework"
readme = "README.md"
keywords = ["web", "yew", "router"]
categories = ["gui", "web-programming"]
license = "MIT/Apache-2.0"
repository = "https://github.com/yewstack/yew"
[dependencies.cfg-if]
version = "1.0.0"

[dependencies.cfg-match]
version = "0.2"

[dependencies.gloo]
version = "0.2.0"
optional = true

[dependencies.js-sys]
version = "0.3.35"
optional = true

[dependencies.log]
version = "0.4.8"

[dependencies.nom]
version = "5.1.1"

[dependencies.serde]
version = "1.0.104"
features = ["derive"]

[dependencies.serde_json]
version = "1.0.48"

[dependencies.stdweb]
version = "0.4.20"
optional = true

[dependencies.wasm-bindgen]
version = "0.2.58"
optional = true

[dependencies.web-sys]
version = "0.3"
features = ["History", "HtmlLinkElement", "Location", "MouseEvent", "PopStateEvent", "Window"]
optional = true

[dependencies.yew]
version = "^0.18.0"
features = ["services", "agent"]
optional = true
default-features = false

[dependencies.yew-router-macro]
version = "^0.15.0"

[dependencies.yew-router-route-parser]
version = "^0.15.0"
[dev-dependencies.uuid]
version = "0.8.1"

[features]
agent = ["service"]
components = ["agent"]
core = ["router", "components"]
default = ["web_sys", "core", "unit_alias"]
router = ["agent"]
service = ["yew"]
std_web = ["yew/std_web", "stdweb"]
unit_alias = []
web_sys = ["yew/web_sys", "gloo", "js-sys", "web-sys", "wasm-bindgen"]
[target."cfg(all(target_arch = \"wasm32\", not(target_os=\"wasi\"), not(cargo_web)))".dependencies.wasm-bindgen]
version = "0.2.58"

It looks nothing like the Cargo.toml file in this repository (https://github.com/yewstack/yew/blob/0f92d4878e8de55a199a1740760829de6d5884d1/packages/yew-router/Cargo.toml#L1).
It has different dependencies, different features, different authors. I have gone back in the git history and cannot find a version of this file which matches what cargo is storing. It does however, look very similar to the version of the Cargo.toml file which is stored in the archived router repository (https://github.com/yewstack/yew_router/blob/6f9e0f580a306fc8fec866eace80374a60313ce6/Cargo.toml#L2).
The Rust code is also completely different. I got a warning message from yew-router (No route matched, provide a redirect prop to the router to handle cases where no route can be matched) and was surprised to not see this message documented anywhere, or the concept of a "redirect prop" documented anywhere. When I came to the repository to look for the code, I couldn't find that warning at all. The warning does show up in the old, archived yew-router codebase (https://github.com/yewstack/yew_router/blob/6f9e0f580a306fc8fec866eace80374a60313ce6/src/router.rs#L230) so I'm now wondering if some updated version of that is the real published version of this crate.
Expected behavior
I expected the code in this repository to match what is published to Crates.io.
Environment:

Yew version: 0.18
Rust version: 1.53.0
Build tool, if relevant: trunk
OS, if relevant: Ubuntu on WSL2

The master branch is the development branch. See the v0.18 branch for the current released version

Thank you, I guess I should have checked the tags more closely. Sorry for the issue noise


Signature conflicts with pre-installed GApps

I still cannot get UnifiedNlp working on my LG P880 (stock Android 4.1.2 with GApps pre-installed of course). I've downloaded the UnifiedNlp.apk from here, and installed it on the device with adb install (see also: #34 ). Can open the app, configured two backends so far (the local GSM and openBmap backend). Still, the Xposed module claims it gets a location, but that it doesn't look like a UnifiedNlp one. Checking logcat revealed:
W/LocationManagerService(13440): org.microg.nlp implements com.android.location.service.GeocodeProvider but its signatures don't match those in com.google.android.location, ignoring

So what's wrong here, and how to fix? Note that uninstalling GApps or using a different ROM is no option for me currently.
Edit: I've tried working around that making UnifiedNlp a system app, but the very same error remains. So whatever it is, it seems to have protectioLevel "signature" (not "signatureOrSystem"), so this work-around doesn't work around.

Just out of curiosity: what happens if you try LegacyNetworkLocation.apk?
see https://github.com/microg/android_packages_apps_UnifiedNlp/blob/HEAD/README.md#android-23---43-gingerbread--honeycomb--ice-cream-sandwich--jelly-bean

@wish7code I don't even need to try that. Package name of that app is com.google.android.location – so I will get a signature mismatch, as a package with that name already is pre-installed (Google Network Location). The instruction you're referring to is for "Google-free devices" (let me know if I'm wrong here, but I cannot install two apps having the same package name).
But pointing to that: what might work is removing the Google Network Location app from the device, and replacing it by LegacyNetworkLocation.apk. I'm not sure about side-effects, so I'd rather avoid playing with that on my "main device" unless it's already proven to be "safe".
I guess a really safe approach would be having "mock locations" as option to switch to (as requested with #27). So if the "native method" cannot be used (as in my case), I'd at least have an alternative – though it might not work with all apps this way (see there). As @mar-v-in pointed out over there:

However I agree that using the mock API might become an interesting additional option.

Looks like this here is one of the places where it would be "interesting" :innocent:

Disclaimer: Let me clarify, that I don't use the GAPPS setup, so I can't examine the actual behaviour further.
I alway thought that XPosed Framework is able to hook up existing system calls, thus redirecting location requests from com.google.android.location to the respective UnifiedNLP counterpart. Using this logic I thought there would actually be no need to replace the GAPPS apks but both could be installed side-by-side under their respective (different) namespaces.
I might be completely wrong with my assumptions, so it would be best if somebody from the developer team could shed some light..

I alway thought that XPosed Framework is able to hook up existing system calls, thus redirecting location requests from com.google.android.location to the respective UnifiedNLP counterpart. Using this logic I thought there would actually be no need to replace the GAPPS apks but both could be installed side-by-side under their respective (different) namespaces.

That were my thoughts as well. But all the corresponding Xposed module lets me do is run a check, where it fails. As far as I knew, shouldn't XposedUnifiedNlp take care for exactly that part of integration I'm failing here? So should I rather open an issue with them in this case? There is already a related issue open. According to the discussion there, I did everything allright. Quoting @Rawi01:

If you want to use UnifiedNlp with this module you have to download and simply install the UnifiedNlp.apk from the UnifiedNlp release page. It only works with this apk. It is not necessary to copy it to some speciall directory or set permissions.

I too am running without gapps so don't have any direct experience, but I see that on F-Droid there is a download for phones with gapps. I suspect that it uses a something other than com.google.android.location and so it can be loaded. I assume that once loaded, xposed can be used to wire up the calls. Has that been tried?
The write up mentions non-stock firmware so perhaps this won't work on a stock setup but it seems like there should be little difference between stock and another ROM with gapps.

@n76 Thanks! That's what I've tried first (if we're both referring to the UnifiedNlp package available from there). Error messages fluctuated with that: either services not working, or no UnifiedNlp source. On suggestion of Rawi (see above) I've switched to the UnifiedNlp.apk from here. Now I've got a consistent error (no UnifiedNlp source). See above.

Step-by-step install instruction for both GAPP and GAPPS-free devices is definitely on the TODO list..
Could you upload a complete logcat in the meantime, so we can trace down, which of the components isn't working? You might also try to activate debug logging in the openbmap UnifiedNLP settings, so logcats become more verbose.

@wish7code I of course can provide a full logcat, but I'd prefer not to place that "in public space". If you could name a more closed place to deposit it (mail/upload? Find my contact info via my profile page here), I'm happily complying.
The core issue seems to be this one:
V/LocationManagerService(13440): com.google.android.location implements com.android.location.service.GeocodeProvider with version 1
V/LocationManagerService(13440): com.google.android.gms implements com.android.location.service.GeocodeProvider with version 0
org.microg.nlp implements com.android.location.service.GeocodeProvider but its signatures don't match those in com.google.android.location, ignoring

com.google.android.location implements com.android.location.service.NetworkLocationProvider with version 1

As you can see: µg tries to offer a GeocodeProvider (though I've no backend set up for that), but is ignored due to signature mismatch. Strange thing is the log doesn't show any hint on its NetworkLocationProvider: 2 backends configured here, which are bound:
D/NlpLocationBackendHelper(23816): Binding to: Intent { act=org.microg.nlp.LOCATION_BACKEND pkg=org.fitchfamily.android.gsmlocation cmp=org.fitchfamily.android.gsmlocation/.GSMService }
D/NlpLocationBackendHelper(23816): Binding to: Intent { act=org.microg.nlp.LOCATION_BACKEND pkg=org.openbmap.unifiedNlp cmp=org.openbmap.unifiedNlp/.services.OpenbmapNlpService }
I/ActivityManager(21057): Start proc org.fitchfamily.android.gsmlocation for service org.fitchfamily.android.gsmlocation/.GSMService: pid=23838 uid=10130 gids={3003, 1015, 1028}
I/ActivityManager(21057): Start proc org.openbmap.unifiedNlp for service org.openbmap.unifiedNlp/.services.OpenbmapNlpService: pid=23848 uid=10145 gids={3003, 1015, 1028}

Taking a closer look, LocationService was started for µg – as shortly after the last msg from above comes a stack trace starting with
Service org.microg.nlp.location.LocationServiceV1 has leaked ServiceConnection org.microg.nlp.location.BackendHelper@42ef8100 that was originally bound here
android.app.ServiceConnectionLeaked: Service org.microg.nlp.location.LocationServiceV1 has leaked ServiceConnection org.microg.nlp.location.BackendHelper@42ef8100 that was originally bound here
    at android.app.LoadedApk$ServiceDispatcher.<init>(LoadedApk.java:971)
    at android.app.LoadedApk.getServiceDispatcher(LoadedApk.java:865)
    at android.app.ContextImpl.bindService(ContextImpl.java:1191)
    at android.app.ContextImpl.bindService(ContextImpl.java:1183)
    at android.content.ContextWrapper.bindService(ContextWrapper.java:394)
    at org.microg.nlp.AbstractBackendHelper.bind(AbstractBackendHelper.java:78)
    at org.microg.nlp.location.BackendFuser.bind(BackendFuser.java:76)
    at org.microg.nlp.location.ThreadHelper.enable(ThreadHelper.java:70)
    at org.microg.nlp.location.ThreadHelper.reload(ThreadHelper.java:43)
    at org.microg.nlp.location.LocationProviderV1.reload(LocationProviderV1.java:54)
    at org.microg.nlp.location.AbstractLocationService.onHandleIntent(AbstractLocationService.java:68)
    at android.app.IntentService$ServiceHandler.handleMessage(IntentService.java:65)
    at android.os.Handler.dispatchMessage(Handler.java:99)
    at android.os.Looper.loop(Looper.java:137)
    at android.os.HandlerThread.run(HandlerThread.java:60)
Service org.microg.nlp.location.LocationServiceV1 has leaked ServiceConnection org.microg.nlp.location.BackendHelper@42ef8290 that was originally bound here
android.app.ServiceConnectionLeaked: Service org.microg.nlp.location.LocationServiceV1 has leaked ServiceConnection org.microg.nlp.location.BackendHelper@42ef8290 that was originally bound here
    at android.app.LoadedApk$ServiceDispatcher.<init>(LoadedApk.java:971)
    …

(second stack is basically the same as the first)
For at least a minute after that (until my logcat stops), no more mention of org.microg. I just see your OpenBMap being initialized (trying to load its data file).

Any news about this? A smart installer as in BusyBox would make this a lot easier!

@timofonic are you referring to this specific issue, probably only affecting an unsupported setup? On devices without GAPPS, installation is already straight-forward and on devices with GAPPS and Android 4.3+ its possible using a third-party project.

Unfortunately I think @timofonic is reffering to the GAPPS case..
Have a look at https://github.com/Rawi01/XposedUnifiedNlp/issues/2 too (Warning bit lenghty): nobody ever confirmed a running GAPPS installation over there..
The problem seems unrelated to the actual location provider, but somehow framework specific..
Without GAPPS everythings works fine..


Supporte TIF TIFF image tile

Download of TIF TIFF image tile fails
Uploaded 2018-Boucherville-13313005-13427550-1_TIF_tile.TIF successfully
but in the annotation interface javascript error
http://localhost:7780/testJPG/files/1imageJPG/2018-Boucherville-13313005-13427550-1_tile.JPG 404 (Not Found)

in progress

convert tiff to jpg+wld file and then use it to create geolocalized geojson

test in progress; refer to #63; closing

test completed ,with unittest and AIDE+ interface


Feature/fixing issue1

fixing the usage of UT core functions

please update the description on this ticket to reflect what the problem is in the title of the review, and ensure that #1 is actually described correctly.


Improve clarity of English language quick-start overview documentation

Continued translation work following on from PR #4043

Translate docs/en/quick-start/overview.md
Fix several links which have the format [title] (url) instead of [title](url), which must be an artefact of how the docs were originally translated.

I found and fixed other incorrect links using the following regular expression: \[([^\]]+)\] +\(

/assgin @Reasno


[ListItem] Add option to Toggle Nested List Items on Primary Action

Continuation of #1648. Closes #1597.
Please review and merge @oliviertassinari.

@subjectix That's a great idea to continue existing PR :+1:. I don't have time to review this now.

Thanks. Sure, take your time ^_^

done


eigh() on float32 matrix leads to cudaErrorIllegalAddress

Description
Calling eigh() on the matrix
a = cupy.asarray([[ 2.0000100e+00,  0.0000000e+00,  0.0000000e+00,
             1.0000000e+07,  0.0000000e+00],
           [ 0.0000000e+00,  0.0000000e+00,  0.0000000e+00,
            -2.0000100e+00,  3.4028235e+38],
           [ 0.0000000e+00,  0.0000000e+00, -5.0000000e-01,
            -2.2098796e+16, -1.9000000e+00],
           [ 1.0000000e+07, -2.0000100e+00, -2.2098796e+16,
            -1.1754944e-38, -4.2423595e+16],
           [ 0.0000000e+00,  3.4028235e+38, -1.9000000e+00,
            -4.2423595e+16, -2.6436685e+16]], dtype=cupy.float32)

leads to
cupy_backends.cuda.api.driver.CUDADriverError: CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered

To Reproduce
>>> import cupy
>>> a = cupy.asarray([[ 2.0000100e+00,  0.0000000e+00,  0.0000000e+00,
...              1.0000000e+07,  0.0000000e+00],
...            [ 0.0000000e+00,  0.0000000e+00,  0.0000000e+00,
...             -2.0000100e+00,  3.4028235e+38],
...            [ 0.0000000e+00,  0.0000000e+00, -5.0000000e-01,
...             -2.2098796e+16, -1.9000000e+00],
...            [ 1.0000000e+07, -2.0000100e+00, -2.2098796e+16,
...             -1.1754944e-38, -4.2423595e+16],
...            [ 0.0000000e+00,  3.4028235e+38, -1.9000000e+00,
...             -4.2423595e+16, -2.6436685e+16]], dtype=cupy.float32)
>>> cupy.linalg.eigh(a)
Traceback (most recent call last):
  File "cupy/_core/core.pyx", line 1704, in cupy._core.core._ndarray_base.__repr__
    return repr(self.get())
  File "cupy/_core/core.pyx", line 1818, in cupy._core.core._ndarray_base.get
    a_gpu.data.copy_to_host(ptr, a_gpu.nbytes)
  File "cupy/cuda/memory.pyx", line 557, in cupy.cuda.memory.MemoryPointer.copy_to_host
    and runtime.streamIsCapturing(stream_ptr)
  File "cupy_backends/cuda/api/runtime.pyx", line 965, in cupy_backends.cuda.api.runtime.streamIsCapturing
    check_status(status)  # cudaErrorStreamCaptureImplicit could be raised here
  File "cupy_backends/cuda/api/runtime.pyx", line 143, in cupy_backends.cuda.api.runtime.check_status
    raise CUDARuntimeError(status)
cupy_backends.cuda.api.runtime.CUDARuntimeError: cudaErrorIllegalAddress: an illegal memory access was encountered

after this, all further operations give the same error. The same matrix with float64 does not give the error.
Installation
Built from GitHub source
Environment
>>> cupy.show_config()
OS                           : Linux-5.13.0-51-generic-x86_64-with-glibc2.34
Python Version               : 3.9.7
CuPy Version                 : 11.0.0b3
CuPy Platform                : NVIDIA CUDA
NumPy Version                : 1.20.3
SciPy Version                : 1.7.1
Cython Build Version         : 0.29.24
Cython Runtime Version       : 0.29.24
CUDA Root                    : /home/aaronmeurer/anaconda3
nvcc PATH                    : /home/aaronmeurer/anaconda3/bin/nvcc
CUDA Build Version           : 11070
CUDA Driver Version          : 11070
CUDA Runtime Version         : 11070
cuBLAS Version               : (available)
cuFFT Version                : 10702
cuRAND Version               : 10210
cuSOLVER Version             : (11, 3, 5)
cuSPARSE Version             : (available)
NVRTC Version                : (11, 7)
Thrust Version               : 101500
CUB Build Version            : 101500
Jitify Build Version         : 4a37de0
cuDNN Build Version          : None
cuDNN Version                : None
NCCL Build Version           : None
NCCL Runtime Version         : None
cuTENSOR Version             : None
cuSPARSELt Build Version     : None
Device 0 Name                : NVIDIA GeForce RTX 2080
Device 0 Compute Capability  : 75
Device 0 PCI Bus ID          : 0000:01:00.0

Additional Information
I tested this with the CuPy git source (commit 7df760a0186ddf609e30d1a176349e56ad114050).

Thanks for reporting, I could reproduce this. CuPy just calls cuSOLVER routine for eigh, so I suspect this is not a CuPy issue. The first error observed by cuda-memcheck suggests there is an out of bounds read in cuSOLVER:
========= Invalid __shared__ read of size 4
=========     at 0x00003920 in void steqr_ker<stedc_params_<float, int=8, int=16>, float, int=0>(int, floatdata_type*, floatdata_type*, int=8*, long, int*, int*, int, int, floatdata_typ
e, floatdata_type, int*)
=========     by thread (0,0,0) in block (0,0,0)
=========     Address 0xfffffffc is out of bounds
=========     Saved host backtrace up to driver entry point at kernel launch time
=========     Host Frame:/usr/lib/x86_64-linux-gnu/libcuda.so.1 [0x2182ec]
=========     Host Frame:/usr/local/cuda-11.7.0/lib64/libcusolver.so.11 [0x74cbcb]
=========     Host Frame:/usr/local/cuda-11.7.0/lib64/libcusolver.so.11 [0x7a8368]
=========     Host Frame:/usr/local/cuda-11.7.0/lib64/libcusolver.so.11 [0x3fe479]
=========     Host Frame:/usr/local/cuda-11.7.0/lib64/libcusolver.so.11 [0x3e0f35]
=========     Host Frame:/usr/local/cuda-11.7.0/lib64/libcusolver.so.11 (cusolverDnXsyevd + 0x35c) [0x4966fc]
=========     Host Frame:/home/maehashi/Development/cupy/cupy_backends/cuda/libs/cusolver.cpython-38-x86_64-linux-gnu.so [0x25d0d]
...

cc/ @anaruse @leofang

Any chance we know the earliest offending CUDA version here? Are there any special properties about the offending matrix?


Annotate code

Add comments to

double.py, balance strategy (Dynamic Resampling)
analysis.py, (average) Time to Discovery

Thanks for the adjustments @qubixes!

Thanks folks!


[Fix #3428] Allow Style/PreferredHashMethods to be configured

Adds support for configuring preference for key?/value? vs has_key?/has_value? fixing #3428. The default remains wiht the short method name style but allows the more descriptive names using the verbose style.

Before submitting the PR make sure the following are checked:

[x] Wrote good commit messages.
[x] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Used the same coding conventions as the rest of the project.
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[x] Added tests.
[x] Added an entry to the Changelog if the new code introduces user-observable changes. See changelog entry format.
[x] All tests are passing.
[x] The new code doesn't generate RuboCop offenses.
[x] The PR relates to only one subject with a clear title
and description in grammatically correct, complete sentences.

One more thing - rebase on top of the current master.

No probs

Done

👍


[Get Registry] Not working as intended

Hi,
According to this documentation: https://clearblade.atlassian.net/wiki/spaces/IC/pages/2207973607/get
There is no need for a parent query param just the name of the registry is enough.
using as name either:
registry-name
or
projects/$1/locations/$2/registries/$3
results in the following error:
Missing 'parent' query param
To reproduce:
curl  -H "ClearBlade-UserToken: ${TOKEN}" https://$X.clearblade.com/api/v/4/webhook/execute/$0/cloudiot?name=projects/$1/locations/$2/registries/$3

Can you please advise?
Cheers.

Hi Charbel,
Would you mind open a service ticket for this issue.  Thanks
https://clearblade.atlassian.net/servicedesk/customer/portal/1/group/1/create/20

Hi @aallsbrook,
Opened a ticket,
Thank you


Speed up the installation of Proofer

The Nokogiri installation is faster this way.

Installation time:

before: 94 s
after: 44 s

Nice. Great suggestion. Thanks for the pull request.


setSelectedNavigationItem do not work in Expandable List mode,

Another old issue. I use ViewPager with tabs (code based on Support4Demo). When I flip page, I use this code in PagerAdapter to update tab selection:
public void onPageSelected(int position) {
 mActionBar.setSelectedNavigationItem(position);
}

This works great until I add few actionbar items and tabs collapsed to expandable list (it happens in landscape mode). Then selection of current tab just didnt work. Is I should use some other method instead of setSelectedNavigationItem()?
This issue appears in beta5 and beta6

This is a platform bug. [This code] doesn't not check if mTabSpinner is being used. It always assumes mTabLayout.
Submitted a patch to AOSP: https://android-review.googlesource.com/32492

I see. Thanks Jake.
Will try to reduce my tabs amount and width :)


Cow: Watch advancewithai.net

Cow requests the watch of the watch_keyword advancewithai\.net. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

advancewithai\.net has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by cigien in Charcoal HQ


test(stub): simplify test cases

Codecov Report
Base: 90.63% // Head: 90.27% // Decreases project coverage by -0.36% :warning:

Coverage data is based on head (bff88ef) compared to base (45fa4d3).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #69      +/-   ##
==========================================
- Coverage   90.63%   90.27%   -0.37%
==========================================
  Files          33       33
  Lines        2477     2477
==========================================
- Hits         2245     2236       -9
- Misses        158      166       +8
- Partials       74       75       +1

Impacted Files
Coverage Δ

internal/engine/mongodb/queue.go
89.84% <0.00%> (-7.04%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


GH-39131: [JS] Add at() for array like types

Arrays in JavaScript have at instead of get so here we are making our arrays more consistent.
This PR still maintains support for get but deprecates it. We can remove get after one version of supporting both.

GitHub Issue: #39131

@trxcllnt and I agreed offline to wrap indexes accesses so that any index is valid. The behavior in js is to only index -length ... length and return undefined otherwise but that seems less consistent than to just always wrap. Also, the types can be composer since we never return undefined.

I see. So you are saying we should keep the get visitor and just make at a method that calls get?
Isn't at as I implemented equivalent to get within the valid range of values? Do we want to keep get because it's slightly faster than at?

Redone in https://github.com/apache/arrow/pull/40730


IsADirectoryError: [Errno 21] Is a directory: '/root/code/StyleID-main/data/sty/.ipynb_checkpoints'

It a promising job, but I have some errors.
IsADirectoryError: [Errno 21] Is a directory: '/root/code/StyleID-main/data/sty/.ipynb_checkpoints'

I think the error is likely due to the your conversion of the code to Python notebook format.
Please check the notebook configuration settings.


Fix race in Pool#pop.

There was a race in the Pool#pop method, between testing the @pool.empty? and actually popping the value with @pool.pop. For example, if there were two contenders and only one store in the pool, one of the contenders could end with a nil store. The fix is to do the pop atomically, so there is no race possible.

Thanks!


GitLab?

README says:

The project moved to GitLab in 2013

Is "GitLab" a typo for "GitHub"?

It would appear so. According to Wayback the project was previously on Launchpad.
Orginal launchpad: https://launchpad.net/scour
Wayback page showing use of launchpad: https://web.archive.org/web/20150304174144/http://www.codedread.com/scour
Launchpad anncounment referencing the move to GitHub: https://launchpad.net/scour/+announcement/13743


NTR: magnetosome assembly

Please provide as much information as you can:

Suggested term label:
magnetosome assembly

Definition (free text)
The aggregation, arrangement and bonding together of a set of components to form a magnetosome. Magnetosomes are  specialized organelles for magnetic navigation that comprise membrane-enveloped, nano-sized crystals of a magnetic iron mineral; they are formed by a diverse group of magnetotactic bacteria (MTB)

Reference, in format PMID:#######

PMID:19575557

PMID:34723168

Gene product name and ID to be annotated to this term
UniProtKB:P0DSO6	MamK-like protein
UniProtKB:Q2W8L8	MamJ paralog LimJ
UniProtKB:Q2W8M7	Methyl-accepting chemotaxis protein Amb0994
UniProtKB:Q2W8Q0	Magnetosome protein MamA
UniProtKB:Q2W8Q6	Actin-like protein MamK
UniProtKB:Q2W8Q7	Magnetosome-associated protein MamJ

Parent term(s)

GO:0070925 organelle assembly

Children terms (if applicable) Should any existing terms that should be moved underneath this new proposed term?

Synonyms (please specify, EXACT, BROAD, NARROW or RELATED)
magnetosome formation

Cross-references

For enzymes, please provide RHEA and/or EC numbers.

Can also provide MetaCyc, KEGG, Wikipedia, and other links.

Any other information

@AndreaAuchincloss
If you spot anything that can be improved, please let me know.
Thanks, Pascale

+[Term]
+id: GO:0140923
+name: magnetosome assembly
+namespace: cellular_component
+def: "The aggregation, arrangement and bonding together of a set of components to form a magnetosome. Magnetosomes are specialized organelles for magnetic navigation that comprise membrane-enveloped, nano-sized crystals of a magnetic iron mineral; they are formed by a diverse group of magnetotactic bacteria (MTB)." [PMID:19575557, PMID:34723168]
+synonym: "magnetosome formation" EXACT []
+is_a: GO:0070925 ! organelle assembly
+property_value: term_tracker_item https://github.com/geneontology/go-ontology/issues/24211 xsd:anyURI
+created_by: pg
+creation_date: 2022-10-18T12:07:11Z
+

oops, it looks like this conforms to the assembly DP yet no logical def...

Thanks, I had also noticed ;) this term now has a LD


Why not use a thread pool for IO？Single thread has a performance bottleneck.

We use a GridView to show a lot of pictures. When we scroll fast, the speed of decoding image won't catch up with the speed of scrolling causes worse experience than native.
Compare to Fresco, it uses a thread pool to decode image.
So i suspect that a single thread for IO is enough?

What is Fresco?
Are you sure the decoding is the bottleneck?
Can you provide a minimal reproduction?
Please add the output of flutter doctor -v.

https://github.com/facebook/fresco
Fresco is a system for displaying images in Android applications.
here is the trace.
The thing is even if the image is slide across the screen only a moment then disappeared, PaintingBinding.instance.instantiateImageCodec(bytes) will be called causes that images is currently showing must wait for the decoding of disappeared images.
image_decode.txt

[✓] Flutter (Channel dev, v1.0.0, on Mac OS X 10.14.1 18B73a, locale zh-Hans-CN)
• Flutter version 1.0.0 at /Users/SupSaiYaJin/Library/Android/flutter
• Framework revision 5391447fae (7 days ago), 2018-11-29 19:41:26 -0800
• Engine revision 7375a0f414
• Dart version 2.1.0 (build 2.1.0-dev.9.4 f9ebf21297)
[✓] Android toolchain - develop for Android devices (Android SDK 28.0.2)
• Android SDK at /Users/SupSaiYaJin/Library/Android/sdk
• Android NDK at /Users/SupSaiYaJin/Library/Android/sdk/ndk-bundle
• Platform android-28, build-tools 28.0.2
• Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1136-b06)
• All Android licenses accepted.
[!] iOS toolchain - develop for iOS devices (Xcode 10.1)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Xcode 10.1, Build version 10B61
✗ libimobiledevice and ideviceinstaller are not installed. To install with Brew, run:
brew update
brew install --HEAD usbmuxd
brew link usbmuxd
brew install --HEAD libimobiledevice
brew install ideviceinstaller
✗ ios-deploy not installed. To install with Brew:
brew install ios-deploy
✗ CocoaPods not installed.
CocoaPods is used to retrieve the iOS platform side's plugin code that responds to your plugin usage on the Dart side.
Without resolving iOS dependencies with CocoaPods, plugins will not work on iOS.
For more info, see https://flutter.io/platform-plugins
To install:
brew install cocoapods
pod setup
[✓] Android Studio (version 3.2)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin version 31.1.1
• Dart plugin version 181.5656
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1136-b06)
[✓] IntelliJ IDEA Ultimate Edition (version 2018.2.5)
• IntelliJ at /Applications/IntelliJ IDEA.app
• Flutter plugin version 30.0.2
• Dart plugin version 182.4892.25
[✓] VS Code (version 1.28.2)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 2.19.0
[✓] Connected device (1 available)
• Redmi Note 4X • 16fedaac0004 • android-arm64 • Android 7.0 (API 24)
! Doctor found issues in 1 category.

/cc @chinmaygarde

This has been on my radar for a while and there is no reason we should not be attempting this already. I'll wire up the parallel task runners for known platforms.

This has been on my radar for a while and there is no reason we should not be attempting this already. I'll wire up the parallel task runners for known platforms.

Thanks for focus on this issue.
We have tried to implement a IO task runnner pool for decoding image multi-threaded. For each thread, we created a GrContext and a SkiaUnrefQueue for it. It works well on decoding, but there is a problem on SkImage's deallocation.
The thing is SkImage must be deallocated in the thread which created it, but I can't make sure the SkImage is unrefed by the new SkiaUnrefQueue we created. The reason is SkImage's reference is holded by Codec and SkPicture at the same time, it's uncertain of which one is the last one to unref it, so it might be unref in wrong SkiaUnrefQueue.
be eager for your help.

It may be easier to perform image decompression on a worker but still perform texture upload on the IO thread. I am prototyping something right now. Do you have a reduced test case for this?

Update: I pushed some WIP changes to prepare a concurrent message loop (Darwin is done, on Linux now). My approach is then to use the SkImageGenerator to first create the pixmap from encoded data on one thread and then perform the MakeCrossContextFromPixmap on the IO thread.

It may be easier to perform image decompression on a worker but still perform texture upload on the IO thread. I am prototyping something right now. Do you have a reduced test case for this?

I have tried decoding 300 images asynchronously in 4 threads. It's more than 3 times faster than single IO thread for Android. For iOS, it's more than 2 times faster.

I believe this would also fix https://github.com/flutter/flutter/issues/32143

@chinmaygarde Hi Chinmay, did you get any closer to a multi-threaded image decoder in Flutter?

Hi All, we have recently launched a flutter app and this issue is currently killing us on older iPhones and we are starting to attract negative reviews. Is there an ETA for this to be fixed?

@chinmaygarde https://github.com/flutter/engine/pull/9486 is your solution for this right? @SupSaiYaJin can you check if that PR covers your concern or test it in your use case by applying the patch locally?

@xster I got a compile error for this patch.

Fixed in https://github.com/flutter/engine/commit/ad582b5089256e56727b2d7306fb097078a3dcdc


style:Signup UI enhancement

@AnupK1234
Here is a Screenshot of my work -

the labels have a short transition to them. its looks great

@AnupK1234  check the PR

@AnupK1234  Done with the form validation!

Sure..

Done ✅

Kindly use the PR template.

@IrfanshaikH-7 Kindly use the PR template.

I'm not sure what that is can you explain


Date format in messages

Format our timestamp just like the gitter does. No date is displayed if the message was received today, and for all past days it displays month, date and time.
AC:

Timestamp should be placed on the same line as the username.
No date is displayed if the message was received today, and for all past days it displays month, date and time.

Let me take this issue!

@kpbot claim

You'll want to wait for #148 to be merged so you have date-fns available to you.

I see.... The pull request is marked size: XL. really a big one 😬 . It migrates to use template string.
date-fns 's function looks great! And I hope my thinking on the timing to update timestamp is still valid.

I merged #148. You can start working @pancreaflame.

Okay. I work on it.

This has been fixed in #152.