Dockerize mara

PR for discussing and brainstorming on the docker-related mara implementations

I had a few additions and problems :-( :

pip had problems installing (or better uninstalling old versions): --ignore-installed added
when using the .venv for both local and in-docker dev, the venv had wrong paths for the respective other way): moved the in-docker venv to .venv-docker and duplicated the code to create/update it into the entrypint.sh script. BEtter fix would be to make the env part configureable in the makefile
The ensure-pushed script would reproduceable fail the first time it would run in an environment (i.e. in-docker vs local directly). So I made the code run the script twice, once as a dry-run.
I needed a way to create the etl db, so I added a small command which basically copies the first part from the migration code in mara-db (flask app.cli.ensure-etl-db) -> this might be something for the data-integration package.
I wanted postgresql client version 11

I ended up with this for the dev docker container (e.g. linking in the source into the container via a volume):
# don't care about slim, we anyway have lots of reasons to poke into the container...

COPY ./docker/dev/entrypoint.sh /
RUN ["chmod", "+x", "/entrypoint.sh"]

EXPOSE 5000

ENV MARA_ENVIRONMENT docker-dev
ENV FLASK_APP "/mara/app/app.py"
ENV FLASK_DEBUG 1
# preactivate the environment, so you can straight do stuff like run pipelines with docker exec
ENV PATH /mara/.venv-docker/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

USER mara

ENTRYPOINT ["/entrypoint.sh"]
CMD ["flask-no-reload"]

entrypoint.sh:
#!/usr/bin/env bash
set -x
set -e

# Create the virtual env for the first time, which is only used for docker
# This prevents problems when a user uses the same source repo for both working
# via docker and local
venv_dir=".venv-docker"
if [ ! -d "${venv_dir}" ]; then
    mkdir -p .venv-docker
    (cd "${venv_dir}" && python3 -m venv --copies .)
    # add the project directory to path, so we might use that to edit source packages
    echo $(pwd) > "$(echo ${venv_dir}/lib/*/site-packages)/mara-path.pth"
    # install minimum set of required packages
    # wheel needs to be early to be able to build wheels
    # --ignore-installed: https://github.com/moby/moby/issues/12327
    # "EnvironmentError: [Errno 39] Directory not empty: '/mara/.venv-docker/lib/python3.7/site-packages/~ip/_internal'"
    "${venv_dir}/bin/python3" -m pip install --ignore-installed  --upgrade pip wheel requests setuptools pipdeptree
    # Workaround problems with un-vendored urllib3/requests in pip on ubuntu/debian
    # This forces .venv/bin/pip to use the vendored versions of urllib3 from the installed requests version
    # see https://stackoverflow.com/a/46970344/1380673
    rm -vf "${venv_dir}/share/python-wheels/{requests,chardet,urllib3}-*.whl"
fi

source "${venv_dir}/bin/activate"

if [ "$1" = "flask-reload" ]; then
    exec flask run --with-threads --host <IP_ADDRESS> --reload --eager-loading
elif [ "$1" = "flask-no-reload" ]; then
    exec flask run --with-threads --host <IP_ADDRESS> --no-reload --eager-loading
elif [ "$1" = "migrate" ]; then
    flask app.cli.ensure-etl-db
    flask mara_db.migrate
elif [ "$1" = "update-packages" ] ; then
    for package_dir in $(mkdir -p packages; cd packages; find . -maxdepth 1 -mindepth 1 -type d) ; do
        # I've no clue, but this has to run twice to work: the first time it will fail, but the second time it succeeds
        echo $(.scripts/mara-app/ensure-pushed.sh packages/${package_dir} > /dev/null 2>&1) > /dev/null 2>&1
        .scripts/mara-app/ensure-pushed.sh packages/${package_dir}
    done
    "${venv_dir}/bin/python3" -m pip install --ignore-installed --requirement=requirements.txt.freeze --src=./packages --upgrade --exists-action=w
    PYTHONWARNINGS="ignore" "${venv_dir}/bin/python3" -m pip install --requirement=requirements.txt --src=./packages --upgrade --exists-action=w
    # copy newer script versions
    rsync --archive --recursive --itemize-changes  --delete packages/mara-app/.scripts/ .scripts/mara-app/
    "${venv_dir}/bin/pipdeptree" --warn=fail
    # write freeze file
    # pkg-ressources is automatically added on ubuntu, but breaks the install.
    # https://stackoverflow.com/a/40167445/1380673
    "${venv_dir}/bin/python3" -m pip freeze | grep -v "pkg-resources" > requirements.txt.freeze
    flask app.cli.ensure-etl-db
    flask mara_db.migrate
elif [ "$1" = "install-packages" ] ; then
    for package_dir in $(mkdir -p packages; cd packages; find . -maxdepth 1 -mindepth 1 -type d) ; do
        # I've no clue, but this has to run twice to work: the first time it will fail, but the second time it succeeds
        echo $(.scripts/mara-app/ensure-pushed.sh packages/${package_dir} > /dev/null 2>&1) > /dev/null 2>&1
        .scripts/mara-app/ensure-pushed.sh packages/${package_dir}
    done
    "${venv_dir}/bin/python3" -m pip install --ignore-installed  --requirement=requirements.txt.freeze --src=./packages --upgrade --exists-action=w
    rsync --archive --recursive --itemize-changes  --delete packages/mara-app/.scripts/ .scripts/mara-app/
    flask app.cli.ensure-etl-db
    flask mara_db.migrate
else
    exec "$@"
fi

I've also build soemthing for our production environment (e.g. code copied into the docker image):
Dockerfile
# use a slim image to get a smaller size
FROM python:3.7-slim-stretch

RUN ["mkdir", "-p", "/mara"]
WORKDIR /mara

RUN groupadd -r mara && useradd --no-log-init -r -g mara mara

COPY requirements.txt.freeze /mara/

RUN \
    apt-get update && apt-get install -y --no-install-recommends gnupg dirmngr \
    # https://github.com/inversepath/usbarmory-debian-base_image/issues/9#issuecomment-466594168
    && mkdir ~/.gnupg && echo "disable-ipv6" >> ~/.gnupg/dirmngr.conf \
	&& apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8 \
	&& echo "deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main" > /etc/apt/sources.list.d/pgdg.list \
    # ugly fix to install postgresql-client without errors in slim-stretch image
    && mkdir -p /usr/share/man/man1 /usr/share/man/man7 \
    && apt-get update && apt-get install -y --no-install-recommends \
	  git \
      curl \
      dialog \
      coreutils \
      graphviz \
      python3-dev \
      python3-venv \
      rsync \
      postgresql-client \
      gcc \
	&& pip install --no-cache-dir  -r requirements.txt.freeze \
	&& apt-get purge -y --auto-remove git gnupg dirmngr gcc \
	&& rm -rf /var/lib/apt/lists/* ;

COPY ./docker/prod/entrypoint.sh /
RUN ["chmod", "+x", "/entrypoint.sh"]

COPY ./docker/prod/local_setup.py /mara/app/local_setup.py
COPY ./app/ /mara/app/

EXPOSE 5000

ENV MARA_ENVIRONMENT docker-prod
ENV FLASK_APP="/mara/app/app.py"

USER mara

ENTRYPOINT ["/entrypoint.sh"]
CMD ["flask-no-reload"]

entrypoint.sh
#!/usr/bin/env bash
set -x
set -e

if [ "$1" = "flask-reload" ]; then
    exec flask run --with-threads --host <IP_ADDRESS> --reload --eager-loading
elif [ "$1" = "flask-no-reload" ]; then
    exec flask run --with-threads --host <IP_ADDRESS> --no-reload --eager-loading
else
    exec "$@"
fi

local_setup.py
# ...

# On local/in-docker, this is filled with some values
defaults = {}

_sentinal = object()

def e(name, default=_sentinal):
    ret = os.environ.get(name.upper(), defaults.get(name))
    if ret is None:
        if default is not _sentinal:
            return default
        raise KeyError(f"{name} not found in default or environment")
    return ret

@patch(mara_db.config.databases)
def databases():
    return {
        # the project requires two databases: 'mara' for the app itself, and 'dwh' for the etl
        'dwh': mara_db.dbs.PostgreSQLDB(user=e("dwh_user"),
                                        host=e("dwh_host"),
                                        port=int(e("dwh_port")),
                                        database=e("dwh_database"),
                                        password=e("dwh_password")),
        'mara': mara_db.dbs.PostgreSQLDB(user=e("mara_user"),
                                         host=e("mara_host"),
                                         port=int(e("mara_port")),
                                         database=e("mara_database"),
                                         password=e("mara_password")),
# ...
patch(data_integration.config.max_number_of_parallel_tasks)(lambda: int(e("max_number_of_parallel_tasks", 11)))
# ...

I've put the Dockerfile/entrypoint.sh into `./docker/{dev|prod}/ and it's now called like this:
### Docker

For local dev, it's assumed that you want to use the locally installed
postgresql instead of one in a docker
container (speed on real disk is better than in a virtualized server.
Wouldn't matter on Linux...).

To be able to use a locally installed postgresql make postgresql listen
to the interface the docker
container have access to:

` ``bash
λ docker-machine ip
<IP_ADDRESS>
λ ifconfig |grep 192.168.99
	inet <IP_ADDRESS> netmask 0xffffff00 broadcast <IP_ADDRESS>
# on windows use ipconfig and a manual search
# -> This means postgresql has to listen at the <IP_ADDRESS> address
` ``

This needs two changes to the postgresql config files:
* in `postgresql.conf`, you need to add `listen_addresses = 'localhost,<IP_ADDRESS>'`and
* in `pg_hba.conf`, you need to add `host    all             all             <IP_ADDRESS>/32        trust`

Afterwards system restart postgresql (this depends on the docker host (in virtualbox)
already being up and running. on mac, the postgresql server starts too early for the
vbox to be online, so I always have to restart postgresql).

To build the container:

` ``bash
# development version, where the source code is on a VOLUME and you run pipelines in the browser
docker build -t mara-app:dev -f ./docker/dev/Dockerfile .
# Production version, where the source code is copied into the container
docker build -t mara-app:prod -f ./docker/prod/Dockerfile .
` ``

Running a development container:

` ``bash
cp docker/.env.example development.env
# edit development.env -> see comment in the file
# If you use a local postgresql, you need to set the DWH_HOST and MARA_HOST to the ipaddress
# from above (e.g. <IP_ADDRESS>)
docker run -i -t --rm --name mara-app --mount type=bind,source=$(pwd),target=/mara -p 5000:5000 --net=bridge --env-file=development.env mara-app:dev
# docker exec also has the python venv in path:
docker exec -ti mara-app flask data_integration.ui.run --path utils
` ``

Running the production container:

` ``bash
cp docker/.env.example production.env
# edit production.env -> see comment in the file
docker run -i -t --rm --name mara-app -p 5000:5000 --net=bridge --env-file=production.env mara-app:prod
# run the utils pipeline in the same container
docker exec -ti mara-app flask data_integration.ui.run --path utils
` ``

Sizes:
λ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
mara-app            dev                 a06cceefb076        4 hours ago         1.03GB
mara-app            prod                c2e0d861fd4c        15 hours ago        424MB

Still Meh :-(

@martin-loetzsch @jankatins who can resolve the conflict?

@gathineou is anyone still working on this?

I copied the Dockerfiles of this branch and build a docker environment myself which is quite similar to this one. In case there is interest, I could share this in a separate PoC repository.

@leo-schick would be more than happy to try!

@soobrosa I published now my simplified current version under https://github.com/mara/docker. The repository is currently private but you should be able to see it as member of the mara organisation.
My docker images are based on this repo with some additional changes from my side. The suggestions from @jankatins are not considered (yet). PRs are welcome 🤟
p.s. did not do any changes to the postgres-cstore_ftd dockerfile since I do not use cstore_ftd.

@gathineou is anyone still working on this?

Hi @jankatins 👋🏽 excuse my late reply, was off for a while and overlooked this.
Unfortunately no one worked more on this for a while so there is no much progress made from our side.


nsh: Decouple with CONFIG_FILE_STREAM

Summary
Replace all fwrite/fread/fgets/... to write/read/... .
Before:
       text    data     bss     dec     hex filename
 109827     601    6608  117036   1c92c nuttx/nuttx

After:
       text    data     bss     dec     hex filename
108053     601    6608  115262   1c23e nuttx/nuttx

After with CONFIG_FILE_STREAM disabled:
       text    data     bss     dec     hex filename
 105667     601    6608  112876   1b8ec nuttx/nuttx

Impact
nsh
Testing
QEMU

CI break is fixed by https://github.com/apache/nuttx/pull/8498

Split to 2 PR (readline relative and nsh)

@no1wudi  @xiaoxiang781216
I noticed that the telnet output is too slow with this PR.
https://user-images.githubusercontent.com/25843920/220650403-e7b9ab4e-521b-4607-97f9-36e4ee6e7248.mov

@no1wudi @xiaoxiang781216
I noticed that the telnet output is too slow with this PR.

Without this PR.
https://user-images.githubusercontent.com/25843920/220651338-00b5de65-62aa-446e-aabc-28e02931ea8f.mov

@no1wudi @xiaoxiang781216
I noticed that the telnet output is too slow with this PR.

Without this PR.

Thanks, I'll look at it soon.

@masayuki2009 Which board/config used in your testing ? Could you check the CONFIG_TELNET_TXBUFFER_SIZE value ?

@masayuki2009 Which board/config used in your testing ? Could you check the CONFIG_TELNET_TXBUFFER_SIZE value ?

I've tested telnetd on esp32c3-devkit, and I can't reproduce this situation on it.

@no1wudi
I’m on holiday today so I can not confirm the config now.
However, the above movies were based on spresense:wifi_smp

@no1wudi

Could you check the CONFIG_TELNET_TXBUFFER_SIZE value

$ grep -i CONFIG_TELNET_TX .config
CONFIG_TELNET_TXBUFFER_SIZE=256

@no1wudi

However, the above movies were based on spresense:wifi_smp

The same issue happens with spresense:rndis_smp too.

@no1wudi
The same issue also happens with rv-virt:netnsh_smp (qemu) on ubuntu 18.04 (x86_64)
However, the host PC is fast so the symptom is not worse than the spresense.

@no1wudi
As for esp32-devkitc, I found out how to reproduce the issue.
Please try wapi_smp without CONFIG_NET_TCP_WRITE_BUFFERS=y.
You would see the same symptom as spresense:wifi_smp
In this case (i.e. esp32-devkitc:wapi_smp without CONFIG_NET_TCP_WRITE_BUFFERS=y), reverting this PR would not cause the issue.
--- a/boards/xtensa/esp32/esp32-devkitc/configs/wapi_smp/defconfig
+++ b/boards/xtensa/esp32/esp32-devkitc/configs/wapi_smp/defconfig
@@ -67,7 +67,6 @@ CONFIG_NET_ICMP=y
 CONFIG_NET_ICMP_SOCKET=y
 CONFIG_NET_STATISTICS=y
 CONFIG_NET_TCP=y
-CONFIG_NET_TCP_WRITE_BUFFERS=y
 CONFIG_NET_UDP=y
 CONFIG_NFS=y
 CONFIG_NSH_ARCHINIT=y

@masayuki2009 Thanks for your information, I'll continue debug it.

@no1wudi As for esp32-devkitc, I found out how to reproduce the issue. Please try wapi_smp without CONFIG_NET_TCP_WRITE_BUFFERS=y. You would see the same symptom as spresense:wifi_smp In this case (i.e. esp32-devkitc:wapi_smp without CONFIG_NET_TCP_WRITE_BUFFERS=y), reverting this PR would not cause the issue.

BTW, in my test, I'm using esp32c3-devkit, is'a single core chip.

BTW, in my test, I'm using esp32c3-devkit, is'a single core chip.

I've just confirmed that non-SMP can reproduce the issue too.
--- a/boards/xtensa/esp32/esp32-devkitc/configs/wapi_smp/defconfig
+++ b/boards/xtensa/esp32/esp32-devkitc/configs/wapi_smp/defconfig
@@ -67,7 +67,6 @@ CONFIG_NET_ICMP=y
 CONFIG_NET_ICMP_SOCKET=y
 CONFIG_NET_STATISTICS=y
 CONFIG_NET_TCP=y
-CONFIG_NET_TCP_WRITE_BUFFERS=y
 CONFIG_NET_UDP=y
 CONFIG_NFS=y
 CONFIG_NSH_ARCHINIT=y
@@ -89,8 +88,6 @@ CONFIG_RTC_DRIVER=y
 CONFIG_RTC_NALARMS=2
 CONFIG_SCHED_LPWORK=y
 CONFIG_SIG_DEFAULT=y
-CONFIG_SMP=y
-CONFIG_SMP_NCPUS=2
 CONFIG_SPI=y
 CONFIG_SPIFFS_NAME_MAX=48
 CONFIG_STACK_COLORATION=y

@masayuki2009 The root case is that write ops in telnet driver is nearly unbuffered ,so there are many 1 byte write to socket without stdio's buffer , I'll fix it soon.


Demo web app

I am interested in making use of this tool given the UI is intuitive and better than existing solutions. Is there a demo web app that I can play around with?

I am looking for an alternative to flower, that is why I came here. At least provide some UI on the documentation so that we can have a grasp of understanding how intuitive the UI is. BR.


Orca Issue: Orca model (.pv) file has incorrect format or belongs to a different platform

Have you checked the docs and existing issues?

[X] I have read all of the relevant Picovoice Orca docs
[X] I have searched the existing issues for Orca

SDK
C
Orca package version
0.2.1
Framework version
orca-android 0.2.1
Platform
Linux (x86_64)
OS/Browser version
Android 12
Describe the bug
Initializing Orca gives OrcaInvalidArgumentException saying that "Orca model (.pv) file has incorrect format or belongs to a different platform".
Also, this is a runtime error that doesn't happen all the time. When I reinstall the same apk, sometimes it errors, sometimes it works fine.
Steps To Reproduce

Download orca_params_male.pv file from https://github.com/Picovoice/orca/tree/main/lib/common
Run code

orca = new Orca.Builder()
                    .setAccessKey(Constants.PICOVOICE_ACCESS_KEY)
                    .setModelPath("orca_params_male.pv")
                    .build(context);

See error

Initialize orca error: ai.picovoice.orca.OrcaInvalidArgumentException: Initialization failed:
06-25 19:53:29.284  8109  9659 D TextToSpeechPlayerImpl:   [0] Orca model (.pv) file has incorrect format or belongs to a different platform.
06-25 19:53:29.284  8109  9659 D TextToSpeechPlayerImpl:   [1] Picovoice Error (code `00000136`)
06-25 19:53:29.284  8109  9659 D TextToSpeechPlayerImpl:   [2] Picovoice Error (code `00000136`)
06-25 19:53:29.284  8109  9661 E TextToSpeechPlayerImpl: OrcaStream uninitialized. Cannot synthesize audio.

Expected Behavior
Orca should initialize successfully.

Hi @xindazhou2000, I was unable to reproduce your result. Could you please provide more detailed steps to reproduce that always cause the error so we can better assist you.

Closing due to inactivity.


fix: navigation problem in nav-menu

Run the examples proejct and go to the NavMenu demo
Tab to the expand/collapse button on a FluentNavmenu, then press cursor down to navigate the menu items.
On some it works, on others it does not.

Fixed in #480


question about the extractNearby() function.

Hi, @TixiaoShan
I am confused about the steps of downsampling nearby key poses.
In my understanding, the  intensity of the key frame pose represents the id of the key frame. However, you used a voxel grid to downsample the key frame poses, which will also average the intensity.
Imagine that two key frames come to the same place at different times, their xyz coordinates are close, but the intensity (id) is very different. Then taking the average of the intensity (id) may make the new intensity (id) point to another key frame far away.
I don’t know if I misunderstood, or if this is indeed an implementation error, I am looking forward to your answer.

Well, after my test, I believe this is indeed a bug, and the index of the key frame should not be downsampled.
The result of this bug is that if the robot returns to the position it has passed before, the global map visualization will be partially missing (picture 1).
I have submitted a pull request to fix this  #205
The global map visualization by the new code is shown in picture 2

picture 1

picture 2


Shellquotes not working

Shell quotes not working
I'm not sure why, and it cost me a few days a year ago . But when trying to use paperclip and a processor for ghostscript I have to overwrite the shellquotes. I have found this fix on the internet and I believe it is used quite sometime. This week I updated a lot of gems and I was hoping that I good remove this dirty fix..
  # receives a pdf to convert to png
  has_attached_file :poster,
    :styles => { :thumb => ['180', :png], :medium => ['x720', :png] },
    :processors => [:ghostscript, :thumbnail],
    :validate_media_type => false,
    :convert_options => { :all => '-colorspace CMYK -flatten -quality 100 -density 8' }

module Paperclip
  class Ghostscript < Processor

    attr_accessor :current_geometry, :target_geometry, :format, :whiny, :convert_options, :source_file_options

    def initialize file, options = {}, attachment = nil
      super
      @file                = file
      @format              = options[:format]

      @current_format      =<EMAIL_ADDRESS>      @basename            =<EMAIL_ADDRESS>@current_format)
    end

    def make
      src = @file
      dst = Tempfile.new([@basename, @format ? ".#{@format}" : ''])
      dst.binmode

      begin
        parameters = []
        parameters << '-dNOPAUSE -dBATCH -sDEVICE=pngalpha -r144 -dUseCIEColor'
        parameters << '-sOutputFile=:dest'
        parameters << ':source'

        parameters = parameters.flatten.compact.join(' ').strip.squeeze(' ')

        success = Paperclip.run('gs', parameters, :source => "#{File.expand_path(src.path)}", :dest => File.expand_path(dst.path))
      rescue PaperclipCommandLineError => e
        raise PaperclipError, "There was an error processing the thumbnail for #{@basename}" if @whiny
      end
      dst
    end
  end
end

But unfortunately I get the dreaded NotIdentifiedByImageMagick error. But I scouerd the internet and found this which fixes my problem;
Cocaine::CommandLine.module_eval do

  def run(interpolations = {})
    @exit_status = nil
    begin
      full_command = command(interpolations)
      log("#{colored("Command")} :: #{full_command}")
      @output = execute(full_command)
    rescue Errno::ENOENT => e
      raise Cocaine::CommandNotFoundError, e.message
    ensure
      @exit_status = $?.respond_to?(:exitstatus) ? $?.exitstatus : 0
    end

    if @exit_status == 127
      raise Cocaine::CommandNotFoundError
    end

    unless @expected_outcodes.include?(@exit_status)
      message = [
        "Command '#{full_command}' returned #{@exit_status}. Expected #{@expected_outcodes.join(", ")}",
        "Here is the command output: STDOUT:\n", command_output,
        "\nSTDERR:\n", command_error_output
      ].join("\n")
      raise Cocaine::ExitStatusError, message
    end

    command_output
  end

  private
  def interpolate(pattern, interpolations)
    interpolations = stringify_keys(interpolations)

    pattern.gsub(/:\{?(\w+)\b\}?/) do |match|

      key = match.tr(":{}", "")

      if interpolations.key?(key)
        "'#{interpolations[key]}'"
      else
        match
      end
    end
  end
end

Well the upperpart is just that cocaine uses my interpolate but why is this still a problem using paperclip?

I realize this is a year old, but in the interest of figuring this out I'd need to know what a command looked like before and after you fix.
FWIW, your fix is not secure. It allows for shell command injection with a specially crafted file name.

This is now well over a year old and has had plenty of time for feedback, so I'm going to close it. If it's still a problem, please resubmit.


Use new VCS roots

We somehow experience slow "Collecting Changes" time on VCS.
Let's try to use a new VCS root with identical settings as before.
Also, this PR resolves this warning by using GitHub token:

@bot-gradle test and merge

OK, I've already triggered a build for you.


Add Pod resource

Changes

:gift: Add helpers to create a pod (f5351b03e1c4e04224e0c5ffed9452fad0414bad)
:broom: Move WithAnnotations and WithLabels to pkg/manifest as this is shared among different resources (91d02a331ca4e7e5180f7510cce7750451af9dc1)

Similar to #452
/kind enhancement

/assign @pierDipi

/hold
need to add the WithPort option

/hold cancel

@pierDipi could you recheck on this PR?


Cannot debug client-side code in VS Code with out-of-the-box SPA project.

I've created a new React SPA for ASP.NET Core 2.1.
Out of the box, if I open the ClientApp directory in VS Code I can then hit F5 to launch Chrome (using the Chrome debugger extension).
However, when I set breakpoints, each breakpoint is greyed out with the following message:
Unverified breakpoint, Breakpoints set but not yet bound

I'm fairly certain this is an issue with the launch.json configuration, specifically the webRoot (and possibly the sourceMapPathOverrides) property.
Here's the standard launch.json that I'm using:
{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "type": "chrome",
            "request": "launch",
            "name": "Launch Chrome against localhost",
            "url": "https://localhost:44311",
            "webRoot": "${workspaceFolder}",
            "sourceMapPathOverrides": {
                "webpack:///./*": "${webRoot}/*",
                "webpack:///src/*": "${webRoot}/*",
                "webpack:///*": "*",
                "webpack:///./~/*": "${webRoot}/node_modules/*",
                "meteor://💻app/*": "${webRoot}/*"
            }
        }
    ]
}

And the output of calling .scripts in a debug session is:
› https://localhost:44311/static/js/bundle.js

Is the assumption that it's an issue with launch.json correct?  If so, what is the correct configuration out of the box?

@SteveSandersonMS - any ideas on this?

We have notes about configuring VS Code debugging at https://github.com/aspnet/JavaScriptServices/tree/master/src/Microsoft.AspNetCore.SpaServices#using-visual-studio-codes-debugger-for-chrome-extension
It's possible that VS code has changed something in the meantime, but those instructions did work before.


added ability to add extra lines to init defaults file

I want to enable ipv4 by adding the following line to the defaults file:
'export _JAVA_OPTIONS="-Djava.net.preferIPv4Stack=true"
But actually others might want to customise some of the variables the init script uses and the shell script, so i thought id add a more flexible option.

@tomzo Ive fixed this now. Any other objections or can you merge? :smile_cat:

@willejs I like this. Some tests would be nice, but'll merge it anyway.
Thanks a lot :)

:+1: I was about to submit a PR for this in the next couple of days for GoCD's own build machines.


[HOPS-751], [HOPS-752], [HOPS-753], [HOPS-754], [HOPS-755], [HOPS-756], [HOPS-757], [HOPS-758], [HOPS-759], [HOPS-760], [HOPS-761], [HOPS-762], [HOPS-763], [HOPS-764], [HOPS-765], [HOPS-766], [HOPS-767], [HOPS-768], [HOPS-770], [HOPS-771], [HOPS-772], [HOPS-773], [HOPS-774], [HOPS-775], [HOPS-776], [HOPS-777], [HOPS-778], [HOPS-779], [HOPS-780] and [HOPS-781]

Make sure there is no duplicate PR for this issue

Please check if the PR fulfills these requirements

[ ] Tests for the changes have been added and passed (for bug fixes / features)
[x] HOPS JIRA issue has been opened for this PR
[x] All commits have been squashed down to a single commit
[x] The commit message has the following format: [HOPS-XXX] message

Post a link to the associated JIRA issue
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-751
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-752
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-753
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-754
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-755
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-756
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-757
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-758
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-759
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-760
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-761
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-762
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-763
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-764
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-765
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-766
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-767
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-768
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-770
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-771
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-772
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-773
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-774
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-775
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-776
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-777
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-778
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-779
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-780
https://hopshadoop.atlassian.net/projects/HOPS/issues/HOPS-781

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

What is the new behavior (if this is a feature change)?

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

Other information:
depends on hopshadoop/hops-metadata-dal#135 and hopshadoop/hops-metadata-dal-impl-ndb#168

Jenkins test this please.

Jenkins test this please.


Add utility function for annotated malloc

Add utility function alloc_annotated
Description
Create utility function alloc_annotated in a common header "annotated_class_util.hpp" to improve the code brevity and support compile-time type check. The new function takes an annotated_arg type as the template parameter and returns an instance of that annotated_arg type. This ensures the properties of the returned memory (e.g. buffer location, alignment) match with the annotations on the kernel arguments
Code samples under hls_flow_interfaces/ using annotated_arg are updated to partially use the new utility function.
An example of using the utility function is
// Create type alias for the annotated_arg type
using karg_c_t = sycl::ext::oneapi::experimental::annotated_arg<
      int *, decltype(sycl::ext::oneapi::experimental::properties{
                 sycl::ext::intel::experimental::buffer_location<kBL3>,
                 sycl::ext::intel::experimental::dwidth<32>,
                 sycl::ext::intel::experimental::latency<0>,
                 sycl::ext::intel::experimental::read_write_mode_write,
                 sycl::ext::oneapi::experimental::alignment<4>})>;

// in host code, call the utility function to get the annotated_arg instance
auto c = fpga_tools::alloc_annotated<karg_c_t>(count, q);

Type of change
Please delete options that are not relevant. Add a 'X' to the one that is applicable.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Implement fixes for ONSAM Jiras

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[x] Command Line
[ ] oneapi-cli
[ ] Visual Studio
[ ] Eclipse IDE
[ ] VSCode
[ ] When compiling the compliler flag "-Wall -Wformat-security -Werror=format-security" was used

Hey Di,
I've discussed this change with Paul and here are the suggestions that we agreed on:

The header should not be in the include folder, as it is experimental
Instead of modifying existing samples, we should create a new simple sample in Tutorials/experimental to demonstrate the use of this helper function
This helper function should be added to the sample if the intention is to be released as part of the compiler in the future

Please let us know what are your thoughts
Thanks!


RPRBLND-1958: Update Warning image for restart render

PURPOSE
Update Warning image for restart render
EFFECT OF CHANGE
Render restarts by itself without warning image if render quality changed
TECHNICAL STEPS
Added creation of new render if render quality changed

Now in viewport render works only in Full mode, the render result of other modes looks like GL render.

I'm not sure if this is an issue?? Do you approve or not?

Now in viewport render works only in Full mode, the render result of other modes looks like GL render.

I'm not sure if this is an issue?? Do you approve or not?

Discussed an issue offline.
Now works as expected.


Converting data to qlib format

What's the recommended way of converting qlib data?
I have a new set of data coming in csv everyday.
The data will be consist of two part:

new data of unadjusted stocks
new data of adjusted stocks as well as their adjusted historical data.

I see qlib have three dump feature: dump_all, dump_update, dump_fix. Their respective function is explained in #535

The dump has three classes:

DumpDataAll: dump all data

Dump all data from csv_path into qlib_dir, which will overwrite all data (calendars/instruments/features)

DumpDataFix: adding a new field, symbol, rewriting of an existing field

Dumping data from csv_path into qlib_dir will overwrite existing features data; if new fields are added, they will be added according to qlib_dir/calendars/<freq>.txt and the qlib_dir/instruments/all. txt file, without modifying existing data

DumpDataUpdate: append new data to existing data

append the data from csv_path to the existing data

Currently, the dump feature part of dump_update takes roughly the same time as dump_all, sometimes more. Note I don't have new features, all the features are the same as my initial dump_all
And if I understand correctly, I have to do a separate dump_fix to fix the old data if they split or dividend that day.
So all I did now is:

separate data in two folder: only_append_folder, fix_historical_and_append_folder.
dump_update only_append_folder
dump_fix fix_historical_and_append_folder

Is this the correct procedure?
Maybe an enhancement on dump_update?

@bbbzhai
hi,Here is the document about CSV conversion to Qlib format
https://qlib.readthedocs.io/en/latest/component/data.html#converting-csv-format-into-qlib-format

If you don't have new features, you can skip dump_fix fix_historical_and_append_folder
split&dividend does not affect dump_bin.

If you don't have new features, you can skip dump_fix fix_historical_and_append_folder
split&dividend does not affect dump_bin.

cool! This is what I hope it should achieve. Thanks!


FFI bug on iOS: Symbol not found at runtime with IPA

I'm cross-referencing this dart FFI issue to flutter because this seems relate to both FFI and flutter toolchain
My app, using FFI linked C++ code as backend, runs fine on the iOS with Release dev build distributed from Xcode directly.
Problem
But with the IPA created from the Release Archive, I get runtime errors
[VERBOSE-2:ui_dart_state.cc(166)] Unhandled Exception: Invalid argument(s): Failed to lookup symbol (dlsym(RTLD_DEFAULT, NativeInit): symbol not found)
#0      DynamicLibrary.lookup (dart:ffi-patch/ffi_dynamic_library_patch.dart:31)
#1      ffiInit (package:myapp/myapp.dart:25)
#2      ffiInit (package:myapp/myapp.dart)
#3      MyAppBackend.init (package:myapp_example/backend.dart:99)
<asynchronous suspension>
#4      MyAppBackend.asyncInit (package:myapp_example/backend.dart:64)
<asynchronous suspension>

The missing symbol should be part of the CPP files added to the Runner Xcode project.
And those files build and run fine with the Release build.
So I expect the IPA would behave exactly the same as the Release build.
What am I missing?
Environment
The IPA is created from the Xcode standard procedure as a Development distribution, running Xcode 11.6.
$ flutter doctor -v
[✓] Flutter (Channel dev, 1.21.0-1.0.pre, on Mac OS X 10.15.5 19F101, locale en-CN)
    • Flutter version 1.21.0-1.0.pre at /Applications/flutter
    • Framework revision f25bd9c55c (2 weeks ago), 2020-07-14 20:26:01 -0400
    • Engine revision 99c2b3a245
    • Dart version 2.9.0 (build 2.9.0-21.0.dev 20bf2fcf56)
    • Pub download mirror https://pub.flutter-io.cn
    • Flutter download mirror https://storage.flutter-io.cn

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.0)
    • Android SDK at /Applications/Android/sdk
    • Platform android-30, build-tools 30.0.0
    • ANDROID_HOME = /Applications/Android/sdk
    • ANDROID_SDK_ROOT = /Applications/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.6)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.6, Build version 11E708
    • CocoaPods version 1.9.1

[✓] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 47.1.2
    • Dart plugin version 193.7361
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)

[✓] VS Code (version 1.46.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.12.1

[✓] Connected device (2 available)
    • ONEPLUS A6000 (mobile)   • <my uuid> • android-arm64 • Android 10 (API 29)
    • MyiPhone (mobile) • <my uuid> • ios           • iOS 13.5.1

• No issues found!

create_flutter_ffi_plugin.sh.zip
I reproduced this with a very simple plugin project based on the official native_add example.
Repro

Use the attached script to create a plugin project: /path/to/create_flutter_ffi_plugin.sh <ProjName> <yes|no>
Build the iOS project in Android Studio to bootstrap.
Open the Runner Xcode workspace.
Configure the iOS provisioning: With my project, I had to use a shared Provisioning Profile instead of using a Team signature.
Add myapp/lib/native_add.cpp to the Xcode project.
Build and run the Runner Xcode project.

You should see the homepage showing the device name and version plus the native_add example: 1+2==3

Now Archive the project
Press Window > Organizer > Distribute App
Select Development, then Next, Next, Select provisioning profile, and confirm through the rest of the steps to generate Runner.ipa.
Open official Apple Configurator 2 and install the generated Runner.ipa onto your iOS device
Run the Runner app on the device.

Now you should see a blank screen.
Observation
The code runs fine on the device with Xcode directly, but can't enter homepage
The device console shows the following log related to the Runner:
	default	20:42:04.072195+0800	kernel	memorystatus: set assertion priority(10) target Runner:270
	error	20:42:04.124898+0800	kernel	Sandbox: Runner(270) deny(1) sysctl-read kern.bootargs
connection	default	20:42:04.203428+0800	Runner	Initializing connection
process	default	20:42:04.204375+0800	Runner	Removing all cached process handles
connection	default	20:42:04.204895+0800	Runner	Sending handshake request attempt #1 to server
connection	default	20:42:04.204987+0800	Runner	Creating connection to com.apple.runningboard
Common	default	20:42:04.207922+0800	Runner	FBSWorkspace connecting to endpoint : <private>
default	default	20:42:04.213118+0800	backboardd	Connection added: IOHIDEventSystemConnection uuid:A226279B-013F-46B6-B4C9-7B366BB8590D pid:270 process:Runner type:Passive entitlements:0x0 caller:BackBoardServices: <redacted> + 364 attributes:{
    HighFrequency = 0;
    bundleID = "com.example.nativeAddExample";
    pid = 270;
} state:0x0 events:0 mask:0x0
Common	default	20:42:04.230750+0800	Runner	FBSWorkspace registering source: <private>
Common	default	20:42:04.230873+0800	Runner	FBSWorkspace connected to endpoint : <private>
general	default	20:42:04.230905+0800	Runner	Added observer for process assertions expiration warning: <_RBSExpirationWarningAssertion: 0x28150f0c0; identifier: com.apple.runningboardservices.processExpirationWarningForHandle; reason: observation; valid: YES>
	default	20:42:04.260195+0800	Runner	kExcludedFromBackupXattrName set on path: <private>
AXSupportCommon	default	20:42:04.260272+0800	Runner	Retrieving resting unlock: 0
Default	default	20:42:04.262358+0800	Runner	Registering for test daemon availability notify post.
Default	default	20:42:04.263381+0800	Runner	notify_get_state check indicated test daemon not ready.
	error	20:42:04.266933+0800	kernel	Sandbox: Runner(270) deny(2) file-test-existence /private/etc/.mdns_debug
connection	default	20:42:04.292603+0800	Runner	Handshake succeeded
connection	default	20:42:04.292682+0800	Runner	Identity resolved as application<com.example.nativeAddExample>
	error	20:42:04.349930+0800	kernel	Sandbox: Runner(270) deny(1) sysctl-read machdep.cpu.brand_string
	default	20:42:04.583446+0800	Runner	HTHangEventCreate: HangTracing is disabled. Not creating a new event.
Common	default	20:42:04.585776+0800	Runner	FBSWorkspace already connected to endpoint : <private>
	default	20:42:04.591634+0800	Runner	flutter:
	default	20:42:04.609826+0800	Runner	flutter: Another exception was thrown: Instance of 'DiagnosticsProperty<void>'
client	default	20:42:04.627820+0800	Runner	Received configuration update from daemon (initial)
default	default	20:42:18.973432+0800	backboardd	Connection removed: IOHIDEventSystemConnection uuid:A226279B-013F-46B6-B4C9-7B366BB8590D pid:270 process:Runner type:Passive entitlements:0x0 caller:BackBoardServices: <redacted> + 364 attributes:{
    HighFrequency = 0;
    bundleID = "com.example.nativeAddExample";
    pid = 270;
} state:0x1 events:0 mask:0x0
connection	default	20:42:21.008084+0800	Runner	Initializing connection
process	default	20:42:21.008176+0800	Runner	Removing all cached process handles
default	default	20:42:21.008470+0800	backboardd	Connection added: IOHIDEventSystemConnection uuid:788A6D31-C6AA-4A99-A9DF-0BD3B12C618E pid:278 process:Runner type:Passive entitlements:0x0 caller:BackBoardServices: <redacted> + 364 attributes:{
    HighFrequency = 0;
    bundleID = "com.example.nativeAddExample";
    pid = 278;
} state:0x0 events:0 mask:0x0
Common	default	20:42:21.012182+0800	Runner	FBSWorkspace connecting to endpoint : <private>
Common	default	20:42:21.012331+0800	Runner	FBSWorkspace registering source: <private>
Common	default	20:42:21.012409+0800	Runner	FBSWorkspace connected to endpoint : <private>
general	default	20:42:21.012547+0800	Runner	Added observer for process assertions expiration warning: <_RBSExpirationWarningAssertion: 0x281a578c0; identifier: com.apple.runningboardservices.processExpirationWarningForHandle; reason: observation; valid: YES>
AXSupportCommon	default	20:42:21.037445+0800	Runner	Retrieving resting unlock: 0
	error	20:42:21.043662+0800	kernel	Sandbox: Runner(278) deny(1) sysctl-read kern.bootargs
	error	20:42:21.043730+0800	kernel	Sandbox: Runner(278) deny(2) file-test-existence /private/etc/.mdns_debug
Default	default	20:42:21.059825+0800	Runner	Registering for test daemon availability notify post.
Default	default	20:42:21.059911+0800	Runner	notify_get_state check indicated test daemon not ready.
connection	default	20:42:21.060075+0800	Runner	Sending handshake request attempt #1 to server
connection	default	20:42:21.079293+0800	Runner	Creating connection to com.apple.runningboard
	default	20:42:21.099663+0800	kernel	memorystatus: set assertion priority(10) target Runner:278
connection	default	20:42:21.116539+0800	Runner	Handshake succeeded
connection	default	20:42:21.116590+0800	Runner	Identity resolved as application<com.example.nativeAddExample>
Common	default	20:42:21.116694+0800	Runner	FBSWorkspace already connected to endpoint : <private>
	error	20:42:21.120500+0800	kernel	Sandbox: Runner(278) deny(1) sysctl-read machdep.cpu.brand_string
	default	20:42:21.294773+0800	Runner	HTHangEventCreate: HangTracing is disabled. Not creating a new event.
	default	20:42:21.296731+0800	Runner	flutter:
	default	20:42:21.304259+0800	Runner	flutter: Another exception was thrown: Instance of 'DiagnosticsProperty<void>'
client	default	20:43:21.231815+0800	Runner	Received configuration update from daemon (initial)

Comparison
I created a default flutter project using Android Studio, repeated the Repro steps, and found that the IPA works as expected.
Thoughts
So this is probably not a regular Dart-land problem, and I start to suspect that this is a flutter/Android Studio bug over FFI.

This problem is blocking us from doing internal QA. Could anyone share an idea?

So using sentry, I've got the same error report from the simple native_add example with the development IPA.
ArgumentError native_add.dart in nativeAdd
Invalid argument(s): Failed to lookup symbol (dlsym(RTLD_DEFAULT, native_add): symbol not found)

Finally found a workaround in this issue . Credits goes to @vladimirdjokic76.
Quoting it here:
In Xcode

Select Target Runner -> Build Settings -> Strip Style -> change from "All
Symbols" to "Non-Global Symbols"

The cause of this problem was discussed in these issues

#41
#39

For now, I'm no longer blocked. But this knowledge is essential and should definitely go into the FFI / C Interop documentation.

I decide to keep this open until it gets attention and receives a documentation label.

/cc @dcharkes

@dcharkes
Can this be closed since https://github.com/flutter/website/pull/4464 is merged?
Thank you

Yes, the info is now in the documentation.

I am still facing this issue, how can I fix it? please

@felipesses please follow the steps to fix the issue from the documentation. If the issue still persists please open a new issue, providing a minimal reproduction, the outputted logs, and the versions of the software you're using.


Update Android part of flutter_view to use the new platform message c…

…hannel.

LGTM but not to Travis.

>>> RUNNING in .: bin/flutter analyze --flutter-repo
Analyzing 967 files...
[lint] Specify type annotations. (examples/flutter_view/lib/main.dart, line 37, col 13)

Thanks for the comments, all addressed.


Bump Xcode, macos image, bazel, and carthage versions

We need to have Xcode 13+ in order to use some of the newer swiftc features, like -scan-dependencies for explicit module compilation.
Also, Xcode 14.1 is out now, so we should be testing something newer than 12.5. Hopefully the next upgrade is marginally smoother 😅

I had started this in #511 but got blocked by Carthage issues

🤦 I had seen that PR and forgot about it until I got blocked on other stuff and then redid it. Let me look through there and see if maybe you caught something I missed 😅
The good news is that I've got the carthage issues sorted! Now I'm just down to (hopefully) one failing test with the xcodeproj generators that I need to resolve.

Think we'll need to update github status checks to remove 12.5 and use the new Xcode check instead

cc @mattrobmattrob @ob @ShawnMa16 ^ JFYI

Think we'll need to update github status checks to remove 12.5 and use the new Xcode check instead

Took me a minute to figure out where to do that, but this is updated to the new check name.


Use loguru instead of builtin logging

Update code to use loguru package for easier logging, at least it seems easier and hopefully more efficient than  the way I had logging setup.  No longer logs to file that is up to the user to configure.
Fixes issue #150.

Codecov Report
Patch coverage: 81.53% and project coverage change: +0.09 :tada:

Comparison is base (0318dbd) 81.80% compared to head (ebe3d17) 81.90%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #153      +/-   ##
==========================================
+ Coverage   81.80%   81.90%   +0.09%
==========================================
  Files         262      261       -1
  Lines       17595    17526      -69
==========================================
- Hits        14394    14354      -40
+ Misses       3201     3172      -29

Impacted Files
Coverage Δ

.../transfer_functions/io/edi/metadata/information.py
88.19% <ø> (-0.09%)
:arrow_down:

mt_metadata/utils/list_dict.py
88.33% <ø> (-0.10%)
:arrow_down:

setup.py
0.00% <ø> (ø)

mt_metadata/base/schema.py
42.23% <16.66%> (-0.71%)
:arrow_down:

mt_metadata/utils/validators.py
81.12% <60.00%> (ø)

mt_metadata/timeseries/stationxml/utils.py
90.21% <66.66%> (ø)

mt_metadata/transfer_functions/io/edi/edi.py
70.97% <66.66%> (ø)

mt_metadata/base/metadata.py
75.23% <75.00%> (+1.15%)
:arrow_up:

...meseries/stationxml/xml_inventory_mt_experiment.py
83.21% <80.00%> (ø)

mt_metadata/transfer_functions/io/zfiles/zmm.py
59.88% <83.33%> (ø)

... and 8 more

... and 1 file with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Stuck at provisioning

Describe the bug
Just updated my React JS code at Github and Amplify got stuck at provisioning.
I've also tried to trigger it from curl and the same occurs.
Screenshots
Build
History
I've created another build from a development branch and the same occurs.

I have this same problem!
I cannot even delete the branch nor the app, everything seems stuck.
I created a new app and the first deployment is also stuck at provisioning

I have the same issue.  I have tried multiple ways to build my app and each is stuck in provisioning.  I can't delete through the web interface or CLI.  I need to get these deleted.

Same issue too. Stuck at provisioning. I can't even cancel the build

I guess it's this issue: https://phd.aws.amazon.com/phd/home#/event-log?eventID=arn:aws:health:us-east-1::event/AMPLIFY/AWS_AMPLIFY_OPERATIONAL_ISSUE/AWS_AMPLIFY_OPERATIONAL_ISSUE_f2e47794-daa0-4522-a28d-90c0e479ac74&eventTab=details

It now moved to "Deploy". It's running more than 2 hours.

It now moved to "Deploy". It's running more than 2 hours.

How did you move to Deploy?

I just did a build and for me it's working again.

I am experiencing this issue currently :(

I removed all running build and started again. Now it's working.

I removed all running builds and started a new one. Now it's working.

Hi, still experiencing this issue. Any fix? These bugs make me feel so insecure about using Amplify as it is truly unreliable.


Unable to load fips provider in release build using config from BIO buffer

Hi,
For fips mode, we are trying to load fips provider from config using BIO buffer instead of from file.
Out fips config looks like below:
constexpr const char FipsConfig[] = R"(config_diagnostics = 1
openssl_conf = openssl_init

[openssl_init]
providers = provider_sect

[provider_sect]
base = base_sect
fips = fips_sect

[base_sect]
activate = 1

[fips_sect]
activate = 1
conditional-errors = 1
security-checks = 1
module-mac = CA:B4:A5:B1:09:35:FE:B2:2C:1E:0C:68:50:FF:55:F8:7D:F9:26:BB:6A:7B:3A:24:F5:77:4E:CD:A4:D9:4B:04)";

To load this config to our custom OSSL_LIB_CTX using BIO buffer, the pseudo flow of the code is:
 std::unique_ptr<CONF, CnfDeleter> cnf{
      NCONF_new_ex(SSL_lib_ctx.get(), nullptr)};
  if (!cnf) {
     log error;
  }

   std::unique_ptr<BIO, BioDeleter> bio{BIO_new_mem_buf(
        fips_config.c_str(), static_cast<int>(fips_config.length()))};
    if (!bio) {
      log error;
    }

   long errline{-1};
   auto ret = NCONF_load_bio(cnf.get(), bio.get(), &errline);
   if (ret <= 0) {
     log error;
   }

   ret = CONF_modules_load(cnf.get(), nullptr, 0);
   if (ret <= 0) {
       log error;
   }

At the end of this setup, I am calling below API to print and check which providers are loaded in my context:
OSSL_PROVIDER_do_all(SSL_lib_ctx.get(), testcallback, nullptr);

The issue I am seeing is in Debug build, I get both base and fips provider correctly loaded but in release mode, without encountering any error, there is only base provider gets loaded.
Also, just to clarify, we are always loading default provider from code so default provider is successfully there in both debug and release build.
Can someone guide me to investigate this further?

The ( and ) in the config look suspicious. Also is the module mac the real SHA2-256 HMAC computed over the fips.so file with the right key?

The ( and ) are the start and end indicator of C++ literal strings. They are not part of config buffer.
I believe HMAC is correct as I have double checked using correct key.
The same config in debug mode (with different module-mac for debug build) works fine. If there was any error in config or wrong module-mac, I was expecting some of the load API to fail and indicate error but that is not happening. My config load code above runs fine without any return error and still fips module is not loaded.

Do you get anything useful in the OpenSSL error stack, e.g. via ERR_print_errors_fp(stderr);?

I do not see in error in stack during above config load operation.
After the config load, later we do:
SSL_CTX *ctx = SSL_CTX_new_ex(SSL_lib_ctx.get(), "", TLS_method());

which gives below error stack:
crypto/evp/evp_fetch.c,inner_evp_generic_fetch,349: error:0308010C:digital envelope routines::unsupported
Non-default library context, Algorithm (EC : 88), Properties ()
crypto/evp/evp_fetch.c,inner_evp_generic_fetch,349: error:0308010C:digital envelope routines::unsupported
Non-default library context, Algorithm (EC : 88), Properties ()
crypto/evp/evp_fetch.c,inner_evp_generic_fetch,349: error:0308010C:digital envelope routines::unsupported
Non-default library context, Algorithm (EC : 88), Properties ()
crypto/evp/evp_fetch.c,inner_evp_generic_fetch,349: error:0308010C:digital envelope routines::unsupported
Non-default library context, Algorithm (EC : 88), Properties ()
 crypto/evp/evp_fetch.c,inner_evp_generic_fetch,349: error:0308010C:digital envelope routines::unsupported
Non-default library context, Algorithm (EC : 88), Properties ()
crypto/evp/evp_fetch.c,inner_evp_generic_fetch,349: error:0308010C:digital envelope routines::unsupported
Non-default library context, Algorithm (EC : 88), Properties ()
crypto/evp/evp_fetch.c,inner_evp_generic_fetch,349: error:0308010C:digital envelope routines::unsupported
Non-default library context, Algorithm (EC : 88), Properties ()
 crypto/evp/evp_fetch.c,inner_evp_generic_fetch,349: error:0308010C:digital envelope routines::unsupported
Non-default library context, Algorithm (X25519 : 91), Properties ()
 crypto/evp/evp_fetch.c,inner_evp_generic_fetch,349: error:0308010C:digital envelope routines::unsupported
Non-default library context, Algorithm (X448 : 92), Properties ()
crypto/evp/evp_fetch.c,inner_evp_generic_fetch,349: error:0308010C:digital envelope routines::unsupported
Non-default library context, Algorithm (DH : 86), Properties ()
crypto/evp/evp_fetch.c,inner_evp_generic_fetch,349: error:0308010C:digital envelope routines::unsupported
Non-default library context, Algorithm (DH : 86), Properties ()
crypto/evp/evp_fetch.c,inner_evp_generic_fetch,349: error:0308010C:digital envelope routines::unsupported
Non-default library context, Algorithm (DH : 86), Properties ()
crypto/evp/evp_fetch.c,inner_evp_generic_fetch,349: error:0308010C:digital envelope routines::unsupported
Non-default library context, Algorithm (DH : 86), Properties ()
crypto/evp/evp_fetch.c,inner_evp_generic_fetch,349: error:0308010C:digital envelope routines::unsupported
Non-default library context, Algorithm (DH : 86), Properties ()
ssl/ssl_lib.c,SSL_CTX_new_ex,3253: error:0A0000A1:SSL routines::library has no ciphers

Also, this is the call I am using to test which providers are loaded after above config load.
int testcallback(OSSL_PROVIDER *prov, void *) {
  print("Loaded provider name: ", OSSL_PROVIDER_get0_name(prov));
  return 1;
}

OSSL_PROVIDER_do_all(SSL_lib_ctx.get(), testcallback, nullptr);

and It logs below:
m=Loaded provider name: : v=base
m=Loaded provider name: : v=default

As I had mentioned below, default provider is loaded first from the code.
in the debug build, I see below log with same config structure:
m=Loaded provider name: : v=base
m=Loaded provider name: : v=default
m=Loaded provider name: : v=fips

Could there be some environment difference between your debug and release builds?
You might want to try running your application through strace and look for where it loads the config file to see if that sheds any light.

I am trying to make sense of this but this is what I see with strace during the config load time.
I see the fips.so is getting loaded in memory but then gets unloaded.
open("/home/devuser/build/mx_rel64/install/bin/../lib/fips.so", O_RDONLY) = 20
read(20, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\220\1\0\0\0\0\0"..., 832) = 832
lseek(20, 1849264, SEEK_SET)            = 1849264
read(20, "\4\0\0\0\20\0\0\0\5\0\0\0GNU\0\2\0\0\300\4\0\0\0\3\0\0\0\0\0\0\0"..., 768) = 768
fstat(20, {st_mode=S_IFREG|0755, st_size=1947080, ...}) = 0
mmap(NULL, 4044072, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 20, 0) = 0x7f3ec5a1e000
mprotect(0x7f3ec5be2000, 2097152, PROT_NONE) = 0
mmap(0x7f3ec5de2000, 94208, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 20, 0x1c4000) = 0x7f3ec5de2000
mmap(0x7f3ec5df9000, 1320, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f3ec5df9000
close(20)                               = 0
mprotect(0x7f3ec5de2000, 90112, PROT_READ) = 0
futex(0x7f3ec5df93f0, FUTEX_WAKE_PRIVATE,<PHONE_NUMBER>) = 0
open("/home/devuser/build/mx_rel64/install/bin/../lib/fips.so", O_RDONLY) = 20
fstat(20, {st_mode=S_IFREG|0755, st_size=1947080, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3eca5cc000
read(20, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\220\1\0\0\0\0\0"..., 4096) = 4096
read(20, " \335\27\0\0\0\0\0\0K<\0\0\0\0\0\10\0\0\0\0\0\0\0\326\330\27\0\0\0\0\0"..., 4096) = 4096
read(20, "\10\0\0\0\0\0\0\0\v\332\27\0\0\0\0\0\10U<\0\0\0\0\0\10\0\0\0\0\0\0\0"..., 4096) = 4096
read(20<EMAIL_ADDRESS>4096) = 4096
.
.
.
.
.
read(20, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
read(20, "\270\235=\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\226\207\1\0\0\0\0\0"..., 4096) = 4096
read(20, "\0\0\0\0\0\0\0\0T\0\0\0\4\0\0\0\2\0\0\0\0\0\0\0\30\t\0\0\0\0\0\0"..., 4096) = 1480
read(20, "", 4096)                      = 0
read(20, "", 4096)                      = 0
close(20)                               = 0
munmap(0x7f3eca5cc000, 4096)            = 0

I can confirm that this code works both in debug and production builds:
#include <string.h>
#include <openssl/conf.h>
#include <stdlib.h>
#include <stdio.h>
#include <openssl/provider.h>
#include <openssl/err.h>
#include <openssl/bio.h>
#include <openssl/buffer.h>

static char *conf = "config_diagnostics = 1\n"
                   "openssl_conf = openssl_init\n"
                   "\n"
                   "[openssl_init]\n"
                   "providers = provider_sect\n"
                   "\n"
                   "[provider_sect]\n"
                   "base = base_sect\n"
                   "fips = fips_sect\n"
                   "\n"
                   "[base_sect]\n"
                   "activate = 1\n"
                   "\n"
                   "[fips_sect]\n"
                   "activate = 1\n"
                   "conditional-errors = 1\n"
                   "security-checks = 1\n"
                   "module-mac = E3:49:8D:DC:F9:55:46:DF:ED:69:5B:09:10:8B:EE:C9:91:EC:75:BD:0C:37:C3:71:85:EB:1C:FF:20:B0:1F:9C\n";

static int cb(OSSL_PROVIDER *p, void *data)
{
    printf("%s\n", OSSL_PROVIDER_get0_name(p));
    return 1;
}

int main() {
    long err = 0;
    CONF *c = NCONF_new(NULL);
    BIO *b = BIO_new_mem_buf(conf, strlen(conf));
    int ret = EXIT_FAILURE;

    if (b == NULL) goto err;
    if (conf == NULL) goto err;

    if (NCONF_load_bio(c, b, &err) <= 0) goto err;
    if (CONF_modules_load(c, NULL, 0) <= 0) goto err;
    OSSL_PROVIDER_do_all(NULL, cb, NULL);
    ret = EXIT_SUCCESS;
err:
    if (ret != EXIT_SUCCESS)
        ERR_print_errors_fp(stdout);
    NCONF_free(c);
    BIO_free(b);
    return EXIT_FAILURE;
}

I'd suggest also add the legacy provider into the mix since it is easier to load than the FIPS one -- it not loading points you to an environmental problem rather than a checksum one.
I suspect your not getting the provider paths set up properly and the FIPS provider isn't being found.

I suspect your not getting the provider paths set up properly and the FIPS provider isn't being found.

strace seems to suggest that it is found but subsequently fails. This suggests a checksum problem.
Attempting to load the legacy provider instead of the fips provider is a good suggestion. This will tell us if there is a fundamental problem loading providers at all, or if it is specific to the fips module.

legacy provider is loading fine. So as you suggested I am checking with our symbol stripping code to make sure that checksum is not changed.

yes, that was it. I thought I had correct checksum but after that we had a step to strip symbols from fips.so which would change the checksum.
Thanks for helping with this.
I think a proper return error or on error stack for the wrong checksum would be very helpful. I think I have seen checksum error when using config load from file but this path of loading from buffer is not giving the error.

marking as inactive, to be closed at the end of 3.4 dev barring further input


Fix typescript errors in Vue files, fix regression in "Recent Commits" chart

Fix all typescript errors in .vue files
Fix regression from https://github.com/go-gitea/gitea/pull/32329 where "Recent Commits" chart would not render.

Will pick this up again in a few days.

I managed to merge from main branch (189722b5e96d2e21d9102302a2d7fb6a61fae300) to resolved the conflicts caused by my PRs.

And since it contains a bug fix, add the 1.23 milestone

Thanks, looking good. This is ready.


Allow configuration per Bounded Context

It's already possible to define "Bounded Contexts" for the Event Store Backend configuration.
It would be useful to be able to override other configuration per BC, too.
For example the event (de)serialization

I'll close this one in favor of #212 => it should all be configured through the Event Store – which BC that belongs to is a different concern


How to export a .pb file

Hi, I have problem exporting a .pb file with the current produced .h5 files. I do not think you have provided the export method in the project file. Could you please give me some advice on that? @bubbliiiing

Sorry, I don't know how to do it. It seems that some layers can't be exported to Pb


Fix loads of pycross_lock_file_repo

I need to figure out why these tests are failing. I assume it's something related to github secret access.

Yes, by default a "3rd party" PR run cannot access secrets: https://stackoverflow.com/a/73867674/250826


Reconsider Return Types in Cursor Value Methods

Currently, we have APIs to extract value variants out of the Cursor as IonResult<Option<T>>:
    /// If the current value is an integer, returns its value as an i64; otherwise, returns None.
    fn read_i64(&mut self) -> IonResult<Option<i64>>;

The reasoning is that error cases are handled distinct from type mismatches (or typed null) cases.  The problem with this approach is that the double wrapping makes using this not particularly ergonomic as ? only works with one of Result<T, E> or Option<T>.
I think something we should consider is encoding these modalities of nothing and error into a new error type because both of those cases are more exceptional than the "I expect a value" case:
pub type IonOptionalResult<T> = Result<T, Option<IonError>>

In this case, we move the Option over to the error, and allow dealing with this modality in error handling cases only.  Furthermore, the conversion implementation impl From<Option<IonError>> for IonError where we collapse the None is pretty straightforward.
This would allow things like the following to "just work"::
let x = cursor.read_i64()?

And only in the case where you care about the typing error (and thus not naturally collapse that case to a simple IonResult), you can handle it explicitly:
let x = match cursor.read_i64() {
  Ok(v) => v,
  Err(None) => 0,
  Err(Some(e)) => e?
}

I've put together a rough prototype here to demonstrate how this might work:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=2ae9fc9889be18170da2e21eb18cdf14

A simpler version of this would be to extend the domain of IonError to simply have a value for NoValueForType or some-such, this reduces the extra structure for Option<IonError> which is arguably simpler.  Since the original code didn't do that, I would like to understand the rationale for the Option factoring before we lose that in flattening the result.


Files in sub directory are not ingested when knowledge tool is accessed as remote tool.

Steps to reproduce the problem:

Get k8s docs made available in local directory using git clone https://github.com/kubernetes/website.git
Run the following script to launch knowledge chat agent using workspace that has k8s documentation - gptscript --workspace /Users/sangeethahariharan/k8sdoc/website/content/en/docs knowledge_remote.gpt

tools: github.com/gptscript-ai/knowledge
context: github.com/gptscript-ai/context/workspace
chat: true

You are a helpful assistant that can retrieve information and answer questions using your tools. When you strat just introduce yourself and wait for the user to prompt you.
You must use the knowledge tool to look up answers to your questions. Do NOT use the read tool. You may only answer questions with information you retrieve from the knowledge tool.
When you give answers, always give a proper citation to the best of your abilities.

knowledge tool is not able to provide any relevant information for any search query

Search results are only from the file under the workspace directory and no sub directories are searched.
Expected Behavior:
All files from sub directory should be ingested recursively by knowledge tool.

askdir and ingest are now recursive by default as per #29 - can you check again with a build from the main branch, please?

Tested with knowledge as remote tool and able to see all files from subdirectory getting ingested. Able to get relevant information when chatting with the tool.


Using pika instead of python-amqp

Library changes: using pika instead on python-amqp
Update versions of requirements
New library version: 2.0.0

👍


TS2362 - Error when doing bitwise operations on number[]

[1] & 1

x side of an arithmetic operation must be of type 'any', 'number', 'bigint' or an enum type

Expected : bitwise operations on arrays evaluate against index 0
Playground link

Expected : bitwise operations on arrays evaluate against index 0

🤔
> [1, 1] & 1
< 0

See also https://stackoverflow.com/questions/41750390/what-does-all-legal-javascript-is-legal-typescript-mean

Yeah it only works with arrays of length 1
It is weird and I can't find documentation about it anywhere, yet it is used extensively in sveltejs
Is there a way to define it locally anyway ?


Preload images

[ ] Build a JavaScript preloader which postpones the application load until all images are ready.
[ ] Show and style a progress display for the preloader

I'd strongly recommend adding ember-cli-workbox as a dependency here and preloading the images with that. The current implementation with the <link rel="preload"> tags has significant performance implications and I'd say service workers are kind of the standard solution for preloading anything now.

I was hoping to keep this addon as light and free of dependencies as possible, but I think you're also right.
This issue however is not about prefetching in the background, but actually blocking application init until all assets are loaded. While this is not good behavior for an existing ember app, it is sensible to do in a blank project that exists just as a prototype.

So the service worker will preload once the application has started but you can of course call the service's load in the application route's model to make sure all images are in fact loaded before the app starts up.


Unable to deploy on Filecoin testnets due to "implementation isn't a contract" error

Summary
I'm trying to deploy Hyperlane to Filecoin's testnet wallaby by following these docs, but when I run
forge script scripts/DeployCore.s.sol --broadcast --rpc-url $RPC_URL

I get an error that
## Setting up (1) EVMs.

Failed with `ERC1967: new implementation is not a contract`:

I've attached the full output in logs.txt, and it looks like the deployment code was executed but the EVM setup failed.
Env Vars
LOCAL=wallaby
RPC_URL=https://wallaby.node.glif.io/rpc/v0
REMOTES=goerli

Config
I initialized the keys under contracts as "", and the script filled in the addresses.
  "wallaby": {
    "id": 31415,
    "owner": "0xbcb8E197F783E2aE4B3f3b6358B582a9692f9F85",
    "contracts": {
      "interchainGasPaymaster": "0xBb2180ebd78ce97360503434eD37fcf4a1Df61c3",
      "mailbox": "0x416C42991d05b31E9A6dC209e91AD22b79D87Ae6",
      "proxyAdmin": "0x34A1D3fff3958843C43aD80F30b94c510645C316",
      "testRecipient": "0xd21060559c9beb54fC07aFd6151aDf6cFCDDCAeB",
      "create2Factory": "0x0000000000000000000000000000000000000000"
    }
  }

Has anyone encountered this error before? Am I missing a step in the setup? I also tried to deploy to hyperspace, another Filecoin testnet, but saw the same failure message.

@Yiwen-Gao any chance you'd be free for a call? we'd love to chat, regardless @asaj and @nambrot or others on our team can offer help here with the issue

@Yiwen-Gao thanks for the thorough bug report!
IIUC forge will first dry-run the script using a fork of wallaby state, writing outputs to the config file. If that succeeds, it will then attempt to send the transactions.
As far as I can tell, the dry-run succeeded and the config was updated but the actual on-chain transactions failed, which is strange.
Are you aware of any differences between the wallaby VM and the EVM?

any chance you'd be free for a call?

Sure, I'm happy to answer any additional questions.

Are you aware of any differences between the wallaby VM and the EVM?

I think the Filecoin EVM (FEVM) is only available on testnets wallaby and hyperspace at the moment. Mainnet is running FVM and has its FEVM launch scheduled for March, so there may be edge cases that the Filecoin team hasn't ironed out yet. (See their roadmap.)
However, I have deployed Solidity contracts to wallaby with Remix + Metamask before, so it seems like it should work via other deployment methods too.

@Yiwen-Gao can you faucet me some tokens to 0xE79898D2cd7DC05244eB55ef25fBD5952fA6c45A and share an RPC URL?
I've been unable to repro on Mumbai/Alfajores, but maybe I can on Wallaby?
Fwiw someone else ran into this on a different network, so it may not be a Filecoin edge case

@Yiwen-Gao I suspect you are not including a private key, you must include one. You can see how here: https://book.getfoundry.sh/reference/forge/forge-script#wallet-options---raw

Ah, that was it--I wasn't including my private key. I'm running into another issue with gas limits now, but I should be able to resolve it myself. Thanks for the help!

@Yiwen-Gao hey this is Honghao from FVM team - we are looking into bridging solutions. can we connect to learn about your use case and inputs for filecoin bridging? if you can share your handle on slack I can connect with you there, thanks!

@Yiwen-Gao hey this is Honghao from FVM team - we are looking into bridging solutions. can we connect to learn about your use case and inputs for filecoin bridging? if you can share your handle on slack I can connect with you there, thanks!

Also i would love to chat with you about what solutions you're weighing for FVM. You can reach me on telegram at @palenimbus or email at<EMAIL_ADDRESS>

great thanks! reaching out on telegram

@honghaoq Sure! I just sent you a DM on the Filecoin Slack.


Serialize ValidationRules and Error when output format is Yaml or Json

Description: Errors are not printed when ouput is JSON or YAML format.
Steps to reproduce:

Create an alert rule file with failures (argocd-example.yaml.txt)
Use example/validation.yaml config file
Execute promruval with --output argument with yaml or json

$ ./promruval validate --config-file=$(pwd)/examples/validation.yaml /tmp/rules/argocd.yaml --output json > ../promruval-workspace/output.json
INFO[2024-12-03T14:06:11+01:00] processing file                               file/tmp/rules/argocd.yaml progress=1/1
INFO[2024-12-03T14:06:11+01:00] successfully dumped cache to file             file_name=.promruval_cache.json
INFO[2024-12-03T14:06:11+01:00] cache dumped                                  duration="246.631µs"
jescalona@MALLL-JESCALON1:~/projects/promruval$ ./promruval validate --config-file=/tmp/rules/argocd.yaml --output yaml > ../promruval-workspace/output.yaml
INFO[2024-12-03T14:06:16+01:00] processing file                               file=/tmp/rules/argocd.yaml progress=1/1
INFO[2024-12-03T14:06:16+01:00] successfully dumped cache to file             file_name=.promruval_cache.json
INFO[2024-12-03T14:06:16+01:00] cache dumped                                  duration="313.77µs"

Current results:
Output on YAML or JSON format does not contain Error values, see attached output files:

JSON: output.json
YAML:
output.txt

Expected results:
Output of YAML and JSON format contain Error values, see attached files:

JSON:
output_ok.json

YAML:
output_ok.yaml.txt

Which operating system are you using? Linux Fedora 41 6.11.10-300.fc41.x86_64
go version go1.23.3 linux/amd64
Additional details:
After some research, it seems JSON or YAML Marshal does not work for Error struct: https://github.com/golang/go/issues/5161

Please, feel free to provide feedback about the code, given i'm newbie on golang :)


Set up travis build

This PR sets up a Travis CI pipeline. The pipeline will publish snapshots using the snapshotProject script when the build was not triggered by a Git Tag. Alternatively, it will publish release versions when the build was triggered by a Git Tag.

You can see that new snapshots are being published from this branch using the Travis Config by querying npm from a terminal:
$ npm show testable-artifact-builder dist-tags

{ 'set-up-travis-build': '0.0.1-set-up-travis-build.17',
  latest: '0.0.1-set-up-travis-build.11' }

I believe latest will be updated when a new release version has been published, but that will require some testing on the master branch.


ExternalLogin

code :
public async Task ExternalLoginCallback(string returnUrl = null, string remoteError = null)
{
if (remoteError != null)
{
ModelState.AddModelError(string.Empty, _localizer["ErrorExternalProvider", remoteError]);
            return View(nameof(Login));
        }

        var info = await _signInManager.GetExternalLoginInfoAsync();

I want to use GitHub as a third party login，but this info and  returnUrl  always null,  the github Authorization callback URL I set  "http://localhost:5000/account/ExternalLoginCallback"   if  is error 。
I Use a MVC client to login 。

Try setup Authorization callback URL to http://localhost:5000.

Try setup on Github - Authorization callback URL to http://localhost:5000.
This problem has been solved, but If I enter an existing username and mailbox ,then get an error, I'm not sure if it's a bug.  the new username have no problem .

the error is
NullReferenceException: Object reference not set to an instance of an object.
AspNetCore.Views_Account_ExternalLoginConfirmation.ExecuteAsync() in ExternalLoginConfirmation.cshtml
+
@Localizer["SubTitle", ViewData["LoginProvider"].ToString()]
I think ,Here should be show the error like "User name Or MailBox  already exists"

OK, let me check it. ;)

This is a bug, I will fix it.
Thanks for reporting!

Fixed on dev branch - can you test it please?
Thanks!

固定在dev树枝上 - 你能测试一下吗？
谢谢！

Okay, I'll test and reply to the results.

I have finished the test. This error has been resolved. The hint I got was that the mailbox or username had already been used. It's perfect.I will continue to use it. If there are other errors, I will give you feedback in time. I hope it will get better and better.

Great, I will look forward to your feedback.
Thanks.


⚠ Bofrost.lu has degraded performance

In 640f379, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1105 ms

Resolved: Bofrost.lu performance has improved in c5ee6af.


Hadhunna/entra/sample

Add a folder for samples.

Added Sample for Mapping Claims

API change check
API changes are not detected in this pull request.


otbr-agent fails to connect to nRF52840 dongle due to RadioSpinelNoResponse

This is my first foray into Thread/OpenThread, so please forgive possible confusions.
I've had a handful of nRF52840 dongles laying around (v1.2.0) and decided to give OT a quick go. Followed the instructions to set up the host service on a Raspberry Pi 4, which went without issues. Then followed this repo's instructions as well, to create and flash the appropriate firmware for the dongle, which was also successful.
The problem happens when I try to start otbr-agent - even though the proper TTY is selected, pretty soon after starting up, throws the error in the title. I've tried using different USB ports, swapping power supplies, to no avail. Same thing happens if I try to run the OTBR stack in Docker.
Steps to reproduce:

Build nRF52840 dongle RCP:

cd ~/ot-nrf528xx
./script/build nrf52840 USB_trans -DOT_BOOTLOADER=USB

Flash RCP firmware to dongle:

arm-none-eabi-objcopy -O ihex build/bin/ot-cli-ftd ot-cli-ftd.hex
nrfutil pkg generate --hw-version 52 --sd-req 0x00 --application-version 1 --application ot-cli-ftd.hex app_dfu_package.zip
nrfutil dfu usb-serial -pkg app_dfu_package.zip -p /dev/ttyACM0

Start OTBR in Docker:

docker run --sysctl "net.ipv6.conf.all.disable_ipv6=0 net.ipv4.conf.all.forwarding=1 net.ipv6.conf.all.forwarding=1" -p 8080:80 --dns=<IP_ADDRESS> -it --volume /dev/ttyACM0:/dev/ttyACM0 --privileged openthread/otbr --radio-url spinel+hdlc+uart:///dev/ttyACM0

Result:
Oct  4 15:08:41 8d069c6d2888 otbr-agent[217]: [INFO]-UTILS---: Running 0.3.0-2af5cbd
Oct  4 15:08:41 8d069c6d2888 otbr-agent[217]: [INFO]-UTILS---: Thread version: 1.2.0
Oct  4 15:08:41 8d069c6d2888 otbr-agent[217]: [INFO]-UTILS---: Thread interface: wpan0
Oct  4 15:08:41 8d069c6d2888 otbr-agent[217]: [INFO]-UTILS---: Backbone interface: eth0
Oct  4 15:08:41 8d069c6d2888 otbr-agent[217]: [INFO]-UTILS---: Radio URL: spinel+hdlc+uart:///dev/ttyACM0
Oct  4 15:08:41 8d069c6d2888 otbr-agent[217]: [WARN]-PLAT----: Error processing notification: Parse
Oct  4 15:08:41 8d069c6d2888 otbr-web[245]: [INFO]-WEB-----: Running 0.3.0-2af5cbd
Oct  4 15:08:41 8d069c6d2888 otbr-web[245]: [INFO]-WEB-----: Border router web started on wpan0
Oct  4 15:08:41 8d069c6d2888 otbr-web[245]: [ERR ]-WEB-----: OpenThread daemon is not running.
Oct  4 15:08:41 8d069c6d2888 otbr-agent[217]: [WARN]-PLAT----: Error processing notification: Parse
Oct  4 15:08:43 8d069c6d2888 otbr-agent[217]: [CRIT]-PLAT----: HandleRcpTimeout() at ../../third_party/openthread/repo/src/lib/spinel/radio_spinel_impl.hpp:2217: RadioSpinelNoResponse

The response is very similar to #71, however the solution won't work for me - the nRF52840 dongle does not have a J-Link OB debugger, and as such, it does not have MSD.

Hi @fonix232, thanks for reporting this. Can you please state which commits are you using for openthread and ot-nrf528xx repos?

hi,
same thing here. i am on:
Submodule path 'openthread': checked out '02d29cd5e343c3eb17840251b7feaa9345892895'
and:
commit 8853d0e9d1ce9471959dcfcf803beb49fae84030 (HEAD -> main, origin/main, origin/HEAD)
Author: dependabot[bot]<EMAIL_ADDRESS>Date:   Wed Nov 24 08:20:05 2021 -0800
submodule: bump openthread from `db9e430` to `02d29cd` (#203)

Bumps [openthread](https://github.com/openthread/openthread) from `db9e430` to `02d29cd`.
- [Commits](https://github.com/openthread/openthread/compare/db9e430876a50b179dd4a20f5159a4d77dfc81bb...02d29cd5e343c3eb17840251b7feaa9345892895)

recipes found on net are wrong:
don't flash the cli, flash ot-rcp instead!
arm-none-eabi-objcopy -O ihex build/bin/ot-cli-ftd ot-cli-ftd.hex needs to be
arm-none-eabi-objcopy -O ihex build/bin/ot-rcp ot-rcp.hex instead!

recipes found on net are wrong: don't flash the cli, flash ot-rcp instead! arm-none-eabi-objcopy -O ihex build/bin/ot-cli-ftd ot-cli-ftd.hex needs to be arm-none-eabi-objcopy -O ihex build/bin/ot-rcp ot-rcp.hex instead!

@markus61 , can you point to the instructions you used so that we can make necessary updates?

@jwhui , the confusion might come from the instructions at this link. Adding a few lines there to point out which build to use for which case might be useful, especially for those new to OpenThread.

I'm trying to do the same on my side. It seems like there is a lot of conflicting information right now with a mix of legacy and current stuff.
I think the best/current documentation for this right now is this:
https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/matter/openthread_rcp_nrf_dongle.html
Though I have yet to test it.

I'm here to report, the documentation mentioned in my previous post worked fine on my end.

Closing stale issue.


E1048: Item not found in script

Functions don't work.
When the command is executed, showing the following message.
Error detected while compiling function fuzzy#ag#Start:
line   34:
E1048: Item not found in script: SetPrompt

or
Error detected while compiling function fuzzy#files#Start:
line   18:
E1048: Item not found in script: split_edit_callbacks
Press ENTER or type command to continue

Hi. I saw you pull request. However, isn't the changed path same as the original one(eg. 'utils/selector.vim' and '../utils/selector.vim')? Could you explain to me what is the differences or why the latter fix the problem?

#28

Also I could not reproduce this bug in my machine. Could you tell me what OS you ran into this bug?

I just upgraded my vim into version 9.1.496.
It fixed the bug.
I thought it was bug from vim, not this plugin.
Thank you for your attention.


Save failing after broken alignment is fixed

Open gE, edit a scripture and save.
Note that Broken alignment link is activated, Click on the icon to open the alignment window.
Complete the alignment and click on Accept.
Note that Save button is enabled, click on Save.
Fails with error on console

Looks good, not seeing any errors after saving in console. Tested with v2.0.0 build 87aa21b QA.

Not seeing the problem in v2.0.0 build 87aa21b QA. Save is working fine after the edits and alignments for large books.

@elsylambert @danielklapp - Added error message for when branch fails to create. Latest fix is in v2.0.0 build 954c4d3

@elsylambert @danielklapp - Added error message for when branch fails to create in a resource card. Latest fix is in v2.0.0 build 78fc9be

@PhotoNomad0 Seeing this Network error when trying to save with a custom test slower connection.

Passing this issue in v2.0.0 build 78fc9be QA. I see the network error the first time user branch is created with slower connections but then afterwards it works fine. Able to edit and save scripture edits and alignments without error.


Application options throw errors: skipUserManagement and devDatabaseType h2Disk

The issue

Two application options throw errors:

skipUserManagement true

error: An error has occurred:
	TypeError
error: Error message:
	skipUserManagement.join is not a function
DEBUG! Error:
TypeError: Cannot read property 'start' of undefined
    at parse (/home/berty/.config/yarn/global/node_modules/jhipster-core/lib/reader/jdl_reader.js:47:55)
    at Object.parseFromFiles (/home/berty/.config/yarn/global/node_modules/jhipster-core/lib/reader/jdl_reader.js:73:10)
    at module.exports.parseJDL (/home/berty/.config/yarn/global/node_modules/generator-jhipster/generators/import-jdl/index.js:104:44)
    at Object.<anonymous> (/home/berty/.config/yarn/global/node_modules/yeoman-generator/lib/index.js:399:25)
    at /home/berty/.config/yarn/global/node_modules/run-async/index.js:25:25
    at new Promise (<anonymous>)
    at /home/berty/.config/yarn/global/node_modules/run-async/index.js:24:19
    at self.env.runLoop.add.completed (/home/berty/.config/yarn/global/node_modules/yeoman-generator/lib/index.js:400:11)
    at runCallback (timers.js:794:20)
    at tryOnImmediate (timers.js:752:5)
    at processImmediate [as _immediateCallback] (timers.js:729:5)
TypeError: Cannot read property 'start' of undefined
Error jhipster:import-jdl jdl.jh --debug

Error: ERROR! Error while parsing applications and entities from the JDL TypeError: Cannot read property 'start' of undefined
    at Environment.error (/home/berty/.config/yarn/global/node_modules/yeoman-environment/lib/environment.js:157:40)
    at module.exports.error (/home/berty/.config/yarn/global/node_modules/generator-jhipster/generators/generator-base.js:1970:18)
    at module.exports.parseJDL (/home/berty/.config/yarn/global/node_modules/generator-jhipster/generators/import-jdl/index.js:155:26)
    at Object.<anonymous> (/home/berty/.config/yarn/global/node_modules/yeoman-generator/lib/index.js:399:25)
    at /home/berty/.config/yarn/global/node_modules/run-async/index.js:25:25
    at new Promise (<anonymous>)
    at /home/berty/.config/yarn/global/node_modules/run-async/index.js:24:19
    at self.env.runLoop.add.completed (/home/berty/.config/yarn/global/node_modules/yeoman-generator/lib/index.js:400:11)
    at runCallback (timers.js:794:20)
    at tryOnImmediate (timers.js:752:5)
    at processImmediate [as _immediateCallback] (timers.js:729:5)

devDatabaseType h2Disk

error: An error has occurred:
	SyntaxError
error: Error message:
	Expected ",", "/*", "applicationType", "authenticationType", "baseName", "buildTool", "cacheProvider", "clientFramework", "clientPackageManager", "databaseType", "devDatabaseType", "enableHibernateCache", "enableSwaggerCodegen", "enableTranslation", "jhiPrefix", "jhipsterVersion", "languages", "messageBroker", "nativeLanguage", "packageName", "path", "prodDatabaseType", "searchEngine", "serverPort", "serviceDiscoveryType", "skipClient", "skipServer", "skipUserManagement", "testFrameworks", "uaaBaseName", "useSass", "websocket", "}", [A-Za-z], or [\n\t\r

] but "2" found.
error: Position:
	At l.17.
DEBUG! Error:
SyntaxError: SyntaxError: Expected ",", "/*", "applicationType", "authenticationType", "baseName", "buildTool", "cacheProvider", "clientFramework", "clientPackageManager", "databaseType", "devDatabaseType", "enableHibernateCache", "enableSwaggerCodegen", "enableTranslation", "jhiPrefix", "jhipsterVersion", "languages", "messageBroker", "nativeLanguage", "packageName", "path", "prodDatabaseType", "searchEngine", "serverPort", "serviceDiscoveryType", "skipClient", "skipServer", "skipUserManagement", "testFrameworks", "uaaBaseName", "useSass", "websocket", "}", [A-Za-z], or [\n\t\r

] but "2" found.
    at parse (/home/berty/.config/yarn/global/node_modules/jhipster-core/lib/reader/jdl_reader.js:48:11)
    at Object.parseFromFiles (/home/berty/.config/yarn/global/node_modules/jhipster-core/lib/reader/jdl_reader.js:73:10)
    at module.exports.parseJDL (/home/berty/.config/yarn/global/node_modules/generator-jhipster/generators/import-jdl/index.js:104:44)
    at Object.<anonymous> (/home/berty/.config/yarn/global/node_modules/yeoman-generator/lib/index.js:399:25)
    at /home/berty/.config/yarn/global/node_modules/run-async/index.js:25:25
    at new Promise (<anonymous>)
    at /home/berty/.config/yarn/global/node_modules/run-async/index.js:24:19
    at self.env.runLoop.add.completed (/home/berty/.config/yarn/global/node_modules/yeoman-generator/lib/index.js:400:11)
    at runCallback (timers.js:794:20)
    at tryOnImmediate (timers.js:752:5)
    at processImmediate [as _immediateCallback] (timers.js:729:5)
SyntaxError: SyntaxError: Expected ",", "/*", "applicationType", "authenticationType", "baseName", "buildTool", "cacheProvider", "clientFramework", "clientPackageManager", "databaseType", "devDatabaseType", "enableHibernateCache", "enableSwaggerCodegen", "enableTranslation", "jhiPrefix", "jhipsterVersion", "languages", "messageBroker", "nativeLanguage", "packageName", "path", "prodDatabaseType", "searchEngine", "serverPort", "serviceDiscoveryType", "skipClient", "skipServer", "skipUserManagement", "testFrameworks", "uaaBaseName", "useSass", "websocket", "}", [A-Za-z], or [\n\t\r

] but "2" found.

Motivation for or Use Case

These options should not throw errors

Reproduce the error

add these two options in a JDL and run import-jdl

Related issues

Suggest a Fix

The JDL file
// Applications

application {
 config {
  applicationType uaa,
  authenticationType uaa,
baseName uaa,
buildTool maven,
cacheProvider hazelcast,
databaseType sql,
devDatabaseType h2Disk,
enableHibernateCache true,
enableSwaggerCodegen false,
enableTranslation true,
jhiPrefix jhi,
jhipsterVersion 4.14.3,
languages [en, fr],
messageBroker false,
nativeLanguage en,
packageName com.thalesgroup.tce.ecwp.uaa,
prodDatabaseType mysql,
searchEngine false,
serverPort 9999,
serviceDiscoveryType eureka,
skipUserManagement true,
skipClient true,
skipServer false,
testFrameworks [cucumber],
uaaBaseName "../uaa",
useSass false,
websocket false
	}
}

Versions

JHipster

5.0.0-beta.1

[x] Checking this box is mandatory (this is just to show you read everything)

It's absolutely not supported to generate an UAA without user management.
The main reason to use an UAA is to combine JHipster user management with Spring @enableauthserver. If you remove user management from the UAA then nothing will work properly.
Eventually we could introduce a check that prevents you from choosing both options with an error message...

Hi Pierre,
Thanks for the reponse. Sorry, I tried to be concise by uniting all options under the uaa but didn't double check if that had an impact in the application context.
I edited the sample JDL file , to show all applications, with appropriate options.

Hi, thanks for the report.Pierre's solution is the way to go.I'll try to do it today.


black background in month view

Hi,
I got an black background issue in month view, but only happen January 2017 and December 2017, please see the screenshot, could you have any idea? Thanks.

can you reproduce with the sample or provide an example activity that reproduce your issue?


[PROD] map org update

@lukaszgryglicki  update as just merged in test

1st run:


implementation of local e2e-tests

Based on Antonio's PR, this PR is introducing local e2e tests. This is also supposed to provide a base to build CI from, so that local tests and CI tests are equivalent.
To tests this, run
./hack/test_e2e.sh

@nehaLohia27 let's take a look at the skipped tests. Maybe we should make them configurable via args we pass to the script.

@FriedrichWilken Yess we can take a input for the skipped flags. but for now we can keep it common for all backends. once we have all features implemented then we can add this via args.. what do you think ?

is this ready to merg?

@FriedrichWilken can you squash ?

/tide help

/label tide/merge-method-squash

/lgtm
/approve

/lgtm
/approve

@nehaLohia27  can you
/lgtm
/approve

looks like this is working now

/lgtm
/approve

Currently the tests are run for every PR and since we haven't covered all testcases in any of the backends, the tests would fail and we would not be able to merge any change. Can we make tests for PR's optional for now and also have an option to have builds periodically or trigger build manually or as @hanamantagoudvk suggested as a trigger from github actions?

My suggestion is we can run two types of jobs/tests if its feasible with given infra we have.
a) Cron Job : To run nightly test suites which will have elaborate coverage.
But this might be difficult to setup and might take more time.
We can explore if its possible at all.
b) Backend specific suites : These backend specific suites can be coupled with keywords.
Whenever developer raises PRs , they can use these keywords and only specific suite is run.
With this approach , it will be easy to merge the PRs.
Currently all backend suites are running , and hence blocking merger of every PR.
Option - (b) is possible  as its already being done in k8s .

/lgtm

/lgtm


Error with makeFilePathAbsolute

Description
Environment:
Vite 3
Windows 11 WSL2 docker node:18.12-alpine3.16
Happened after updating from 0.4.77 to 0.4.93
pinning 0.4.77 is working fine
Error Message + Error Stack
> vite -c config/vavite.config.ts
error when starting dev server:
Error<EMAIL_ADDRESS>You stumbled upon a bug in vite-plugin-ssr's source code. Reach out at https://github.com/brillout/vite-plugin-ssr/issues/new or https://discord.com/invite/qTq92FQzKb and include this error stack (the error stack is usually enough to fix the problem). A maintainer will fix the bug (usually under 24 hours). Don't hesitate to reach out as it makes vite-plugin-ssr more robust.
    at makeFilePathAbsolute (/src/node_modules/.pnpm/vite-plugin-ssr@0.4.93_ima47bflrjqvy2qu33dh2efdbq/node_modules/vite-plugin-ssr/dist/cjs/utils/makeFilePathAbsolute.js:20:25)
    at /src/node_modules/.pnpm/vite-plugin-ssr@0.4.93_ima47bflrjqvy2qu33dh2efdbq/node_modules/vite-plugin-ssr/dist/cjs/node/plugin/plugins/devConfig/determineOptimizeDepsEntries.js:25:60
    at Array.forEach (<anonymous>)
    at determineOptimizeDepsEntries (/src/node_modules/.pnpm/vite-plugin-ssr@0.4.93_ima47bflrjqvy2qu33dh2efdbq/node_modules/vite-plugin-ssr/dist/cjs/node/plugin/plugins/devConfig/determineOptimizeDepsEntries.js:24:19)
    at async configResolved (/src/node_modules/.pnpm/vite-plugin-ssr@0.4.93_ima47bflrjqvy2qu33dh2efdbq/node_modules/vite-plugin-ssr/dist/cjs/node/plugin/plugins/devConfig.js:49:48)
    at async Promise.all (index 9)
    at async resolveConfig (file:///src/node_modules/.pnpm/vite@3.2.5_@types+node@18.15.2/node_modules/vite/dist/node/chunks/dep-5605cfa4.js:62878:5)
    at async createServer (file:///src/node_modules/.pnpm/vite@3.2.5_@types+node@18.15.2/node_modules/vite/dist/node/chunks/dep-5605cfa4.js:61943:20)
    at async CAC.<anonymous> (file:///src/node_modules/.pnpm/vite@3.2.5_@types+node@18.15.2/node_modules/vite/dist/node/cli.js:707:24)

I See what’s going on. I’ll Release a fix.
On Mon 13. Mar 2023 at 19:43, nitedani @.***> wrote:

Description
Environment:
Windows 11 WSL2 docker node:18.12-alpine3.16
Happened after updating from 0.4.77 to 0.4.93
pinning 0.4.77 is working fine
Error Message + Error Stack

vite -c config/vavite.config.ts
error when starting dev server:
Error: @.[Bug] You stumbled upon a bug in vite-plugin-ssr's source code. Reach out at https://github.com/brillout/vite-plugin-ssr/issues/new or https://discord.com/invite/qTq92FQzKb and include this error stack (the error stack is usually enough to fix the problem). A maintainer will fix the bug (usually under 24 hours). Don't hesitate to reach out as it makes vite-plugin-ssr more robust.
at makeFilePathAbsolute @._ima47bflrjqvy2qu33dh2efdbq/node_modules/vite-plugin-ssr/dist/cjs/utils/makeFilePathAbsolute.js:20:25)
at @._ima47bflrjqvy2qu33dh2efdbq/node_modules/vite-plugin-ssr/dist/cjs/node/plugin/plugins/devConfig/determineOptimizeDepsEntries.js:25:60
at Array.forEach ()
at determineOptimizeDepsEntries @._ima47bflrjqvy2qu33dh2efdbq/node_modules/vite-plugin-ssr/dist/cjs/node/plugin/plugins/devConfig/determineOptimizeDepsEntries.js:24:19)
at async configResolved @._ima47bflrjqvy2qu33dh2efdbq/node_modules/vite-plugin-ssr/dist/cjs/node/plugin/plugins/devConfig.js:49:48)
at async Promise.all (index 9)
at async resolveConfig @.@.@./node_modules/vite/dist/node/chunks/dep-5605cfa4.js:62878:5)
at async createServer @.@.@./node_modules/vite/dist/node/chunks/dep-5605cfa4.js:61943:20)
at async CAC. @.@.@./node_modules/vite/dist/node/cli.js:707:24)

—
Reply to this email directly, view it on GitHub
https://github.com/brillout/vite-plugin-ssr/issues/703, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAHVQRVKZPIL4JHU6G5LAC3W35TGJANCNFSM6AAAAAAVZOXKVQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Fix released in 0.4.94.


Minify our post-install program

I removed minification because it was getting bad character encoding, spewing garbage and making a terminal state corrupt.
This is the commit to revert https://github.com/aspect-build/rules_js/commit/4794f1f40ce18b53c432db037c05c0c3e263f2ad
<EMAIL_ADDRESS>is the package to add a unit test - it calls node-gyp.
Probably there's some file encoding problem in ncc, or maybe we should run terser on the file outside of ncc to get more control.

In particular, either terser or ncc (which I think just calls terser) produces
let nonASCIIidentifierStartChars="ªµºÀ-ÖØ-öø-ˁˆ-ˑˠ-ˤˬˮͰ-ʹͶͷͺ-ͽͿΆΈ-ΊΌΎ-ΡΣ-ϵϷ-ҁҊ-ԯԱ-Ֆՙՠ...
where the original program has
let nonASCIIidentifierStartChars = "\xaa\xb5\xba\xc0-\xd6\xd8-\xf6\xf8-...

already landed


aws_securityhub_finding table returns incorrect account_id

Describe the bug
The account_id represents the account_id of the finding, not of the account where the finding was generated.
See: https://docs.aws.amazon.com/securityhub/1.0/APIReference/API_AwsSecurityFinding.html#securityhub-Type-AwsSecurityFinding-AwsAccountId
Steampipe version (steampipe -v)
0.19.5
Plugin version (steampipe plugin list)
AWS 0.101.0
To reproduce
This issue requires Security Hub Delegated Administrator to be setup. When multiple accounts are streaming findings to a single security account, the AwsAccountId represents the account where the finding was generated, not the account that is being queried.
Expected behavior
account_id should reflect the value indicated by the AWS API.
Additional context
The id field for a finding in a delegated admin scenario is an arn with the correct ID. If I wanted the the account I queries I'd use _ctx. Otherwise account_id should represent the account from the id field.

Hey @jchrisfarris , in general, we usually keep the account_id reserved for the calling account ID, as this is a common column in all of our tables.
To get the finding's account ID, I believe @gabrielsoltz has opened a PR to address this, https://github.com/turbot/steampipe-plugin-aws/pull/1703. We're leaning toward calling it source_account_id, as AWS uses the term source in some other API calls for matching account IDs like this.
Would this PR help your use case?

yeah, source_account_id would be acceptable to deconflict with the plugin standard columns. I looked in the issues, but not the Open PRs.  I'll watch that PR and close this issue. Thanks!