Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath = DLLpath, ...):

Describe the bug, including details regarding any error messages, version, and platform.
Hi there - I am trying to install the R package arrow on my Mac OS M1.
I have tried to re-install the apache-arrow library with home brew and multiple other solution I read from different github and stackoverflow issue but it still does not solve the issue.
Could you help me out there?
Here is the log below:
install.packages("arrow")
Installing package into ‘/opt/homebrew/lib/R/4.3/site-library’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/arrow_<IP_ADDRESS>.tar.gz'
Content type 'application/x-gzip' length 4263864 bytes (4.1 MB)
==================================================
downloaded 4.1 MB

* installing *source* package ‘arrow’ ...
** package ‘arrow’ successfully unpacked and MD5 sums checked
** using staged installation
*** pkg-config found.

**** Printing contents of build log because the build failed while ARROW_R_DEV was set to FALSE
++ pwd
+ : /private/var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T/RtmpaBXKm8/remotes1351f3d741409/arrow
+ : tools/cpp
+ : /var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T//RtmpjgVixX/file138d4f47f4aa
+ : libarrow/arrow-14.0.0
+ : /var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T//RtmpjgVixX/file138d4449acdf6/cmake-3.26.4-macos-universal/bin/cmake
++ cd tools/cpp
++ pwd
+ SOURCE_DIR=/private/var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T/RtmpaBXKm8/remotes1351f3d741409/arrow/tools/cpp
++ mkdir -p libarrow/arrow-14.0.0
++ cd libarrow/arrow-14.0.0
++ pwd
+ DEST_DIR=/private/var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T/RtmpaBXKm8/remotes1351f3d741409/arrow/libarrow/arrow-14.0.0
+ '[' 2 = '' ']'
+ '[' false '!=' '' ']'
++ echo false
++ tr '[:upper:]' '[:lower:]'
+ LIBARROW_MINIMAL=false
+ '[' false = false ']'
+ ARROW_DEFAULT_PARAM=ON
+ case "$CXX" in
+ mkdir -p /var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T//RtmpjgVixX/file138d4f47f4aa
+ pushd /var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T//RtmpjgVixX/file138d4f47f4aa
/var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T/RtmpjgVixX/file138d4f47f4aa /private/var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T/RtmpaBXKm8/remotes1351f3d741409/arrow
+ /var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T//RtmpjgVixX/file138d4449acdf6/cmake-3.26.4-macos-universal/bin/cmake -DARROW_BOOST_USE_SHARED=OFF -DARROW_BUILD_TESTS=OFF -DARROW_BUILD_SHARED=OFF -DARROW_BUILD_STATIC=ON -DARROW_ACERO=ON -DARROW_COMPUTE=ON -DARROW_CSV=ON -DARROW_DATASET=ON -DARROW_DEPENDENCY_SOURCE=BUNDLED -DAWSSDK_SOURCE= -DARROW_FILESYSTEM=ON -DARROW_GCS=OFF -DARROW_JEMALLOC=ON -DARROW_MIMALLOC=ON -DARROW_JSON=ON -DARROW_PARQUET=ON -DARROW_S3=OFF -DARROW_WITH_BROTLI=ON -DARROW_WITH_BZ2=ON -DARROW_WITH_LZ4=ON -DARROW_WITH_RE2=ON -DARROW_WITH_SNAPPY=ON -DARROW_WITH_UTF8PROC=ON -DARROW_WITH_ZLIB=ON -DARROW_WITH_ZSTD=ON -DARROW_VERBOSE_THIRDPARTY_BUILD=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_FIND_DEBUG_MODE=OFF -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_PREFIX=/private/var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T/RtmpaBXKm8/remotes1351f3d741409/arrow/libarrow/arrow-14.0.0 -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON -DCMAKE_UNITY_BUILD=OFF -Dxsimd_SOURCE= -Dzstd_SOURCE= -G 'Unix Makefiles' /private/var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T/RtmpaBXKm8/remotes1351f3d741409/arrow/tools/cpp
inst/build_arrow_static.sh: line 67: /var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T//RtmpjgVixX/file138d4449acdf6/cmake-3.26.4-macos-universal/bin/cmake: No such file or directory
**** Complete build log may still be present at /var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T//RtmpjgVixX/file138d4584a69f5.log
Warning messages:
1: In rep(yes, length.out = len) : 'x' is NULL so the result will be NULL
2: In system(cmake_cmd, ignore.stdout = TRUE, ignore.stderr = TRUE) :
  error in running command
------------------------- NOTE ---------------------------
There was an issue preparing the Arrow C++ libraries.
See https://arrow.apache.org/docs/r/articles/install.html
----------------------------------------------------------

Test compile error: <stdin>:1:10: fatal error: 'arrow/api.h' file not found
#include <arrow/api.h>
         ^~~~~~~~~~~~~
1 error generated.
Failing compile command: clang++ -arch arm64 -E -I/opt/R/arm64/include  -falign-functions=64 -Wall -g -O2 -std=gnu++17 -xc++ -
PKG_CFLAGS=
PKG_LIBS=
ERROR: configuration failed for package ‘arrow’
* removing ‘/Users/psadil/git/arrow/renv/library/R-4.2/aarch64-apple-darwin20/arrow’
* restoring previous ‘/Users/psadil/git/arrow/renv/library/R-4.2/aarch64-apple-darwin20/arrow’
Warning message:
In i.p(...) :
  installation of package ‘/var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T//RtmpaBXKm8/remotes1351f3d741409/arrow’ had non-zero exit status

Created on 2023-12-12 with reprex v2.0.2
<IP_ADDRESS> (works)
remotes::install_version('arrow', "<IP_ADDRESS>")
#> Downloading package from url: https://packagemanager.posit.co/cran/latest/src/contrib/Archive/arrow/arrow_<IP_ADDRESS>.tar.gz
#> Installing package into '/Users/psadil/git/arrow/renv/library/R-4.2/aarch64-apple-darwin20'
#> (as 'lib' is unspecified)

Created on 2023-12-12 with reprex v2.0.2

Session info

sessionInfo()
#> R version 4.2.3 (2023-03-15)
#> Platform: aarch64-apple-darwin20 (64-bit)
#> Running under: macOS 14.1.1
#>
#> Matrix products: default
#> BLAS:   /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRblas.0.dylib
#> LAPACK: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib
#>
#> locale:
#> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
#>
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base
#>
#> loaded via a namespace (and not attached):
#>  [1] digest_0.6.33     withr_2.5.2       R.methodsS3_1.8.2 lifecycle_1.0.4
#>  [5] magrittr_2.0.3    reprex_2.0.2      evaluate_0.23     rlang_1.1.2
#>  [9] cli_3.6.2         remotes_<IP_ADDRESS>   rstudioapi_0.15.0 fs_1.6.3
#> [13] R.utils_2.12.3    R.oo_1.25.0       vctrs_0.6.5       styler_1.10.2
#> [17] rmarkdown_2.25    tools_4.2.3       R.cache_0.16.0    glue_1.6.2
#> [21] purrr_1.0.2       xfun_0.41         yaml_2.3.8        fastmap_1.1.1
#> [25] compiler_4.2.3    htmltools_0.5.7   knitr_1.45

@psadil using the package version <IP_ADDRESS> worked for me as well! It did the trick!
I don't know how I did not think about installing an other version before, probably the killing overload before the christmas break killed me ! 🤦♂ 😶🌫
Thanks for the quick response, sharing and support @psadil !!
However, I will let the issue open to get the apache-arrow support team be aware about the issue with the package version > <IP_ADDRESS>!

cc @assignUser @paleolimbot @thisisnic

hm this reads like maybe the precompiled binaries where build without s3/gcs but the package was (correctly) build with them enabled? I don't have a mac so I will have to look at the binary but maybe @paleolimbot can test it? (which we did prior to release so it's weird that it's not working now...)

I was working with renv and had the package source set to the Posit Package Manager. Testing different sources, this is what I see:

from CRAN, the <IP_ADDRESS> binary loads (but versions later than <IP_ADDRESS> do not build when  type = "source", as with remotes::install_version)
from PPM, the <IP_ADDRESS> binary does not load (and versions later than <IP_ADDRESS> do not build when type = "source")

CRAN <- "https://cloud.r-project.org/"
PPM <- "https://packagemanager.posit.co/cran/latest"

# fails -----
install.packages("arrow", repos = PPM)
#> Installing package into '/Users/psadil/git/arrow/renv/library/R-4.3/aarch64-apple-darwin20'
#> (as 'lib' is unspecified)
#>
#> The downloaded binary packages are in
#>  /var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T//RtmpyLusWt/downloaded_packages
callr::r(\(x) library("arrow"))
#> Error: ! in callr subprocess.
#> Caused by error:
#> ! package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath = DLLpath, ...):
#>  unable to load shared object '/Users/psadil/git/arrow/renv/library/R-4.3/aarch64-apple-darwin20/arrow/libs/arrow.so':
#>   dlopen(/Users/psadil/git/arrow/renv/library/R-4.3/aarch64-apple-darwin20/arrow/libs/arrow.so, 0x0006): symbol not found in flat namespace '__ZN5arrow12ArrayBuilder12AppendScalarERKNS_6ScalarEx'

# fails -----
remotes::install_version("arrow", "<IP_ADDRESS>", repos = PPM)
#> Downloading package from url: https://packagemanager.posit.co/cran/latest/src/contrib/arrow_<IP_ADDRESS>.tar.gz
#> Installing package into '/Users/psadil/git/arrow/renv/library/R-4.3/aarch64-apple-darwin20'
#> (as 'lib' is unspecified)
#> Warning in i.p(...): installation of package
#> '/var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T//RtmpyLusWt/remotes1cf77b6510ae/arrow'
#> had non-zero exit status
remotes::install_version("arrow", "<IP_ADDRESS>", repos = CRAN)
#> Downloading package from url: https://cloud.r-project.org/src/contrib/arrow_<IP_ADDRESS>.tar.gz
#> Installing package into '/Users/psadil/git/arrow/renv/library/R-4.3/aarch64-apple-darwin20'
#> (as 'lib' is unspecified)
#> Warning in i.p(...): installation of package
#> '/var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T//RtmpyLusWt/remotes1cf7134a78d1/arrow'
#> had non-zero exit status

# succeeds -----
install.packages("arrow", repos = CRAN)
#> Installing package into '/Users/psadil/git/arrow/renv/library/R-4.3/aarch64-apple-darwin20'
#> (as 'lib' is unspecified)
#>
#> The downloaded binary packages are in
#>  /var/folders/v_/kcpb096s1m3_37ctfd2sp2xm0000gn/T//RtmpyLusWt/downloaded_packages
callr::r(\(x) library("arrow"))
#> [1] "arrow"     "stats"     "graphics"  "grDevices" "datasets"  "utils"
#> [7] "methods"   "base"

remotes::install_version("arrow", "<IP_ADDRESS>", repos = PPM)
#> Downloading package from url: https://packagemanager.posit.co/cran/latest/src/contrib/Archive/arrow/arrow_<IP_ADDRESS>.tar.gz
#> Installing package into '/Users/psadil/git/arrow/renv/library/R-4.3/aarch64-apple-darwin20'
#> (as 'lib' is unspecified)

Created on 2023-12-13 with reprex v2.0.2

Session info

sessionInfo()
#> R version 4.3.2 (2023-10-31)
#> Platform: aarch64-apple-darwin20 (64-bit)
#> Running under: macOS Sonoma 14.1.1
#>
#> Matrix products: default
#> BLAS:   /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRblas.0.dylib
#> LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0
#>
#> locale:
#> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
#>
#> time zone: America/New_York
#> tzcode source: internal
#>
#> attached base packages:
#> [1] stats     graphics  grDevices datasets  utils     methods   base
#>
#> loaded via a namespace (and not attached):
#>  [1] vctrs_0.6.5       cli_3.6.2         knitr_1.45        rlang_1.1.2
#>  [5] xfun_0.41         processx_3.8.3    purrr_1.0.2       styler_1.10.2
#>  [9] renv_1.0.3        glue_1.6.2        htmltools_0.5.7   ps_1.7.5
#> [13] rmarkdown_2.25    R.cache_0.16.0    evaluate_0.23     fastmap_1.1.1
#> [17] yaml_2.3.8        lifecycle_1.0.4   compiler_4.3.2    fs_1.6.3
#> [21] rstudioapi_0.15.0 R.oo_1.25.0       R.utils_2.12.3    digest_0.6.33
#> [25] R6_2.5.1          reprex_2.0.2      callr_3.7.3       magrittr_2.0.3
#> [29] R.methodsS3_1.8.2 tools_4.3.2       withr_2.5.2       remotes_<IP_ADDRESS>

I can repro on github actions:
https://github.com/assignUser/test-repo-a/actions/runs/7202352843/job/19620310001#step:3:96

The non-working PPM binary is caused by the issue reported by Jeroen in #38855 where the host system is used to download the pre-compiled binary which is of course not suiteable for the target system (mac):
arrow_cran.so: Mach-O 64-bit x86_64 dynamically linked shared library, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|WEAK_DEFINES|BINDS_TO_WEAK|NO_REEXPORTED_DYLIBS|HAS_TLV_DESCRIPTORS>
arrow_ppm.so: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, BuildID[sha1]=efa1a6bb9cc2686f83c1e21b3e6ded56cf73c7b5, stripped

The non working source build is likely #38777 which I still have to investigate

I've been running into this issue on my M2 Mac as well when I use renv, and have been using <IP_ADDRESS> for the time being. Just wanted to confirm that I was able to install and use <IP_ADDRESS> using CRAN. In the console I ran:
renv::install("arrow", repos = "https://cloud.r-project.org/", rebuild = TRUE)
and the package loaded normally afterwards.

We believe we've now fixed this for arrow <IP_ADDRESS> in Package Manager. Older arrow versions with the issue haven't been rebuilt yet, but we'll get those fixed soon as well.
@sj-io If you're able to reinstall that arrow binary from Package Manager, could you provide the output of running packageDescription("arrow") in R?

@glin I believe it's working. I removed the package and uninstalled/reinstalled a couple different ways (including using repos = "https://packagemanager.posit.co/cran/latest", a regular install with no conditions, and installing inside and outside of an renv package) and didn't run into any issues when I tried to render pages that use arrow.
> packageDescription("arrow")
Package: arrow
Title: Integration to 'Apache' 'Arrow'
Version: <IP_ADDRESS>
Authors@R: c( person("Neal", "Richardson", email =
<EMAIL_ADDRESS>role = c("aut")),
       person("Ian", "Cook", email =<EMAIL_ADDRESS>       role = c("aut")), person("Nic", "Crane", email =
<EMAIL_ADDRESS>role = c("aut", "cre")),
       person("Dewey", "Dunnington", role = c("aut"), email =
<EMAIL_ADDRESS>comment = c(ORCID =
       "0000-0002-9415-4582")), person("Romain",
       "Fran\u00e7ois", role = c("aut"), comment = c(ORCID =
       "0000-0002-2444-4226")), person("Jonathan", "Keane",
       email =<EMAIL_ADDRESS>role = c("aut")),
       person("Drago\u0219", "Moldovan-Gr\u00fcnfeld", email =
<EMAIL_ADDRESS>role = c("aut")),
       person("Jeroen", "Ooms", email =<EMAIL_ADDRESS>       role = c("aut")), person("Jacob", "Wujciak-Jens", email
       =<EMAIL_ADDRESS>role = c("aut")),
       person("Javier", "Luraschi", email =
<EMAIL_ADDRESS>role = c("ctb")), person("Karl",
       "Dunkle Werner", email =
<EMAIL_ADDRESS>role = c("ctb"),
       comment = c(ORCID = "0000-0003-0523-7309")),
       person("Jeffrey", "Wong", email =
<EMAIL_ADDRESS>role = c("ctb")),
       person("Apache Arrow", email =<EMAIL_ADDRESS>       role = c("aut", "cph")) )
Description: 'Apache' 'Arrow' <https://arrow.apache.org/> is a
       cross-language development platform for in-memory data.
       It specifies a standardized language-independent
       columnar memory format for flat and hierarchical data,
       organized for efficient analytic operations on modern
       hardware. This package provides an interface to the
       'Arrow C++' library.
Depends: R (>= 3.4)
License: Apache License (>= 2.0)
URL: https://github.com/apache/arrow/,
       https://arrow.apache.org/docs/r/
BugReports: https://github.com/apache/arrow/issues
Encoding: UTF-8
Language: en-US
SystemRequirements: C++17; for AWS S3 support on Linux,
       libcurl and openssl (optional)
Biarch: true
Imports: assertthat, bit64 (>= 0.9-7), glue, methods, purrr,
       R6, rlang (>= 1.0.0), stats, tidyselect (>= 1.0.0),
       utils, vctrs
RoxygenNote: 7.2.3
Config/testthat/edition: 3
Suggests: blob, curl, cli, DBI, dbplyr, decor, distro, dplyr,
       duckdb (>= 0.2.8), hms, jsonlite, knitr, lubridate,
       pillar, pkgload, reticulate, rmarkdown, stringi,
       stringr, sys, testthat (>= 3.1.0), tibble, tzdb, withr
LinkingTo: cpp11 (>= 0.4.2)
Collate: 'arrowExports.R' 'enums.R' 'arrow-object.R' 'type.R'
       .....
NeedsCompilation: yes
Packaged: 2023-12-01 18:37:34 UTC; jwj
Author: Neal Richardson [aut], Ian Cook [aut], Nic Crane [aut,
       cre], Dewey Dunnington [aut]
       (<https://orcid.org/0000-0002-9415-4582>), Romain
       François [aut]
       (<https://orcid.org/0000-0002-2444-4226>), Jonathan
       Keane [aut], Dragoș Moldovan-Grünfeld [aut], Jeroen
       Ooms [aut], Jacob Wujciak-Jens [aut], Javier Luraschi
       [ctb], Karl Dunkle Werner [ctb]
       (<https://orcid.org/0000-0003-0523-7309>), Jeffrey Wong
       [ctb], Apache Arrow [aut, cph]
Maintainer: Nic Crane<EMAIL_ADDRESS>Repository: RSPM
Date/Publication: 2023-12-02 01:30:03 UTC
Built: R 4.3.0; x86_64-pc-linux-gnu; 2024-01-03 00:08:18 UTC;
       unix

When I load it outside of a project with renv it lists "Repository: CRAN".

@sj-io Thanks for confirming that.
We've also now rebuilt arrow with the fix for 14.0.0 and <IP_ADDRESS>, so installing these older versions should work as well.

Sorry, but I still get errors (I think is the same as before). Am I doing something wrong?
I tried uninstalled previous version before installing the new one, but with no luck either.
> install.packages("arrow")
Installing package into ‘/Users/user/.local/R_libs’
(as ‘lib’ is unspecified)
trying URL 'https://cran.rstudio.com/src/contrib/arrow_<IP_ADDRESS>.tar.gz'
Content type 'application/x-gzip' length 4263864 bytes (4.1 MB)
==================================================
downloaded 4.1 MB

* installing *source* package ‘arrow’ ...
** package ‘arrow’ successfully unpacked and MD5 sums checked
** using staged installation
*** pkg-config found.
*** Trying Arrow C++ found by pkg-config: /opt/homebrew/Cellar/apache-arrow/15.0.0
**** Not using: C++ library version (15.0.0) does not match R package (<IP_ADDRESS>)
*** Found libcurl and OpenSSL >= 3.0.0
*** Successfully retrieved libarrow (darwin-arm64-openssl-3.0)
*** Checksum validated successfully for libarrow: darwin-arm64-openssl-3.0/arrow-14.0.0.zip

[...]

installing to /Users/user/.local/R_libs/00LOCK-arrow/00new/arrow/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Users/user/.local/R_libs/00LOCK-arrow/00new/arrow/libs/arrow.so':
  dlopen(/Users/user/.local/R_libs/00LOCK-arrow/00new/arrow/libs/arrow.so, 0x0006): symbol not found in flat namespace '__ZN5arrow2fs10GcsOptions15FromAccessTokenERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS2_6chrono10time_pointINSB_12system_clockENSB_8durationIxNS2_5ratioILl1ELl1000000000EEEEEEE'
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Users/user/.local/R_libs/arrow’
Warning in install.packages :
  installation of package ‘arrow’ had non-zero exit status

The downloaded source packages are in
	‘/private/var/folders/24/dq2wn8216jv17zwrzgsnxndw0000gn/T/RtmpbPRmzF/downloaded_packages’

I think there are two different issues in this thread, one about installing arrow from source on Homebrew R, and one about installing the precompiled arrow package binaries from Posit Package Manager (https://packagemanager.posit.co/cran/latest).
We've addressed the issues with installing precompiled arrow from Posit Package Manager, but I don't think there was a resolution with installing arrow from Homebrew R.
I'm not sure what the Homebrew issue may be, but @juanfcocontreras you can try installing arrow from https://packagemanager.posit.co/cran/latest instead of https://cran.rstudio.com/. CRAN binary packages on macOS typically aren't compatible with Homebrew R, but the Package Manager binary packages may happen to work on Homebrew R.

Hello @glin ,
Thank you very much for your detailed reply, I wasn't aware of it, until now (and until version 13 of arrow) everything was working correctly from CRAN.
I just tried to install from packagemanager.postit:
install.packages('arrow', repos = "https://packagemanager.posit.co/cran/latest")
But I'm still getting errors. Did I do something wrong or it just doesn't work from pacakgemanager either?
> install.packages('arrow', repos = "https://packagemanager.posit.co/cran/latest")
Installing package into ‘/Users/user/.local/R_libs’
(as ‘lib’ is unspecified)
trying URL 'https://packagemanager.posit.co/cran/latest/src/contrib/arrow_<IP_ADDRESS>.tar.gz'
Content type 'binary/octet-stream' length 4267835 bytes (4.1 MB)
==================================================
downloaded 4.1 MB

* installing *source* package ‘arrow’ ...
** package ‘arrow’ successfully unpacked and MD5 sums checked
** using staged installation

[...]

installing to /Users/user/.local/R_libs/00LOCK-arrow/00new/arrow/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Users/user/.local/R_libs/00LOCK-arrow/00new/arrow/libs/arrow.so':
  dlopen(/Users/user/.local/R_libs/00LOCK-arrow/00new/arrow/libs/arrow.so, 0x0006): symbol not found in flat namespace '__ZN5arrow2fs10GcsOptions15FromAccessTokenERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS2_6chrono10time_pointINSB_12system_clockENSB_8durationIxNS2_5ratioILl1ELl1000000000EEEEEEE'
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Users/user/.local/R_libs/arrow’
Warning in install.packages :
  installation of package ‘arrow’ had non-zero exit status

The downloaded source packages are in
	‘/private/var/folders/24/dq2wn8216jv17zwrzgsnxndw0000gn/T/RtmpTk6Cx1/downloaded_packages’

@juanfcocontreras Sorry, I forgot that Homebrew R has binary package installation disabled entirely, not just from CRAN, so trying to install binaries from Posit Package Manager won't work out of the box either. That is still attempting a source installation where R tries to compile arrow, rather than installing the precompiled binary from Package Manager.
I'd recommend switching to the R installation from CRAN if possible, rather than Homebrew, as you'll have a much better package installation experience in general, being able to use all of CRAN's precompiled packages.
If that's not feasible, I think it may still be possible to force a binary package installation on Homebrew R, but it's not straightforward. You can install directly from the binary files using:
install.packages("https://packagemanager.posit.co/cran/latest/bin/macosx/big-sur-arm64/contrib/4.3/arrow_<IP_ADDRESS>.tgz", repos = NULL)

Where you can find the binary package file URL from the Package Manager website, for a given R version and macOS arch (x86 or arm64): https://packagemanager.posit.co/client/#/repos/cran/packages/overview?search=arrow. This example is for R 4.3, macOS arm64.
This method won't install dependencies by default, so you'll have to individually install each dependency manually.
For why Homebrew R isn't compatible with CRAN, here is some background: https://stat.ethz.ch/pipermail/r-sig-mac/2016-March/011890.html

I forgot that Homebrew R has binary package install disabled
oO TIL

@juanfcocontreras I am going to close this issue as the actual matter is closed (Thanks for that and the brew help too @glin!)
As @glin said homebrew R is not recommended for arrow specifically and also overall ;) If you want the ease of a packagemanager to update and switch R versions I would highly recommend https://github.com/r-libs/rig!

Just as a side note, while Homebrew's R formula has the issues mentioned above, the Cask version works great (it's just the CRAN .pkg installer), so saying homebrew's R isn't recommended isn't quite right since Homebrew is great at managing the CRAN version of R.

Thank you so much for the suggestion! It's working great!!

Just as a side note, while Homebrew's R formula has the issues mentioned above, the Cask version works great (it's just the CRAN .pkg installer), so saying homebrew's R isn't recommended isn't quite right since Homebrew is great at managing the CRAN version of R.

I have had success with the --cask method from @juanfcocontreras and @amoeba. Thank you. I did have to remove the previous "formula"-based homebrew install of R before arrow would properly install. After that it worked great.


Adding a VM to a resource pool is not reflected in tfstate

When cloning VM templates, the specified pool is not stored correctly. Note how the following main.tf adds the 3 VMs to the pool "production":
locals {
  hosts = {
    "swarm-w01" = { ipconfig = "ip=<IP_ADDRESS>/24,gw=<IP_ADDRESS>", hagroup = "pve01-first" },
    "swarm-w02" = { ipconfig = "ip=<IP_ADDRESS>/24,gw=<IP_ADDRESS>", hagroup = "pve02-first" },
    "swarm-w03" = { ipconfig = "ip=<IP_ADDRESS>/24,gw=<IP_ADDRESS>", hagroup = "pve03-first" },
  }
}

resource "proxmox_vm_qemu" "vm" {
  for_each = tomap(local.hosts)

  name = each.key
  target_node = "pve01"
  pool = "production"
  clone = "debian12-cloudinit"

  os_type = "cloud-init"
  onboot = true
  bios = "ovmf"

  cores   = 4
  sockets = 1
  memory  = 4096

  scsihw = "virtio-scsi-pci"
  disks {
    scsi {
      scsi0 {
        disk {
          storage = "pve-ceph"
          emulatessd = true
          discard = true
          size = "8G"
        }
      }
    }
    ide {
      ide2 {
        cloudinit {
          storage = "pve-ceph"
        }
      }
    }
  }

  network {
    model = "virtio"
    bridge = "vnet002"
  }
  ipconfig0 = each.value.ipconfig
  agent = 1

  hagroup = each.value.hagroup
  hastate = "started"

  ciuser = "user"
  sshkeys = data.http.ssh_keys.response_body
}

When performing terraform apply the VMs actually get added to the resource pool. This is not reflected in terraform.tfstate:

Thus, without any changes, when terraform plan is executed directly afterwards, it assumes the pool property needs to be changed:
$ terraform plan
proxmox_vm_qemu.vm["swarm-w01"]: Refreshing state... [id=pve01/qemu/121]
proxmox_vm_qemu.vm["swarm-w02"]: Refreshing state... [id=pve02/qemu/122]
proxmox_vm_qemu.vm["swarm-w03"]: Refreshing state... [id=pve03/qemu/123]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with
the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # proxmox_vm_qemu.vm["swarm-w01"] will be updated in-place
  ~ resource "proxmox_vm_qemu" "vm" {
        id                     = "pve01/qemu/121"
        name                   = "swarm-w01"
      + pool                   = "production"
        tags                   = null
        # (48 unchanged attributes hidden)

        # (3 unchanged blocks hidden)
    }

  # proxmox_vm_qemu.vm["swarm-w02"] will be updated in-place
  ~ resource "proxmox_vm_qemu" "vm" {
        id                     = "pve02/qemu/122"
        name                   = "swarm-w02"
      + pool                   = "production"
        tags                   = null
        # (48 unchanged attributes hidden)

        # (3 unchanged blocks hidden)
    }

  # proxmox_vm_qemu.vm["swarm-w03"] will be updated in-place
  ~ resource "proxmox_vm_qemu" "vm" {
        id                     = "pve03/qemu/123"
        name                   = "swarm-w03"
      + pool                   = "production"
        tags                   = null
        # (48 unchanged attributes hidden)

        # (3 unchanged blocks hidden)
    }

Plan: 0 to add, 3 to change, 0 to destroy.

@meerumschlungen this seems to be fixed in the current master branch


বাংলা ট্রান্সলেশন

বাংলা ট্রান্সলেশন এর কোন সিন্টেক্স/কীভাবে কল করা লাগবে কোন ইন্সট্রাকশন পাইনি ডকুমেন্টেশন এ।
ধন্যবাদ এই এপিআই এর জন্য

@abusayed0206 We will add Bangla translation very soon.
We have plan to add other translations too. But cloudflare pages can only have atmost 20,000 files.

We've added Bangla translation in the /api/surahNo/ayahNo endpoint.
Please note that we may later change the structure of how we serve the other translations. Even though we will notify you before the change happens, it is better not to use this in serious projects.
Jazakallahu Khairan.

@abusayed0206 We're using CF pages because it offers unlimited bandwidth for static files. Where Vercel, Netlify etc has a 100GB bandwidth limit.
That's the reason we can have no rate limit.
And even if we serve from the database, we'll still have a latency problem.
And this site is getting ~4GB of traffic every week. And it's growing every day.
What we're thinking right now is to store all the files in github or github pages. And then we can use raw.github.com or jsdeliver cdn to serve the files.


Copy store files from the noisepy repo into noisepy-io

Start with the following files: numpystore.py, stores.py, asdfstore.py and datatypes.py
into the new one under src/noisepy/seis/io (additional files may be needed, which we can discuss)

Closed at https://github.com/noisepy/noisepy-io/pull/2


feat(fw): Gas test generator

🗒 Description
Creates a gas test generator based off the MCOPY memory copy tests.
It can be applied to *any test and the test writer only needs to provide a couple of fixtures for it to work.
*It only works on legacy EVM code at the moment and not EOF, but I have a couple of ideas of how to make it work.
🔗 Related Issues
None
✅ Checklist

[ ] All: Set appropriate labels for the changes.
[ ] All: Considered squashing commits to improve commit history.
[ ] All: Added an entry to CHANGELOG.md.
[ ] All: Considered updating the online docs in the ./docs/ directory.
[ ] Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
[ ] Tests: A PR with removal of converted JSON/YML tests from ethereum/tests have been opened.
[ ] Tests: Included the type and version of evm t8n tool used to locally execute test cases:  e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
[ ] Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

*It only works on legacy EVM code at the moment and not EOF, but I have a couple of ideas of how to make it work.

Why? I tried and just wrapped test_code in a container and it seems to just have worked. It would need to "know" whether it should generate legacy or EOF or both (either directly or by combining with #610). Were there any other issues?

*It only works on legacy EVM code at the moment and not EOF, but I have a couple of ideas of how to make it work.

Why? I tried and just wrapped test_code in a container and it seems to just have worked. It would need to "know" whether it should generate legacy or EOF or both (either directly or by combining with #610). Were there any other issues?

No other issues, I meant working automatically, and yes I agree #610 should solve this.

@pdobacz @shemnon I rebased to main and renamed this generator to exact_gas_test so it differentiates from the one in #713.
I also added the EOF marker @pytest.mark.with_all_evm_code_types to the MCOPY test and it works out of the box in conjunction with this generator to produce legacy and EOF tests automatically.

@shemnon @marioevz I'm totally on the fence with the #713 vs #719, ever so slightly in favor of this #719, just because it already has OOG testing and easier integration with the EOF marker and no GAS opcode dependency. Or is the plan to have both? Maybe this is the way, and we just write both kinds for each case to test, serving two slightly different purposes.
Would other people on the testing team have a preference here?


Filter user articles need to be in redux store.

@elenakot set up a function in userArticles that now properly sends filtered articles to redux store.

---
However, on click of generatelink I am getting a 500 error response from the DB.

---
As far as I can tell the three args (userId, filteredarticles, message) are present. Would you mind pulling down branch sharingsReduxStore and checking this out? Thanks!


Support TLS client authentication

I was wondering whether it could be possible to implement TLS  client authentication in such a way that client certificate and key could be passed to the CLI or stored in .jfrog/security.
Currently, I think the only way to perform such authentication is to use 'curl --key client.key --cert client.crt' and artifactory REST APIs. It would be great if jfrog CLI could be used instead, especially when used as part of Maven or Gradle Builds.

I could also benefit from such a change.

Thanks for sharing this requirement @acabarbaye and @lukasHD.
A few questions:

Currently, with JFrog CLI, if you add the certificate file to the $JFROG_CLI_HOME/security directory, does the CLI authenticate with Artifactory successfully?
Can you share some more information about the use case? Why do you need to use different certificates when  running different CLI commands?

I can try to elaborate about my use case.
I am using Circle CI as a build system and trying to push artifacts to a local instance of artifactory after every build. To connect to this instance, we have a reverse proxy which requires TLS client authentication. jfrog CLI does not have such a capability at the moment and hence, I have to use Curl and artifactory REST APIs instead.

Thanks @acabarbaye.
If you place the certificate in the ~/.jfrog/security directory, does the CLI manage to authenticate with Artifactory successfully?

@acabarbaye,
I'm closing this issue gor now, but do let me know if you need further assistance with this,

In our corporate environment we are using user+apikey and in addition client certificates. We would like to use jfro cli but currently have to revert to curl
curl --cert ~/path/to/client_cert.pem -H 'X-JFrog-Art-Api: MyApiKey' -O https://file/to/download


Language Server - parseType records

( from https://kolmafia.us/threads/ash-language-server-features.26098/post-163762 )
This changes the logic of parseType to attempt to parse records even when not allowed (again, with the continuous parsing, we need to try anyway).
This is possible because the method parseRecord starts by looking for the token "record", immediately returning null if it doesn't find it.
"record" is a reserved word exclusively used to declare new records, so nothing is hindered by checking for them right away.

This PR was automatically closed because we rebuilt the main branch on moving day. To reopen this PR:

Copy this branch
Hard reset this branch to main
Cherry pick your commits from your copy onto this branch
Hit "Reopen"!


flutter doctor: crash after installing xcode

Steps to Reproduce
installed xcode from App Store. Then ran flutter doctor twice. crashed each time with the following
error:
flutter doctor

Agreeing to the Xcode/iOS license requires admin privileges, please run “sudo xcodebuild -license” and then retry this command.

Building flutter tool...

Oops; flutter has exited unexpectedly.
Sending crash report to Google.
Failed to send crash report. Server responded with HTTP status code 400
Crash report written to /Users/salah/flutter_01.log;
please let us know at https://github.com/flutter/flutter/issues.
➜  ~ sudo flutter doctor
   Woah! You appear to be trying to run flutter as root.
   We strongly recommend running the flutter tool without superuser privileges.
  /
📎

You have not agreed to the Xcode license agreements, please run 'xcodebuild -license' (for user-level acceptance) or 'sudo xcodebuild -license' (for system-wide acceptance) from within a Terminal window to review and agree to the Xcode license agreements.
/Users/salah/local/flutter/bin/flutter: line 75: [: !=: unary operator expected

Oops; flutter has exited unexpectedly.
Sending crash report to Google.
Failed to send crash report. Server responded with HTTP status code 400
Crash report written to /Users/salah/flutter_02.log;
please let us know at https://github.com/flutter/flutter/issues.

Logs
crash log flutter_01:
Flutter crash report; please file at https://github.com/flutter/flutter/issues.
command
flutter doctor
exception
VersionCheckError: VersionCheckError: Command exited with code 69: git log -n 1 --pretty=format:%ad --date=iso
Standard error:
Agreeing to the Xcode/iOS license requires admin privileges, please run “sudo xcodebuild -license” and then retry this command.
#0      _runSync (package:flutter_tools/src/version.dart:439)
#1      FlutterVersion._latestGitCommitDate (package:flutter_tools/src/version.dart:129)
#2      FlutterVersion.frameworkCommitDate (package:flutter_tools/src/version.dart:120)
#3      FlutterVersion.checkFlutterVersionFreshness (package:flutter_tools/src/version.dart:235)
<asynchronous suspension>
#4      FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:252)
<asynchronous suspension>
#5      CommandRunner.run.<anonymous closure> (package:args/command_runner.dart:109)
#6      new Future.sync (dart:async/future.dart:222)
#7      CommandRunner.run (package:args/command_runner.dart:109)
#8      FlutterCommandRunner.run (package:flutter_tools/src/runner/flutter_command_runner.dart:166)
#9      run.<anonymous closure> (package:flutter_tools/runner.dart:90)
<asynchronous suspension>
#10     AppContext._run (package:flutter_tools/src/base/context.dart:76)
<asynchronous suspension>
#11     AppContext.runInZone.<anonymous closure> (package:flutter_tools/src/base/context.dart:66)
#12     _rootRun (dart:async/zone.dart:1126)
#13     _CustomZone.run (dart:async/zone.dart:1023)
#14     runZoned (dart:async/zone.dart:1501)
#15     AppContext.runInZone (package:flutter_tools/src/base/context.dart:65)
#16     run (package:flutter_tools/runner.dart:61)
<asynchronous suspension>
#17     main (package:flutter_tools/executable.dart:48)
<asynchronous suspension>
#18     main (file:///Users/salah/local/flutter/packages/flutter_tools/bin/flutter_tools.dart:16)
#19     _startIsolate.<anonymous closure> (dart:isolate-patch/dart:isolate/isolate_patch.dart:277)
#20     _RawReceivePortImpl._handleMessage (dart:isolate-patch/dart:isolate/isolate_patch.dart:165)

flutter doctor
encountered exception: VersionCheckError: Command exited with code 69: git log -n 1 --pretty=format:%ad --date=iso
Standard error:

Agreeing to the Xcode/iOS license requires admin privileges, please run “sudo xcodebuild -license” and then retry this command.

#0      _runSync (package:flutter_tools/src/version.dart:439)
#1      FlutterVersion._latestGitCommitDate (package:flutter_tools/src/version.dart:129)
#2      FlutterVersion.frameworkCommitDate (package:flutter_tools/src/version.dart:120)
#3      FlutterVersion.frameworkDate (package:flutter_tools/src/version.dart:83)
#4      _FlutterValidator.validate (package:flutter_tools/src/doctor.dart:236)
<asynchronous suspension>
#5      Doctor.diagnose (package:flutter_tools/src/doctor.dart:112)
<asynchronous suspension>
#6      _doctorText.<anonymous closure> (package:flutter_tools/runner.dart:237)
#7      AppContext._run (package:flutter_tools/src/base/context.dart:76)
<asynchronous suspension>
#8      AppContext.runInZone.<anonymous closure> (package:flutter_tools/src/base/context.dart:66)
#9      _rootRun (dart:async/zone.dart:1126)
#10     _CustomZone.run (dart:async/zone.dart:1023)
#11     runZoned (dart:async/zone.dart:1501)
#12     AppContext.runInZone (package:flutter_tools/src/base/context.dart:65)
#13     _doctorText (package:flutter_tools/runner.dart:237)
<asynchronous suspension>
#14     _createLocalCrashReport (package:flutter_tools/runner.dart:212)
<asynchronous suspension>
#15     _handleToolError (package:flutter_tools/runner.dart:167)
<asynchronous suspension>
#16     run.<anonymous closure> (package:flutter_tools/runner.dart:94)
<asynchronous suspension>
#17     AppContext._run (package:flutter_tools/src/base/context.dart:76)
<asynchronous suspension>
#18     AppContext.runInZone.<anonymous closure> (package:flutter_tools/src/base/context.dart:66)
#19     _rootRun (dart:async/zone.dart:1126)
#20     _CustomZone.run (dart:async/zone.dart:1023)
#21     runZoned (dart:async/zone.dart:1501)
#22     AppContext.runInZone (package:flutter_tools/src/base/context.dart:65)
#23     run (package:flutter_tools/runner.dart:61)
<asynchronous suspension>
#24     main (package:flutter_tools/executable.dart:48)
<asynchronous suspension>
#25     main (file:///Users/salah/local/flutter/packages/flutter_tools/bin/flutter_tools.dart:16)
#26     _startIsolate.<anonymous closure> (dart:isolate-patch/dart:isolate/isolate_patch.dart:277)
#27     _RawReceivePortImpl._handleMessage (dart:isolate-patch/dart:isolate/isolate_patch.dart:165)

crash log flutter_02:
`Flutter crash report; please file at https://github.com/flutter/flutter/issues.
command
flutter doctor
exception
VersionCheckError: VersionCheckError: Command exited with code 69: git log -n 1 --pretty=format:%ad --date=iso
Standard error:
You have not agreed to the Xcode license agreements, please run 'xcodebuild -license' (for user-level acceptance) or 'sudo xcodebuild -license' (for system-wide acceptance) from within a Terminal window to review and agree to the Xcode license agreements.
#0      _runSync (package:flutter_tools/src/version.dart:439)
#1      FlutterVersion._latestGitCommitDate (package:flutter_tools/src/version.dart:129)
#2      FlutterVersion.frameworkCommitDate (package:flutter_tools/src/version.dart:120)
#3      FlutterVersion.checkFlutterVersionFreshness (package:flutter_tools/src/version.dart:235)
<asynchronous suspension>
#4      FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:252)
<asynchronous suspension>
#5      CommandRunner.run.<anonymous closure> (package:args/command_runner.dart:109)
#6      new Future.sync (dart:async/future.dart:222)
#7      CommandRunner.run (package:args/command_runner.dart:109)
#8      FlutterCommandRunner.run (package:flutter_tools/src/runner/flutter_command_runner.dart:166)
#9      run.<anonymous closure> (package:flutter_tools/runner.dart:90)
<asynchronous suspension>
#10     AppContext._run (package:flutter_tools/src/base/context.dart:76)
<asynchronous suspension>
#11     AppContext.runInZone.<anonymous closure> (package:flutter_tools/src/base/context.dart:66)
#12     _rootRun (dart:async/zone.dart:1126)
#13     _CustomZone.run (dart:async/zone.dart:1023)
#14     runZoned (dart:async/zone.dart:1501)
#15     AppContext.runInZone (package:flutter_tools/src/base/context.dart:65)
#16     run (package:flutter_tools/runner.dart:61)
<asynchronous suspension>
#17     main (package:flutter_tools/executable.dart:48)
<asynchronous suspension>
#18     main (file:///Users/salah/local/flutter/packages/flutter_tools/bin/flutter_tools.dart:16)
#19     _startIsolate.<anonymous closure> (dart:isolate-patch/dart:isolate/isolate_patch.dart:277)
#20     _RawReceivePortImpl._handleMessage (dart:isolate-patch/dart:isolate/isolate_patch.dart:165)

flutter doctor
encountered exception: VersionCheckError: Command exited with code 69: git log -n 1 --pretty=format:%ad --date=iso
Standard error:
You have not agreed to the Xcode license agreements, please run 'xcodebuild -license' (for user-level acceptance) or 'sudo xcodebuild -license' (for system-wide acceptance) from within a Terminal window to review and agree to the Xcode license agreements.

#0      _runSync (package:flutter_tools/src/version.dart:439)
#1      FlutterVersion._latestGitCommitDate (package:flutter_tools/src/version.dart:129)
#2      FlutterVersion.frameworkCommitDate (package:flutter_tools/src/version.dart:120)
#3      FlutterVersion.frameworkDate (package:flutter_tools/src/version.dart:83)
#4      _FlutterValidator.validate (package:flutter_tools/src/doctor.dart:236)
<asynchronous suspension>
#5      Doctor.diagnose (package:flutter_tools/src/doctor.dart:112)
<asynchronous suspension>
#6      _doctorText.<anonymous closure> (package:flutter_tools/runner.dart:237)
#7      AppContext._run (package:flutter_tools/src/base/context.dart:76)
<asynchronous suspension>
#8      AppContext.runInZone.<anonymous closure> (package:flutter_tools/src/base/context.dart:66)
#9      _rootRun (dart:async/zone.dart:1126)
#10     _CustomZone.run (dart:async/zone.dart:1023)
#11     runZoned (dart:async/zone.dart:1501)
#12     AppContext.runInZone (package:flutter_tools/src/base/context.dart:65)
#13     _doctorText (package:flutter_tools/runner.dart:237)
<asynchronous suspension>
#14     _createLocalCrashReport (package:flutter_tools/runner.dart:212)
<asynchronous suspension>
#15     _handleToolError (package:flutter_tools/runner.dart:167)
<asynchronous suspension>
#16     run.<anonymous closure> (package:flutter_tools/runner.dart:94)
<asynchronous suspension>
#17     AppContext._run (package:flutter_tools/src/base/context.dart:76)
<asynchronous suspension>
#18     AppContext.runInZone.<anonymous closure> (package:flutter_tools/src/base/context.dart:66)
#19     _rootRun (dart:async/zone.dart:1126)
#20     _CustomZone.run (dart:async/zone.dart:1023)
#21     runZoned (dart:async/zone.dart:1501)
#22     AppContext.runInZone (package:flutter_tools/src/base/context.dart:65)
#23     run (package:flutter_tools/runner.dart:61)
<asynchronous suspension>
#24     main (package:flutter_tools/executable.dart:48)
<asynchronous suspension>
#25     main (file:///Users/salah/local/flutter/packages/flutter_tools/bin/flutter_tools.dart:16)
#26     _startIsolate.<anonymous closure> (dart:isolate-patch/dart:isolate/isolate_patch.dart:277)
#27     _RawReceivePortImpl._handleMessage (dart:isolate-patch/dart:isolate/isolate_patch.dart:165)

`
Run your application with flutter run and attach all the log output.
Run flutter analyze and attach any output of that command also.
Flutter Doctor
Paste the output of running flutter doctor -v here.

For more information about diagnosing and reporting Flutter bugs, please see https://flutter.io/bug-reports/.

There has been a recent change to escaping on that shell script that I wonder if may have fixed this:
https://github.com/flutter/flutter/pull/15087
Are you still seeing the issue with the latest beta?


Command to time code execution

It could be useful to have a timeit-like bot command to time code execution. Some ideas of features the command could have are:

The ability to specify how many times the code should be run. It could also be interesting if this was calculated automatically like how the timeit Command-Line Interface does.
The ability to take in multiple snippets of code to compare
The output could just be the time taken as timeit.timeit gives (e.g 0.3018611848820001) but it would be nicer if the output was in a more human friendly format (e.g. the timeit CLI outputs more like 10000 loops, best of 5: 30.2 usec per loop)

I would assume it wouldn't be too difficult to implement as it could borrow from/use most of the same code as !eval but I am not sure.

This sounds pretty easy. It can generate a timeit() code snippet and insert the given code into the first argument. It'd need to have a way to escape quotes. That new code can then be sent to Snekbox.
Supporting multiple code snippets is possible, but adds complexity to the input parser. Furthermore, Snekbox still has a time limit which all snippets would share. I think it's best to not support that; it isn't much trouble for users to invoke a separate eval.

That being said, it may make more sense to add another endpoint to Snekbox, since it's pretty simple on that side to just invoke python -m timeit rather than python -c.

I think the implementation we choose for this feature would depend on if decide to introduce a mypy endpoint; if we already have two endpoints I think it would make sense to have a third one, otherwise wrapping the code in a timeit snippet makes more sense IMO

Id like to implement this feature
It would be fairly trivial to wrap the code in a timeit snippet to test the time it takes. Currently Snekbox only has one endpoint and im not sure it would make sense to reimplement eval with python -m timeit <code> when its easy to just eval timeit.main([<code>])

@davisschenk That approach sounds fine, though it is noteworthy that main() is undocumented. I'm not sure if this is unintentional or if it implies that it's not part of the public API. In any case, I'll assign you to this issue.

main([<args>]) is just the equivalent of python -m timeit <args>
Anyone have any opinions on whether or not this should be implemented as a sub-command of eval or as a standalone command?

Standalone. It's not returning the code's output so it's not strictly an eval of the original input. Furthermore, a subcommand would make it impossible to use the subcommand name at the start of the code for an eval unless it was in a code block. Lastly, I imagine it'd complicate parsing of revaluations.

It seems like this issue has been abandoned. I would like to implement it if there are no objections.

It seems like this issue has been abandoned. I would like to implement it if there are no objections.

@WillDaSilva
If @ToxicKidz isn't interested in continuing their PR that sounds fine, although I think part of the reason that PR stalled is because there wasn't much of a conclusion on what the implementation should be.
Do you have a plan for implementation? Do you want to continue @ToxicKidz's PR (in which case do you have a plan for addressing Numerlor's comment there) or do you have a different idea?

@wookie184  On second thought I would rather not implement this anymore. Sorry for any trouble.

@wookie184  On second thought I would rather not implement this anymore. Sorry for any trouble.

No worries


fix(vercel-integration): Correct link syntax

Currently link can not be rendered:

@janpio - thanks for fixing the syntax here!


help needed pairing Ubuntu and/or Windows with Bosch PLR 30 C

After pressing the Bluetooh key on the Bosch PLR 30 C, the bluetooth item appears in the Bosch PLR 30 C's screen, and Ubuntu does discover the Bosch PLR 30 C. In Ubuntu, when I click on Connect, Ubuntu says:

After Confirming that ... still Disconnected.
Tip how to get the Bluetooth connection working?

I activated logging like described https://stackoverflow.com/a/38462866/1492917 ... but easy grep -i  "bosch" or "plr" does not reveal anything

... it neither works from Windows:
Windows does see the PLR30C, it connects, the PLR30C shows an extra symbol to the bluetooth symbol ... and then nothing

FWIW: I can connect from Android to the PLR30C (so Bluetooth is working), but that is besides the point as I can't run the python script on Android.
Is the Bosch PLR30C actively refusing connections from PC's?

Not shure if thats any helpfull here.
What I discovered is, that it works at least better, if I don't try to connect from the PC, but rather let the script do the connection, and just accept it from Ubuntu. I have only GLM50C though.


Removing Simply

Removing Simply from Line 105
Description of the change: Removed the word simply from line 105
Reason for the change: Simply is a little condescending
Link to original source:  https://docs.google.com/spreadsheets/d/15cVnU07Z-bbW7hBVX98W1JdWKyBf3E3pTIRegUa716A/edit#gid=0

Closes #

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hello @nathanleusink,Thanks again for the PR! We want to show our appreciation by sending                                              you some swag. Could you please fill out                                              this form so we can send it to you? Thanks!                                             Team SendGrid DX


Apply CCHDO changes from docker to helm charts, bump version

@yemoski tagging you as an FYI - this takes your CCHDO addition from the Gleaner yaml in the Docker deployment, and applies it to the Gleaner yaml in the Helm deployment. I haven't figured out how to make them use the same file yet.

Thanks for letting me know!


mypy_extensions missing from requirements.txt

python -m raiden_contracts.deploy raiden --rpc-provider http://<IP_ADDRESS>:8545  --private-key /home/yoichi/xud-eth/blkchain1/keystore/UTC--2019-03-26T15-46-54.573149705Z--b772b74bc6021aab88ef154d76eb3174258e3865 --gas-price 10 --gas-limit 6000000  --contracts-version 0.3._
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/yoichi/.virtualenvs/offerm-script/lib/python3.7/site-packages/raiden_contracts/deploy/__main__.py", line 14, in <module>
    from mypy_extensions import TypedDict
ModuleNotFoundError: No module named 'mypy_extensions'

So mypy_extensions needs to be in requirements.txt.

@pirapira We are not using TypedDict in raiden. This seems to be used only in contracts.

Yeah, this is a raiden-contracts issue.  So it should be in that repo... oh it's here.

@pirapira I can move the issue for you if you want. Or we can just simply leave it and close it when you fix it in the contracts.

To me this page is in contracts
https://github.com/raiden-network/raiden-contracts/issues/778 .

Hm the heck? For me too now. I can swear it was in Raiden before ^_^. Anyway!


iOS Simulator and Real device have different results

Describe the bug
When using requestType with RequestType.video I get different results in the iOS simulator than on a real device. In the iOS simulator, only video assets are returned. This is the correct behaviour. When the flutter app is built in release mode and used on a real iOS device (iPhone XR, OS: 16.6) all media is available to select. Filtering for video-only assets is not functioning.
How to reproduce
AssetPicker.pickAssets(context,
  pickerConfig: const AssetPickerConfig(
    maxAssets: 1,
    requestType: RequestType.video,
))

Steps to reproduce the behavior:
On a real iOS device

Trigger the asset picker
See that all assets are available, not filtered by asset type.

Expected behavior

Trigger the asset picker
See only video assets

Version information

Device: iPhone XR
OS: iOS 16.6

Simulator iPhone XR
OS: iOS 16.4

Package Version: 8.7.1
Flutter Version: 3.13.6

Additional context
Behaviour in the simulator is not the same as behaviour on real device. Validating this incorrect behaviour is not possible on the simulator, as it will provide the expected result.

https://github.com/fluttercandies/flutter_photo_manager?tab=readme-ov-file#filtering-only-live-photos

Cheers mate!
For anyone finding this later on. The link provided suggests the problem is that livephotos are included as "video" by default. This meant on my real device I had significantly more media displaying than expected. You need to pass an ios specific option to filter out livephotos as well.
I changed my code from the above example to the following, and got my expected result:
AssetPicker.pickAssets(context,
  pickerConfig: AssetPickerConfig(
    maxAssets: 1,
    requestType: RequestType.video,
    filterOptions: FilterOptionGroup(containsLivePhotos: false),
))

https://github.com/fluttercandies/flutter_wechat_assets_picker/blob/0015480e385a59a9188c6cba7f329a96a9245ff1/README.md?plain=1#L321-L324


Add basic module image

A basic (and empty) module image should:

Retrieve the Quartermaster source code in one of its early stages

At the moment this is only possible by cloning the repo, in the future, it'd use releases.
Repo URL/release number as a build argument.

@somayeh-najafi I'll just push an empty Dockerfile (e.g., from alpine) because we need to push QMSTR/broker-test's code in this repository as required by #7, and then add that code in the generic module container.

@somayeh-najafi I'll just push an empty Dockerfile (e.g., from alpine) because we need to push QMSTR/broker-test's code in this repository as required by #7, and then add that code in the generic module container.

The Dockerfile will be something like this.

The Dockerfile will be something like this.


Convert Docker.template file for use with local

resin-cli version: 6.10.2
Raspberry pi debian stretch

Issue #504 seems relevant but ultimately not helpful enough to resolve.
I duplicated and renamed Dockerfile.template to Dockerfile and then tried:
$ sudo resin local push 963d724.local -s .
* Building..
- Stopping and removing any previous 'local-app' container
Step 1/16 : FROM resin/%%RESIN_MACHINE_NAME%%-debian:stretch
rdt push failed. Error: invalid reference format: repository name must be lowercase Error: invalid reference format: repository name must be lowercase
    at Stream.<anonymous> (/Users/aj/.npm-packages/lib/node_modules/resin-cli/node_modules/resin-sync/build/docker-utils.js:146:23)
    at emitOne (events.js:96:13)
    at Stream.emit (events.js:188:7)
    at drain (/Users/aj/.npm-packages/lib/node_modules/resin-cli/node_modules/through/index.js:36:16)
    at Stream.stream.queue.stream.push (/Users/aj/.npm-packages/lib/node_modules/resin-cli/node_modules/through/index.js:45:5)
    at Parser.parser.onToken (/Users/aj/.npm-packages/lib/node_modules/resin-cli/node_modules/resin-sync/node_modules/JSONStream/index.js:132:18)
    at Parser.proto.write (/Users/aj/.npm-packages/lib/node_modules/resin-cli/node_modules/resin-sync/node_modules/jsonparse/jsonparse.js:135:34)
    at Stream.<anonymous> (/Users/aj/.npm-packages/lib/node_modules/resin-cli/node_modules/resin-sync/node_modules/JSONStream/index.js:23:12)
    at Stream.stream.write (/Users/aj/.npm-packages/lib/node_modules/resin-cli/node_modules/through/index.js:26:11)
    at IncomingMessage.ondata (_stream_readable.js:555:20)
    at emitOne (events.js:96:13)
    at IncomingMessage.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at IncomingMessage.Readable.push (_stream_readable.js:134:10)
    at HTTPParser.parserOnBody (_http_common.js:123:22)
    at Socket.socketOnData (_http_client.js:363:20)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at Socket.Readable.push (_stream_readable.js:134:10)
    at TCP.onread (net.js:547:20)

and then
$ sudo resin local push 963d724.local -s .
* Building..
- Stopping and removing any previous 'local-app' container
Step 1/16 : FROM resin/raspberrypi-debian:stretch
rdt push failed. Error: manifest for resin/raspberrypi-debian:stretch not found Error: manifest for resin/raspberrypi-debian:stretch not found
    at Stream.<anonymous> (/Users/aj/.npm-packages/lib/node_modules/resin-cli/node_modules/resin-sync/build/docker-utils.js:146:23)
    at emitOne (events.js:96:13)
    at Stream.emit (events.js:188:7)
    at drain (/Users/aj/.npm-packages/lib/node_modules/resin-cli/node_modules/through/index.js:36:16)
    at Stream.stream.queue.stream.push (/Users/aj/.npm-packages/lib/node_modules/resin-cli/node_modules/through/index.js:45:5)
    at Parser.parser.onToken (/Users/aj/.npm-packages/lib/node_modules/resin-cli/node_modules/resin-sync/node_modules/JSONStream/index.js:132:18)
    at Parser.proto.write (/Users/aj/.npm-packages/lib/node_modules/resin-cli/node_modules/resin-sync/node_modules/jsonparse/jsonparse.js:135:34)
    at Stream.<anonymous> (/Users/aj/.npm-packages/lib/node_modules/resin-cli/node_modules/resin-sync/node_modules/JSONStream/index.js:23:12)
    at Stream.stream.write (/Users/aj/.npm-packages/lib/node_modules/resin-cli/node_modules/through/index.js:26:11)
    at IncomingMessage.ondata (_stream_readable.js:555:20)
    at emitOne (events.js:96:13)
    at IncomingMessage.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at IncomingMessage.Readable.push (_stream_readable.js:134:10)
    at HTTPParser.parserOnBody (_http_common.js:123:22)
    at Socket.socketOnData (_http_client.js:363:20)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at Socket.Readable.push (_stream_readable.js:134:10)
    at TCP.onread (net.js:547:20)

The problem you're having here is simply that there's no stretch tag for this base image at all, because the raspberrypi-debian image is actually obsolete, and quite out of date (see https://hub.docker.com/r/resin/raspberrypi-debian/).
You can browse all the resin base images at https://hub.docker.com/r/resin/, and there's more detailed documentation on all this over here: https://docs.resin.io/runtime/resin-base-images/
If you replace raspberrypi-debian:stretch with rpi-raspbian:stretch, I think this will give you what you're expecting. Does that work for you?
As an aside, the proper fix for Dockerfile.template support is #604 - feel free to subscribe to that one for notifications when this is fully supported!

Although in fact, I've just been reminded that the real problem here is that the RESIN_MACHINE_NAME for raspberry pi is raspberry-pi, not raspberrypi (i.e. you need a dash).
That means you probably wanted raspberry-pi-debian:stretch, which will work as well (and should be exactly the same image as rpi-raspbian:stretch)

(the documentation is getting fixed in https://github.com/resin-io/docs/pull/657 )

@aj-ptw I'm going to close this and assume the above workaround solves your issue, but feel free to ping here if you're still having problems with this.


Inject into text.html.markdown to support fenced code blocks

I have requested the same change to the TOML extension: https://github.com/bungcip/better-toml/issues/29.
If you want to use the Kotlin extension as an example, here are the relevant bits:

https://github.com/mathiasfrohlich/vscode-kotlin/blob/090fe4cd054d6142d7eaefdb69c12d4b063a089e/syntaxes/codeblock.json
https://github.com/mathiasfrohlich/vscode-kotlin/blob/090fe4cd054d6142d7eaefdb69c12d4b063a089e/package.json#L68-L77

https://github.com/slackhq/vscode-hack/pull/100 is the fix.

#100 is the fix.

💯


Update for latest jig breaking changes

@amallia I've made most of the changes for the latest jig breaking changes, but I'm getting this error when running prepare with jig:
[2019-04-19 04:02:01.367] [info] [Batch 54] Processed documents [540000, 550000)
[2019-04-19 04:02:02.962] [info] [Batch 55] Processed documents [550000, 556081)
[2019-04-19 04:02:02.969] [info] Merging titles
[2019-04-19 04:02:02.988] [info] Merging URLs
[2019-04-19 04:02:02.990] [info] Collecting terms
Segmentation fault (core dumped)
Traceback (most recent call last):
  File "/index", line 16, in <module>
    .format(name, path), shell=True)
  File "/usr/lib/python3.5/subprocess.py", line 271, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'zcat -r /input/collections/robust04/* | parse_collection -f trectext -b 10000 --stemmer porter2 --content-parser html -o ./robust04.fwd' returned non-zero exit status 139

Can you take it from here?


[#4] Ability to install as python package from git

Fixes #4

This is ready to merge @lilyclemson

This is ready to merge @lilyclemson

@lilyclemson the error you get is weird and its not from our changes as your machine is failed to build lxml for some reason, can you retry again


how to skip processing of other events like data?

Hi ,
I have a code which make use of csv-parser and validates the csv file. I'm not using the strict:true option. I'm checking some checks with the headers(like valid headers which my application expects) and parse data in data event etc. However, Can you please let me know how to skip processing data event when I see some error in "header" event?
Ex: fileStream.pipe(csv()) .on('headers', (headers) => { }). on('data',(data) => { }). on('end', () => { });

just destroy the stream
fileStream.destroy()
// or
fileStream.destroy(new Error('bad headers'))

largeCSV.txt
Thank you . This helped in destroying the stream. I'm facing another issue where in I'm not able to parse entire csv file. Sometimes it is reading only 3k lines out of 8k lines file and sometimes it is reading only 6.5k lines of 10k lines file(which is of 220kb). Please find attached sample file(rename txt extension to csv) which I use. Also, below is the code snippet:
fileStream.pipe(csv())
 .on('headers', (headers) => {
  console.log('headers', headers);
})
.on('data', (data) => {
_.each(_.keys(data), (key) => {
   if(_.isEmpty(data[key])) {
  console.log('key: ', key ,' has empty value');
});
})
.

on('end',() => {});

Also, I noticed on testing multiple files, that I'm able to process only files of size  < 130KB only.  Do you know if there is any specific setting to be set to process large files?

Sorry.Not the issue with csv-parser. I'm facing backpressure problem when streaming the data and hence I'm able to read only 128kb. Will fix it separately. However, thanks for the help on the stream.destroy(). This issue can be closed.


Update date-fns dependency to 3.0 version

Hello!
A new version of date-fns just released. Are you going to update this dependency?
Thank you for your job!

This is currently on hold, as date-fns-tz is not compatible yet with date-fns@3. There is an open issue https://github.com/marnusw/date-fns-tz/issues/260

Is there a reason to keep date-fns-tz as a library?
Or could the functionality be replaced by the new available https://github.com/date-fns/utc ?
Didn't look deep into it, so might be totally on the wrong track 🙈


Bug in blueprint fusing algorigthm

Code below throws a "Bug in blueprint fusing algorigthm" exception.
First two triangles share one vertex, so fuse2D returns Blueprints. Fusing Blueprints with a triangle that overlaps only one of the blueprints raises an error.
const triangles = [
  [
    [8, 27],
    [7, 35],
    [11, 15],
  ],
  [
    [11, 15],
    [0, 15],
    [3, 17],
  ],
  [
    [11, 15],
    [3, 17],
    [4, 18],
  ],
]

const colors = ['red', 'green', 'blue']

 function main({BlueprintSketcher, fuse2D}) {
  const blueprints = triangles.map(([a, b, c]) =>
    new BlueprintSketcher().movePointerTo(a).lineTo(b).lineTo(c).close(),
  )

  // return blueprints.map((b,i) => ({shape:b.sketchOnPlane('XY').extrude(1), color: colors[i]}))

  const fused = blueprints.reduce(
    (acc, bp) => fuse2D(acc, bp),
    new BlueprintSketcher().hLine(1e-5).vLine(1e-5).close(),
  )

  return fused.sketchOnPlane('XY').extrude(1)
}

(a note for a separate question: all triangles here are in CW order and that works as expected, but if I change them into CCW - fuse gets confused about inside and outside of a blueprint. At the same time sketchRectangle works in CCW.
Either I am missing something obvious, or there's a difference in how sketches and blueprints work?
one way to test CCW triangles is to change the lambda inside map: triangles.map(([a, **c**, b])
)

Code in question is here: https://github.com/sgenoud/replicad/blob/c9371132de5fdf2232a25e17dc0298adc0efc453/packages/replicad/src/blueprints/boolean2D.ts#L89
I think genericIntersects returns true, because triangles touch at one point. But then genericFuse returns Blueprints because it cannot combine two triangles. This edge case is considered a "bug".
It was actually really hard to tell what is the difference between Blueprints and CompoundBlueprint.
The former produces Sketches the latter - CompoundSketch.
CompoundSketch is an array of N sketches with first one representing outer "shell", and remaining N-1 representing holes.
If that is correct, then wrapping every triangle into Blueprints should have worked:
function main({BlueprintSketcher, fuse2D, Blueprints}) {
   const blueprints = triangles.map(([a, b, c]) =>
     new BlueprintSketcher().movePointerTo(a).lineTo(b).lineTo(c).close(),
   )

   const fused = blueprints.reduce(
     (acc, bp) => fuse2D(acc, new Blueprints([bp])),
     new Blueprints([new BlueprintSketcher().hLine(1e-5).vLine(1e-5).close()]),
   )

   return fused.sketchOnPlane('XY').extrude(1)
}

But it doesn't, which may mean an unrelated bug?

Note that you might alos have made some weird behaving with stuff with new BlueprintSketcher().hLine(1e-5).vLine(1e-5).close() - which is close to the precision of the fusing algorithm. Changing to the following does not explode (while the fuse is not working):
 function main({BlueprintSketcher, fuse2D}) {
  const blueprints = triangles.map(([a, b, c]) =>
    new BlueprintSketcher().movePointerTo(a).lineTo(b).lineTo(c).close(),
  )

  // return blueprints.map((b,i) => b.sketchOnPlane('XY').extrude(1+i*0.2))

  const fused = blueprints.reduce(
    (acc, bp) => fuse2D(acc, bp),
    new BlueprintSketcher().hLine(1e-5).vLine(1e-5).close(),
  )

  return fused.sketchOnPlane('XY').extrude(1)
}

As an aside I would advice you to use the draw API which hides some complexity (typically with the blueprints vs blueprint):
function main({ draw }) {
  const drawings = triangles.map(([a, b, c]) =>
    draw().movePointerTo(a).lineTo(b).lineTo(c).close()
  );

  const fused = drawings
    .slice(1)
    .reduce((acc, bp) => acc.fuse(bp), drawings[0]);

  return fused.sketchOnPlane("XY").extrude(1);
}

All of this said, it looks like you have found a bug with the single common point - I will have a look! Note that the spirit of the error message is to say there is a bug in my algo because it reached a state I thought it would not. But you found an edge case where it does.

Yes, that "invisible" triangle was a workaround to get an empty Blueprint. I haven't yet found a correct way to do it, and assumed it doesn't exist.

Changing to the following does not explode (while the fuse is not working)

I went over your "changed" code sample several times to find what's changed and suspect you may have copy-pasted a wrong file 😉
But looking at your second example with slice(1) and and drawings[0] as the starting accumulator I think I know what you've meant. It does fix the "Bug in blueprint fusing" exception but result is incorrect.

Oups, yeah, copy paste error!
All in all, I have found (and fixed) the bug - can you confirm that once the latest version of the visulalizer is loaded it works?

(seems like my comment disappeared, maybe I hit a network glitch)
I can confirm that v0.12.3 works great for the original test case.
However, I'm now hitting an infinite call loop in cleanEdgeCases / handleNestedBlueprints.
My test code is here, will try to clean it up to be under 200 lines of code.
Do you prefer to open a new issue (so we'd have a sense of progress because this one is fixed) or add a new test case here?
P.S. is visualizer closed source? I could not find its source code in the main repo.

Your triangle mesh code is perfect to find bugs in my fusing algorithm. I solved the bugs you originally reported here, but it uncovered more.
The general gist is that the mesh creates shapes that do not intersects themselves but touch themselves in one point. It messes with the assumptions I had made for my algo a lot by opening a bunch of edge cases I had not thought about. You can see my progress here

Glad you are having fun! I was worried I sent too many bug reports at once :)
These triangles are from an experiment in unwrapping  faces on a flat surface (think papercraft). So none of them should overlap (save for some bugs in earlier stages of my code).
Does OCJS expose any APIs to use OCCT's 2d boolean operations? I'm very new to these libraries, but this forum thread suggests there may be a function for that.

This is for merging stuff and creating faces (i.e. a 2D surface in 3D space). What I am trying to achieve with drawings (and blueprints) is to have stuff in 2D in a 2D plane - not notion of the third dimension.
For the papercraft stuff, I am not sure that merging the triangles is the good way of doing it - you still want the borders to of the triangles to exists (to make folding patterns).
Is this what you expect from the merge? Is my algorithm dropping some triangles it should not, or are there some triangles missing in your example)?

Yes, your image matches the one I get if I simply extrude each triangle separately.
Closing this issue now.
Thanks a lot for fixing it!


Fix Django rest framework issues for mkdocstrings pages in vast_pipeline

Currently the following files are skipped in the mkdocstrings generation because of an no attribute error:
vast_pipeline/serializers.py
vast_pipeline/views.py

https://github.com/askap-vast/vast-pipeline/blob/18178eb6a4e9e01d15c7382129028f64e0546636/docs/gen_doc_stubs.py#L10-L14
I believe the same issue as described in https://github.com/mkdocstrings/mkdocstrings/issues/141.

I revisited this as [pytkdocs](https://github.com/mkdocstrings/pytkdocs) v0.12.0 was released that included support for Django models (though basic support is noted in the issues).
It seems that this actually regresses the documentation as with the update other files begin to fail with errors such as:
ERROR    -  mkdocstrings.extension: 'GenericForeignKey' object has no attribute 'null'
            Traceback (most recent call last):
              File "/Users/adam/anaconda3/envs/vast-pipeline-dev/lib/python3.7/site-packages/pytkdocs/cli.py", line 205, in main
                output = json.dumps(process_json(line))
              File "/Users/adam/anaconda3/envs/vast-pipeline-dev/lib/python3.7/site-packages/pytkdocs/cli.py", line 114, in
            process_json
                return process_config(json.loads(json_input))
              File "/Users/adam/anaconda3/envs/vast-pipeline-dev/lib/python3.7/site-packages/pytkdocs/cli.py", line 91, in
            process_config
                obj = loader.get_object_documentation(path, members)
              File "/Users/adam/anaconda3/envs/vast-pipeline-dev/lib/python3.7/site-packages/pytkdocs/loader.py", line 358, in
            get_object_documentation
                root_object = self.get_module_documentation(leaf, members)
              File "/Users/adam/anaconda3/envs/vast-pipeline-dev/lib/python3.7/site-packages/pytkdocs/loader.py", line 426, in
            get_module_documentation
                root_object.add_child(self.get_class_documentation(child_node))
              File "/Users/adam/anaconda3/envs/vast-pipeline-dev/lib/python3.7/site-packages/pytkdocs/loader.py", line 506, in
            get_class_documentation
                child = self.get_class_documentation(child_node)
              File "/Users/adam/anaconda3/envs/vast-pipeline-dev/lib/python3.7/site-packages/pytkdocs/loader.py", line 506, in
            get_class_documentation
                child = self.get_class_documentation(child_node)
              File "/Users/adam/anaconda3/envs/vast-pipeline-dev/lib/python3.7/site-packages/pytkdocs/loader.py", line 538, in
            get_class_documentation
                add_method,
              File "/Users/adam/anaconda3/envs/vast-pipeline-dev/lib/python3.7/site-packages/pytkdocs/loader.py", line 599, in
            add_fields
                root_object.add_child(add_method(child_node))
              File "/Users/adam/anaconda3/envs/vast-pipeline-dev/lib/python3.7/site-packages/pytkdocs/loader.py", line 726, in
            get_django_field_documentation
                if prop.null:
            AttributeError: 'GenericForeignKey' object has no attribute 'null'
ERROR    -  Error reading page 'reference/forms.md':
ERROR    -  Could not collect 'vast_pipeline.forms'

and if you try and ignore this particular error you get to:
 AttributeError: 'ModelFormOptions' object has no attribute 'get_fields'

These errors all stem from the pytkdocs new addition to support Django models:
https://github.com/mkdocstrings/pytkdocs/blob/080079ff693190271733c4e08fb6fb4ddc0aa6ee/src/pytkdocs/loader.py#L724-L751
My understanding is that GenericForeignKey is not a standard Django model object and hence why it fails on this check.
But it's unfortunate that it's such a regression that a lot of files don't process the docstrings whereas they currently do at least generate something. I'm unsure if this is a mistake on the pipeline side or pytkdocs needs to be edited such that it has fallback options rather than just crashing out.
But currently pytkdocs should not be updated to > 0.11.1.


Evidence in an Academic Setting

My primary use-case for evidence.dev is to use it to publish archaeological fieldwork reports (and similar academic outputs[^1].) It would be nice to have an entirely self-contained workflow that I can encapsulate into a github action such that depositing a sqlite file into the repo is sufficient to generate something not wholly unlike my colleague's https://minicomp.github.io/wax/. In the best of all possible worlds, I'd be able to deploy this on a raspberry Pi (and/or a stone dvd[^2]) and use it as part of data repatriation to Indigenous communities here in Australia.
As such, being able to point to within-repo SQLite (and/or well-formed CSVs that are effectively SQLite tables) to run queries against would make it wonderfully self-contained and excellent for archival academic publishing.
[^1]: I'd love to see how it does against the TSVs scraped using the Latin Epigraphy scraper I made https://github.com/mqAncientHistory/Lat-Epig, but since that's tsv->sqlite...
[^2]: Archival media is really tedious. Being able to have FAIR data on archival media is even harder. I can see this being a major tool to display said archival media.

@Denubis this is fascinating. The map on your project page is 👌
Just some housekeeping: I am going to open an issue for CSV support and one for SQLite, and I am going to rename this to 'Evidence in an Academic Setting' and open it up to be more of a discussion.
Upfront, I'll also flag that today, Evidence is not the right answer for long-term archival of data :) The project is only about 80 days old, it's still very much in Alpha and it's built on top of a web framework which itself is very new.  Hopefully we can get it to that point though. I would be stoked if there were Evidence projects running on raspberry Pi's in Indigenous communities in Australia.
I took a look at WAX, and was really nodding my head on this piece:

The skills needed to create Wax sites are agnostic.
This means they are largely transferable for use in other digital projects. ‘Learning Wax’ does not mean learning how to use a platform.

I love that sentiment, and that's definitely how we want to build Evidence. The web is the platform, let's build tools that make it easy for data people to build great stuff for the web.
My biggest question for you is why are you thinking of Evidence for these purposes? You are obviously very adept with Jupyter. Why not Jupyter or WAX?

CSV Support: #107
SQLite Support: #108

@mcrascal Good questions.
Very much noting that this is in alpha :) (Having my own project in alpha, I rather understand the shape of things here...)
Let me rephrase your question as: "What tools are available to present data to a static site in an academic setting"
Many of my projects attempt to answer this question:

onwork.edu.au has paperpile -> python to recursively trawl paperpile citations -> generate many many many markdown pages -> serve on github pages. The upside is that it works. (Rather a big upside, TBH). However, it's clunky and needs a dedicated server and the hacks I needed to do for categories were tedious in the extreme. (Lots of list-writing to YAML). Having something that could write citations into a relational database (SQLite) to reduce the complexity of the presentation layer means that there would be a chance someone else might be able to maintain the frontend. Also, Jekyll/Ruby is... really frustrating and slow. The specific value proposition is that instead of writing liquid to iterate over lists generated by python writing categories and themes recusrively and having a 20 minute jekyll build, I could Select category from citation_category where citation_id = foo; and... be done with it. Massive reduction in complexity.
I'm working on a publication with @ewansc which is literally a SQLite db of roman inscriptions to an Open Context publication. It would be great to have a github pages SQL/GIS layer so my PhD student can show work-in-progress stuff to his other supervisors. (It turns out that supervisors aren't very keen on CSVs, much less SQL). We have tried to show this with datasette but the various hacks we had to do to get something that could run datasette from a url (I can't even find the messages where we figured out how to do it) were utter hacks. In many ways, this request is "What if Datasette, but for static sites?" -- I've also poked at https://phiresky.github.io/blog/2021/hosting-sqlite-databases-on-github-pages/ which... is a lovely proof of concept, but lacks the markdown formatting that you folks promise.
There are also some in-uni projects which would have benefited from this. In short, the need is "The ability to present structured, relational, data in a maintainable fashion" (Also maps. Always maps. ::sigh:: Let me know when you want to talk about GIS stuff)
As part of the major project I'm on, we're getting to the point where we need to think about data export. That project is using a NoSQL (CouchDB) backend, but archaeologists and field researchers expect their output in ... spreadsheets. Having a proof of concept "Here's your project report" as part of an export is very exciting to me indeed.

The idea of a Node project (As a note, they just bumped LTS up to 16 from 14. I've yet to transition my CI infrastructure, but... it's probably worth doing sooner rather than later) that is part of a robust archival workflow ... should be hilarious. But if I do anything serious with this I'll smush it into a docker image and ship that, to prevent dependency hell from breaking everything.

@denubis - this might be worth another look for you! We've added early support for maps, added sql lite, and duck db, might be worth another look! Closing this for now, but thank you again for all the great ideas early on.


Fixes #5871. Segfault when calling pyapi.call() passing args as None

As stated in the discussion on issue #5871 the fix was just a simple matter of replacing args = self.get_null_object() for args = self.tuple_new(0) at numba/core/pythonapi.py.
If you will, could you add hacktoberfest-accepted to the labels of this PR so it can count for me as a valid contribution to the festival?

Thanks for the patch. Given this is fixing a segfault I think the reproducer should be included as a test, probably in a new test module e.g. numba.tests.test_pyapi.py.

This needs to decref the resulting tuple after the call, doesn't it?

Yes, I think so.


Simplify blobstore tests, add coverge for BlobNotFoundError

Codecov Report

Merging #125 into master will increase coverage by 0.55%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #125      +/-   ##
==========================================
+ Coverage   88.39%   88.95%   +0.55%
==========================================
  Files          10       10
  Lines         724      724
==========================================
+ Hits          640      644       +4
+ Misses         84       80       -4

Impacted Files
Coverage Δ

ssds/blobstore/gs.py
93.10% <0.00%> (+0.86%)
:arrow_up:

ssds/blobstore/s3.py
98.57% <0.00%> (+2.14%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ac7433c...883f8b8. Read the comment docs.


2023 wiki link

Is there a way to view the wiki in its 2023 form? I thought I remembered seeing a link somewhere, but I searched google and Chief Delphi and couldn’t find it.

The usage example pages for 2023 PPLib are linked at the bottom of the wiki home page. If you need pages for the GUI, try the way back machine: https://web.archive.org/web/20230000000000*/github.com/mjansen4857/pathplanner/wiki


Fix surt browse

This is to fix #114 and #115.

Regarding #114: A user can nominate URLs of up to 300 characters in length. When nominating a URL, a surt attribute is created to hold in value a SURT form of the URL. The SURT form of an URL can be a few characters longer than the URL, so this PR increases the allowed length of value to 305 to accommodate the longest URLs' SURT forms.
Regarding #115: on a project's url listing page, top level domains are given as links to a listing of URLs matching the SURT prefix of the top level domain. The SURT prefix, should end in a comma, but the way we are using the top-level domain in our browse structure and with how we are displaying the link text, we don't look up SURT values currently with the comma which causes matches for .co to include .com etc. Adding a comma to the template link allows for going to the correct SURT prefix for the view.

ping @clarktr1 @somexpert this is ready for review

👍


Feat(linter): declare_oxc_lint proc_macro

Rules have metadata we want to access to, but it is tedious to implement a RuleMeta trait on every lint rule.
In this good first issue, we will follow the more difficult paths of following rust-clippy, e.g.
https://github.com/rust-lang/rust-clippy/blob/e1ac1332b668f584919aba86ad3e81f267251e9b/clippy_lints/src/almost_complete_range.rs#L11-L31
So we need to:

add a declare_oxc_lint proc_macro crate
define a RuleMeta trait
hook up the NoDebugger Rule with the proc macro
add some tests for this, maybe in the same file where the RuleMeta trait is.

The first prototype should probably just contain a name and a documentation. Other metadata can be added later.
See

Rome https://github.com/rome/tools/blob/5c60450e016318c761590e883fa33d780491d08c/crates/rome_analyze/src/rule.rs#L83
Ruff https://github.com/charliermarsh/ruff/blob/3179fc110d815d9b5cb9a3179bad6e1246a5bbf8/crates/ruff_macros/src/lib.rs#L29-L34
Clippy https://github.com/rust-lang/rust-clippy/blob/master/declare_clippy_lint/src/lib.rs https://github.com/rust-lang/rust-clippy/blob/e1ac1332b668f584919aba86ad3e81f267251e9b/clippy_lints/src/almost_complete_range.rs#L11-L31

Drop a comment if you wish to tackle this, so I can assign this issue to you.

@Boshen issue can be closed


Limited functionality of CalculableHook

Due to bug in OrientDB https://github.com/orientechnologies/orientdb/issues/4158 we have to disable calculation for newly created document. To take actual result: please reload object after creation.
To fix this issue in Orienteer just uncomment CalculablePropertiesHook.onRecordAfterCreate() when related issue in OrientDB will be fixed.

Related issue in OrientDB: https://github.com/orientechnologies/orientdb/issues/4575

No response from OrientDB guys: need to move to 1.2

Not yet implemented in OrientDB. Moving to next release


Move FileTarget.FileCompressor to NLog.FileCompressorWrapper

Remove the following dependencies from NLog-core:
<Reference Include="System.IO.Compression" />
See also #2729, #2296 and #1314
For NLog 6.0

Maybe also add the option to flip the NTFS-compression flag on file after it has been archived:
https://stackoverflow.com/questions/624125/compress-a-folder-using-ntfs-compression-in-net
https://blogs.msdn.microsoft.com/oldnewthing/20110622-00/?p=10363

I'll take it :) @snakefoot

@noamyogev84 I guess this up-for-graps is a little more tricky, since you probably have to implement this first:  #1249 (And it requires one to invent a great API that FileCompressorWrapper can hook into).

Got it.
Any other place I can contribute?
On Sun, Oct 4, 2020, 15:23 Rolf Kristensen<EMAIL_ADDRESS>wrote:

@noamyogev84 https://github.com/noamyogev84 I guess this up-for-graps
is a little more tricky, since you probably have to implement this first:
#1249 https://github.com/NLog/NLog/issues/1249 (And it requires one to
invent a great API that FileCompressorWrapper can hook into).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/NLog/NLog/issues/2732#issuecomment-703247801, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AB26MJJYHXX5E6A4BQHUEETSJBSLFANCNFSM4FA63A5Q
.

Guess #4032 could be a good candidate for first-issue.


Error when trying to display a figure from GLMakie through juliacall

Hi,
I'd like to call Julia code from python to display some interactive figures. MWE:
In [1]: from juliacall import Main as jl

In [2]: jl.seval("using GLMakie")

In [3]: jl.Figure()
Out[3]: Figure()

In [4]: f = jl.scatter(jl.rand(2,5))

In [5]: jl.display(f)
Out[5]: GLMakie.Screen(...)

A new Makie window shows up. But nothing in it. And after a while

Any idea how to make it work?

My guess is that, since Make is interactive, plotting happens asynchronously. Unfortunately, Julia's event loop only runs while you're actually running Julia code - once you're back in Python it stops.
Maybe calling jl.yield() several times from the prompt will work?

Thanks!

BTW I'm about to release a new version which has a function juliacall.interactive() which runs the Julia event loop in the background of the Python REPL. I don't know it will work in IPython, I imagine not.
Also, you should probably put a time.sleep(0.001) in that loop I suggested above, so as not to thrash your CPU.


Repo sync for protected CLA branch

The pull request is created from master637439344939153327 to master to fix git push error for protected CLA branch

@ghogen : Thanks for your contribution! The author(s) have been notified to review your proposed change. @evanwindom


remove legacy runtime

because:

seeing pretty decent adoption (edit: of kube-runtime) across github code search
haven't gotten complaints about new runtime
no need to keep this entirely unmaintained module around to confuse people

some counter-points. There is still matt butcher's blog-post that made it to hackernews which uses it, but that's hopefully passed now?
I have updated all my old blog posts (and even written a new one).
Can hold off until we have written something official about kube-runtime potentially.
What do people think?

i'll trust the thumbs up consensus :-)


fix(@embark/ens): fix hashing of node in lookup

Fixes #1626

Tests are failing on CI with this change. However it doesn't seem clear why. Restarted the build just to check if maybe CI is flaky.

@3esmit tests are definitely failing due to this change: https://dev.azure.com/embark-framework/Embark/_build/results?buildId=1079&view=logs&jobId=7d6c1058-6481-58d0-a804-c0782e5d5d94&taskId=6fa47090-aaed-5a41-5591-259e51cf39ea&lineStart=2335&lineEnd=2336&colStart=1&colEnd=1
PTAL

As the links to output lines in AzP don't seem to work, here's what tests are failing (for transparency-sake):
(Output line: 2228)
embark-dapp-test-app:   1) EmbarkJS functions
embark-dapp-test-app:        should have access to ENS functions:
embark-dapp-test-app:      Error: Address not associated to a resolver
embark-dapp-test-app:       at lookup (/home/vsts/work/1/s/packages/embark/dist/lib/modules/codeRunner/vm.js:411:15)
embark-dapp-test-app:       at <anonymous>
embark-dapp-test-app:       at process._tickCallback (internal/process/next_tick.js:189:7)

(Output line 2333):
embark-dapp-test-app:   1) ENS functions
embark-dapp-test-app:        should allow directives in ENS subdomains:
embark-dapp-test-app:      Error: Address not associated to a resolver
embark-dapp-test-app:       at lookup (/home/vsts/work/1/s/packages/embark/dist/lib/modules/codeRunner/vm.js:411:15)
embark-dapp-test-app:       at <anonymous>
embark-dapp-test-app:       at process._tickCallback (internal/process/next_tick.js:189:7)

Very well done @3esmit ! Merging


ONNX exported model count

The ONNX exporting process currently exports five separate models:

encoder.onnx
decoder.onnx
joiner.onnx
joiner_encoder_proj.onnx
joiner_decoder_proj.onnx

However, these models are just called sequentially, it doesn't look like there's much reason to keep them separate. It appears these models could be merged into just two .onnx models like so during the export:

"encoder": encoder+joiner_encoder_proj
"joiner": decoder+joiner_decoder_proj+joiner

Some sample code to illustrate what I mean:
class OnnxEncoder(nn.Module):
    def __init__(self, encoder, joiner):
        super().__init__()
        self.encoder = encoder
        self.encoder_proj = joiner.encoder_proj

    def forward(self, x, h, c):
        x_lens = torch.tensor([9], dtype=torch.int64)
        x, _, new_states = self.encoder(x, x_lens, (h, c), 1.0)
        x = self.encoder_proj(x)

        return x, new_states[0], new_states[1]

class OnnxJoiner(nn.Module):
    def __init__(self, decoder, joiner):
        super().__init__()
        self.decoder = decoder
        self.joiner = joiner

    def forward(self, context, encoder_out):
        decoder_out = self.decoder(context, False)
        decoder_out = self.joiner.decoder_proj(decoder_out)

        joiner_out = self.joiner(encoder_out, decoder_out, False)

        return joiner_out

This could greatly reduce the amount of code and complexity required to perform inference, from loading and calling 5 models to just 2. Is there any reason they're separate and not merged like that?

The reason is that encoder_proj and decoder_proj are inside the joiner network.
When exporting, we export each model independently.
We can change the export script by following your suggested code.
Would you mind making a PR?

I'll try and make a PR then

Thanks!

I realized it might not be a good idea to merge the decoder and joiner like I did in my sample. After a blank token is emitted, the subsequent call to the merged decoder+joiner with the new encoder output would not re-use the previous decoder output, even though the context is the exact same. The stateless decoder seems small so I'm not sure it would matter too much for performance though.

I realized it might not be a good idea to merge the decoder and joiner like I did in my sample. After a blank token is emitted, the subsequent call to the merged decoder+joiner with the new encoder output would not re-use the previous decoder output, even though the context is the exact same. The stateless decoder seems small so I'm not sure it would matter too much for performance though.

I think you can leave the joiner as it is. Just move decoder_proj from the joiner to decoder, like what you are doing for the encoder.

@abb128
Please have a look at https://github.com/k2-fsa/icefall/pull/879