CICD Improvements

About
this is a work in progress; it aims to check feasibility for further discussion
Following #93, further improve CI + CD
Tasks

[x] add coverage reports tasks and ci

[x] store all py vers X OS coverage files
[x] combine all py vers X OS coverage files
[x] create html cov
[x] store coverage report artifact

[x] add optional binary distribution to wheels via setup.py

[x] add a none-any build just in case and publish it
[x] add sdist build ( .tar.gz ) build for publishing

[x] store built wheels by tox

reason for publishing to tested wheels

[x] build wheels for all distributions ( matrix = py ver + os )
[x] build py3-none-any wheel + sdist ( .tar.gz )
[x] consolidation hera CICD into a single gh workflow

motivation re-use test / build job as a pre-publishing job
motivation to leverage artifacts passing between jobs rather than between different workflows
[x] add publish job ( similar to exiting one )
[x] change event triggers
[x] add if: to publish job
[x] delete file .github/workflows/hera_build_and_publish.yaml
[x] rename file .github/workflows/hera_pr.yaml -> .github/workflows/cicd.yaml ( or .. )
[x] fix jobs badge workflow name hera_build_and_publish.yaml -> cicd.yaml ( or .. ) in README.md

[x] enable upload to pypi-test for the purpose of validating entire publish job ( see publish job failure )
[x] replace upload to pypi-test with upload to pypi in publish job
[ ] setup codecov if we want a coverage badge to be introduced in this PR ( or propose a different solution )

inspiration credit https://github.com/samuelcolvin/pydantic/blob/master/.github/workflows/ci.yml

i love coverage and observability, what can I do
hera has 95% coverage now, which is great.
Unfortunately, it's not observable in CI and.or for developers and maintainers. So I gave a try of adding a report to CI...
To create a proper report, one needs to combine the coverage reports created by all py versions builds tests.
As you can see if you look inside -- this fails .. why ?
Astox runs the tests in isolated envs, with installation of hera, coverage runs on this installation ..
thus a report can't be created, as in the coverage-report gh-wf job these tox envs never existed.

uploading and downloading all these envs as artifacts is not an option afaic; they are too big

while this is annoying, it took me again to luigi to see how they dealt with it.
luigi uses codecov uploader to "solve this" but actually get much more from what I can see

note they are using it in a way that will soon be deprecated, so the reference is relevant only in part

codecov is free for public projects, and from what I can tell can integrate easily, "solves" this issue, and provide with some quite cool visibility features built into github -
https://docs.codecov.com/docs/github-checks
@flaviuvadan - I'd like to give covecov a try. what do you say ?
https://github.com/apps/codecov

@flaviuvadan - for some reason I can't mention https://github.com/hirosassa ; can you please add him to the discussion ?

@flaviuvadan - for some reason I can't mention https://github.com/hirosassa ; can you please add him to the discussion ?

@hirosassa 🙂

hera has 95% coverage now, which is great.

Oh, this is great!
I'll take a look about failure CI.

@hirosassa @flaviuvadan — as for the coverage combine issue, the following looks like a the solution —
https://github.com/nedbat/coveragepy/issues/597
https://github.com/nedbat/coveragepy/issues/597#issuecomment-554766758
i’ll gove it a try.
nevertheless— @flaviuvadan — would you like to install codecov so we can try it out ?
thanks,
harel

Hi @flaviuvadan
I've pushed a solution for coverage combine issue described above, with no use of codecov.
For some reason newly pushed commits to this PR do not trigger checks ( except the license check :\ ), and thus I can't verify it really works and continue with the rest of this PR tasks.
Do you maybe know why ?
Thanks,
Harel

hi @flaviuvadan @hirosassa

sorry for the previous "spam"; i've deleted the no longer irrelevant questions.

(1) coverage combine fixed
I've added a solution of the coverage combine issue. it took a bit longer then expected; eventually i used toxs own tox.ini as a ref - https://github.com/tox-dev/tox/blob/master/tox.ini which did the trick. you can see the result in https://github.com/argoproj-labs/hera-workflows/runs/5456858785?check_suite_focus=true
(2) next tasks and wrap up

let's agree on these

I'd like to consolidate hera's CICD into a single workflow file, for purposes of artifacts re-use, eliminating the current duplication of test/build using job dependencies etc.
I've already added a publish job, which currently only downloads the build artifacts ( wheels ) and lists them -
https://github.com/argoproj-labs/hera-workflows/runs/5456863640?check_suite_focus=true
These artifacts can be uploaded to pypi "as is" without first uploading to pypi test like the current CD.
This is thanks to tox which tests those build via install in (so-called) isolated envs.
Such a change will also require a -
(2.1) if condition
  publish:
    needs: [coverage]
    runs-on: ubuntu-latest
    if: "success() && (startsWith(github.ref, 'refs/tags/') || github.ref == 'refs/heads/master')"

(2.2) change of the workflow event triggers
on:
  push:
    branches:
      - main
  pull_request: {}

What say ya ?

This looks awesome 🙂 I left some questions mostly to clarify my own understanding of the contribution.

I still think there's a lot of value in first publishing to the Test PyPI as it ensures the whole installation and usage workflow works as expected. I am open to be convinced of this not being the case, though. I recall an issue in which a broken Hera package was published - the publication step and tests were successful because of accessibility to Hera via the editable mode, but the installed package from PyPI was broken because of some setup.py changes. Any thoughts @harelwa and @hirosassa?

Oh, another question if I may - does the consolidation in a single GHA workflow file support differentiation between PRs and main pushes to the degree that is necessary in order to support publication of PR artifacts, such as coverage, and then publish the Hera package independently when the PR is merged?

@hirosassa do you think we should merge this first and then https://github.com/argoproj-labs/hera-workflows/pull/94, maybe? I wonder what's the best order and mix of PRs that minimizes the effort you and @harelwa need to put in to merge these (due to conflicts).

Hi @flaviuvadan

I still think there's a lot of value in first publishing to the Test PyPI as it ensures the whole installation and usage workflow works as expected. I am open to be convinced of this not being the case, though. I recall an issue in which a broken Hera package was published - the publication step and tests were successful because of accessibility to Hera via the editable mode, but the installed package from PyPI was broken because of some setup.py changes. Any thoughts @harelwa and @hirosassa?

If you recall, I was the one to point the publish issue #51 where I also suggested a solution https://github.com/argoproj-labs/hera-workflows/issues/51#issuecomment-1010464186 and ended up hopefully contributing the suggestion in this
PR !
I'll try and explain why publishing to Test PyPi is an overhead with the CI flows suggested in this PR -
As we are using tox now, the actual wheels that are going to be published (90% true; I'll explain more below) are installed and tested thus you do not expect pitfalls like we saw in #51 usually involving using pytest on sources, rather on build due to PyTest PYTHONPATH Fun. I've we also twine check explicitly to verify pypi artifacts health.
having said that - of course we can add such a job  / step if you are still don't feel comfortable with that.
What I did want to to use Test PyPi is to "test" the real publish flow in in single GH workflow in this PR  ( more below; it's not pushed yet )

Oh, another question if I may - does the consolidation in a single GHA workflow file support differentiation between PRs and main pushes to the degree that is necessary in order to support publication of PR artifacts, such as coverage, and then publish the Hera package independently when the PR is merged?

(1) coverage
If we wish to add a coverage shield ( and we do :), we can have such "external" uploads taken care of in a different work flow, but actually also in the same one.
There are a few options of the coverage, e.g. -
luigi use codecove; see https://github.com/spotify/luigi/blob/master/.github/workflows/pythonbuild.yml
pydantic uses smokeshow; see https://github.com/samuelcolvin/pydantic/blob/master/.github/workflows/upload-previews.yml
(2) publishing
Indeed. I provided a snippet for how it's done in https://github.com/argoproj-labs/hera-workflows/pull/101#issuecomment-1061247450 (2.1) + (2.2)
refs

https://github.com/samuelcolvin/pydantic/blob/master/.github/workflows/ci.yml
https://github.com/samuelcolvin/pydantic/blob/master/.github/workflows/upload-previews.yml

@hirosassa do you think we should merge this first and then https://github.com/argoproj-labs/hera-workflows/pull/94, maybe? I wonder what's the best order and mix of PRs that minimizes the effort you and @harelwa need to put in to merge these (due to conflicts).

I tend to agree it would be wiser to merge this PR first

@harelwa thank you for the explanations 🙂 I feel like I have a much richer understanding now. I think this PR looks good and we can ship once the builds are fixed. Then we can focus on @hirosassa's Poetry contribution and make that fit with the contributions here!

Hi @flaviuvadan + @hirosassa
I think we are pretty much done here.
Here are the final leftovers:

[x]  delete file .github/workflows/hera_build_and_publish.yaml
[x]  rename file .github/workflows/hera_pr.yaml -> .github/workflows/cicd.yaml ( or .. )
[x]  fix jobs badge workflow name hera_build_and_publish.yaml -> cicd.yaml ( or .. ) in README.md
[x]  remove the || true from publish job if: ( added so publish runs as part of this PR for wf validation )
[ ]  enable upload to pypi-test for the purpose of validating entire publish job ( see publish job failure )
[ ]  replace upload to pypi-test with upload to pypi in publish job
[ ]  setup codecov if we want a coverage badge to be introduced in this PR ( or propose a different solution )
[ ]  question  do you want to add windows-latest to the os matrix ?

Thanks,
Harel

Hi @flaviuvadan -
Handing this over to you as all remaining tasks require maintainer permissions, access to secrets etc.
As for the merge process to main, here are  my 3 cents --
Integrate as is -> [ it will ] publish to pypi-test -> validate upload manually -> change to pypi official
Harel

Thank you for the efforts @harelwa 🙂 I was on vacation for a week and am just getting back! I am going to merge this PR today and add the necessary secrets.

Hi @flaviuvadan
I see the unfortunate failures.
Following -
https://stackoverflow.com/questions/59451069/binary-wheel-cant-be-uploaded-on-pypi-using-twine
https://peps.python.org/pep-0513/#rationale
I see auditwheel can fix that.
I'll give it a try

Hi @flaviuvadan
I see the unfortunate failures.
Following -
https://stackoverflow.com/questions/59451069/binary-wheel-cant-be-uploaded-on-pypi-using-twine
https://peps.python.org/pep-0513/#rationale
I see auditwheel can fix that.
I'll give it a try

Thank you for being on top of this, @harelwa! I started reading about manylinux and cibuildwheel myself to diagnose the issue and think about a fix.

Hi @flaviuvadan
I see the unfortunate failures.
Following -
https://stackoverflow.com/questions/59451069/binary-wheel-cant-be-uploaded-on-pypi-using-twine
https://peps.python.org/pep-0513/#rationale
I see auditwheel can fix that.
I'll give it a try

Thank you for being on top of this, @harelwa! I started reading about manylinux and cibuildwheel myself to diagnose the issue and think about a fix.

Hi @flaviuvadan -
The immediate fix with minimal changes is to not publish the platform wheels.
You can achieve this by removing / commenting out the upload step in test job.
Then only none-any will be published ( as did thus far ).
As for repairs etc. -- perhaps for later --
When trying to repair with auditwheel I get -
>$ auditwheel repair ./tox/dist/hera_workflows-2.2.1-cp39-cp39-linux_x86_64.whl
INFO:auditwheel.main_repair:Repairing hera_workflows-2.2.1-cp39-cp39-linux_x86_64.whl
INFO:auditwheel.main_repair:This does not look like a platform wheel

as for the discussion in https://github.com/pypa/auditwheel/issues/247 -- a bit odd to me as libc et. al. issues are a matter of python it self, not only compiled code in the package...
From what I can see cibuildwheel can also be used; note it requires docker
What do you think ?

You can achieve this by removing / commenting out the upload step in test job -

I agree with the proposed approach to comment out the upload steps for now! Let's do that, and then investigate how we can introduce cibuildwheel as a package to build wheels with.

You can achieve this by removing / commenting out the upload step in test job -

I agree with the proposed approach to comment out the upload steps for now! Let's do that, and then investigate how we can introduce cibuildwheel as a package to build wheels with.

great. will you take care of it or you'd rather me open a PR ?

Hi! Does hera have any compiled extensions in the wheel? If not, I think the stock py3-none-any wheel will cover all platforms automatically and we can skip any cibuildwheel stuff. Quoting from the setuptools docs:

Platform Wheels are wheels that are specific to a certain platform like Linux, macOS, or Windows, usually due to containing compiled extensions.
The wheel package will detect that the code is not pure Python, and build a wheel that’s named such that it’s only usable on the platform that it was built on. For details on the naming of wheel files, see PEP 425.

And testing a bit, everything looks good:
$ sh -c 'pip3 install wheel && pip3 install hera-workflows' | grep hera_workflows # Apple M1
  Downloading hera_workflows-2.2.1-py3-none-any.whl (41 kB)
$ docker run --rm -it --platform=linux/x86-64 python:3.7 sh -c 'pip3 install wheel && pip3 install hera-workflows' | grep hera_workflows
  Downloading hera_workflows-2.2.1-py3-none-any.whl (41 kB)
$ docker run --rm -it --platform=linux/x86-64 python:3.9 sh -c 'pip3 install wheel && pip3 install hera-workflows' | grep hera_workflows
  Downloading hera_workflows-2.2.1-py3-none-any.whl (41 kB)
$ docker run --rm -it --platform=linux/amd64 python:3.7 sh -c 'pip3 install wheel && pip3 install hera-workflows' | grep hera_workflows
  Downloading hera_workflows-2.2.1-py3-none-any.whl (41 kB)
$ docker run --rm -it --platform=linux/amd64 python:3.9 sh -c 'pip3 install wheel && pip3 install hera-workflows' | grep hera_workflows
  Downloading hera_workflows-2.2.1-py3-none-any.whl (41 kB)

--
I think this might also explain what @harelwa saw with:
>$ auditwheel repair ./tox/dist/hera_workflows-2.2.1-cp39-cp39-linux_x86_64.whl
INFO:auditwheel.main_repair:Repairing hera_workflows-2.2.1-cp39-cp39-linux_x86_64.whl
INFO:auditwheel.main_repair:This does not look like a platform wheel

-> there is no compiled code?

Hi, apologies for jumping in! Does hera have any compiled extensions in the wheel? If not, I think the stock py3-none-any wheel will cover all platforms automatically and we can skip any cibuildwheel stuff. Quoting from the setuptools docs:

Platform Wheels are wheels that are specific to a certain platform like Linux, macOS, or Windows, usually due to containing compiled extensions.
The wheel package will detect that the code is not pure Python, and build a wheel that’s named such that it’s only usable on the platform that it was built on. For details on the naming of wheel files, see PEP 425.

And testing a bit, everything looks good:
$ sh -c 'pip3 install wheel && pip3 install hera-workflows' | grep hera_workflows # Apple M1
  Downloading hera_workflows-2.2.1-py3-none-any.whl (41 kB)
$ docker run --rm -it --platform=linux/x86-64 python:3.7 sh -c 'pip3 install wheel && pip3 install hera-workflows' | grep hera_workflows
  Downloading hera_workflows-2.2.1-py3-none-any.whl (41 kB)
$ docker run --rm -it --platform=linux/x86-64 python:3.9 sh -c 'pip3 install wheel && pip3 install hera-workflows' | grep hera_workflows
  Downloading hera_workflows-2.2.1-py3-none-any.whl (41 kB)
$ docker run --rm -it --platform=linux/amd64 python:3.7 sh -c 'pip3 install wheel && pip3 install hera-workflows' | grep hera_workflows
  Downloading hera_workflows-2.2.1-py3-none-any.whl (41 kB)
$ docker run --rm -it --platform=linux/amd64 python:3.9 sh -c 'pip3 install wheel && pip3 install hera-workflows' | grep hera_workflows
  Downloading hera_workflows-2.2.1-py3-none-any.whl (41 kB)

--
I think this might also explain what @harelwa saw with:
>$ auditwheel repair ./tox/dist/hera_workflows-2.2.1-cp39-cp39-linux_x86_64.whl
INFO:auditwheel.main_repair:Repairing hera_workflows-2.2.1-cp39-cp39-linux_x86_64.whl
INFO:auditwheel.main_repair:This does not look like a platform wheel

-> there is no compiled code?

Thank you @JacobHayes
Yes. I agree. I think we can drop the platform wheels all together.


Not ready to be used as a (static) library in projects

Hi!
It would be nice if this nice project would be shaped in a way to be usable as a static library in other projects. Specifically, some makefiles included for some platforms/compilers (as a starting point); simple directory structure added; and a release be prepared with corresponding tarball. That would enable to reference the tarball to be downloaded and compiled wherever needed?
(I'm looking at a possibility to use it in LibreOffice, and seeing a sample makefile here would allow me to try and come back with whatever improvements we could create downstream; but the task to create this from scratch and keep a LO-own hosted tarball, without knowing that our changes would be accepted upstream, would mean supporting our own fork, which is not good for us.)
Thank you very much for this piece of software!

Thanks for the compliment :-)
I am not sure it is worth the work to make a library out of so few functions.
Also, a static library meant we have to lock-in some #defines, losing flexibility.
Example, below is a makefile, assuming FE_TONEAREST, and not using FAST_BF96
(FAST_BF96 does the scaling in 1 step, instead of upto 8, at the cost of a 8k lookup table)
CC = gcc -Wall -std=gnu99
CFLAGS = -c -O2 -Id:/cpp -Ic:/ $(ARG)
LIB = libdtoafast.a
DEL = rm -f

all:            $(LIB)
clean:; @$(DEL) *.o

OBJS = dtoa-fast.o dtoa-ifmt1.o

$(LIB): $(OBJS)
        $(DEL) $(LIB)
        ar rc $(LIB) $(OBJS)

# Assumed FE_TONEAREST, GMP for ULP tie-breaker
dtoa-fast.o: dtoa-fast.c dtoa-aux.c strtod-fast.c strtod-aux.c strtod-gmp.c
        $(CC) $(CFLAGS) -DUSE_MAPM=0 dtoa-fast.c

# DTOA_IFMT_G defined: 1.0 -> "1.0"
dtoa-ifmt1.o: dtoa-ifmt1.c
        $(CC) $(CFLAGS) -DDTOA_IFMT_G dtoa-ifmt1.c

This showed makefile work as expected
d:\cpp\dtoa> make
gcc -Wall -std=gnu99 -c -O2 -Id:/cpp -Ic:/  -DUSE_MAPM=0 dtoa-fast.c
gcc -Wall -std=gnu99 -c -O2 -Id:/cpp -Ic:/  -DDTOA_IFMT_G dtoa-ifmt1.c
rm -f libdtoafast.a
ar rc libdtoafast.a dtoa-fast.o dtoa-ifmt1.o

And, to test the static library, using 2 examples from dtoa-fast.txt
d:\cpp\dtoa> run igcc.py -Ic:\mapm -idtoa-fast.h -L. -ldtoafast -lgmp-10

gcc:1> double x; char *s; int sgn, len, dec
gcc:2> x = strtod_fast("4.3915445623487279e17", NULL)
gcc:3> s = dtoa_fast(x, 0, &sgn, &len, &dec)
gcc:4> puts(dtoa_ifmt(s, sgn, len, dec, 'g'))
439154456234872770.0

gcc:5> s = dtoa_fast(x, 10, &sgn, &len, &dec)
gcc:6> puts(dtoa_ifmt(s, sgn, len, dec, 'g'))
4.391544562e+017

gcc:7> x = strtod_fast("4.502463450621225E-321", NULL)
gcc:8> s = dtoa_fast(x, 0, &sgn, &len, &dec)
gcc:9> puts(dtoa_ifmt(s, sgn, len, dec, 'g'))
4.5e-321

Both results, behind the scene
"4.3915445623487279e17"
0x0.9c04dd78,8a7b2f00,00000000p+56      mantissa only
0x0.c30614d6,ad19fac0,00000000p+59      scaled binary
        round down
0x0.c30614d6,ad19f800,00000000p+59      rounded double
0x0.9c04dd78,8a7b2ccc,cccccccbp+56      mantissa only

mantissa =><PHONE_NUMBER>3487276 + 0.79999999999836291
safe ulp => 3.1999999998137358
safe =<PHONE_NUMBER>3487274 to<PHONE_NUMBER>3487279

edge = .4391544562348728E18             unable to round-trip
0x0.f9a1625a,772b8000,00000000p+52      mantissa only
0x0.c30614d6,ad19fc00,00000000p+59      scaled binary

best =<PHONE_NUMBER>3487277
+0x1.860c29ad5a33fp+58 = +4.3915445623487277e17

"4.502463450621225E-321"
0x0.ffef7769,6a529000,00000000p+52      mantissa only
0x0.e3d3c286,4c639dbc,50b85aa6p-1064    scaled binary
0x0.00000000,001c7a78,50c98c73p-1021    fix subnormal
        round down
0x0.e3c00000,00000000,00000000p-1064    rounded double
0x0.9fe7cafe,7ef5582c,ab1387c1p+56      mantissa only

mantissa => 4500 + 0.93803361375601735
safe ulp => 2.4703282290624431
safe = 4499 to 4503
best = 45
+0x0.000000000038fp-1022 = +4.5e-321

Thanks for the compliment :-)

This is well-deserved. You had put your expertise in it, to create great functions!

I am not sure it is worth the work to make a library out of so few functions.

It does worth it. Suppose someone wants to use it; one may take your code directly, and use in one's projects by copy-pasting, and then one would need to check from time to time if there's an update here addressing some issue, or improving something maybe (e.g., increasing performance even more). Or one could reference the library from one's in their projects, and have its up-to-date code on builds. Take into account that the functions, although "few", include a difficult-to-understand math, that not everyone could support in their projects by themselves.

Also, a static library meant we have to lock-in some #defines, losing flexibility.

No. The makefiles could (must!) take passed arguments into account, thus relying on user passing correct arguments. The makefiles should check the passed arguments (environment variables), and if the defiles are missing, then assume some defaults, but otherwise use the passed values.

Example, below is a makefile, assuming FE_TONEAREST, and not using FAST_BF96
(FAST_BF96 does the scaling in 1 step, instead of upto 8, at the cost of a 8k lookup table)

Thank you! This partially answers to my doubt (I always doubted if FAST_BF96 only sacrifices code size, and not accuracy of the result; if it actually means the fastest possible code at the cost of the size, and without any inaccuracies introduced). If using FAST_BF96 is the fastest, in my use case, I would be using the fastest mode (the 8K table doesn't matter here), in no-GPL mode (LibreOffice use MPLv2, so we can't incorporate GPL code).

...
dtoa-fast.o: dtoa-fast.c dtoa-aux.c strtod-fast.c strtod-aux.c strtod-gmp.c
...

IIUC you #include dtoa-aux.c and strtod-aux.c into dtoa-fast.c and strtod-fast.c resp., so are they needed as separate units, and shouldn't they be .h instead of .c? And why dtoa-fast.c includes strtod-fast.c? Shouldn't the common defines be moved into an own .h?

The MAPM library used here includes dtoa-* files that likely belong here, not to MAPM? And supposedly, MAPM should also be a static library usable from here? So that one could use both libraries in the project, and build MAPM first with the chosen configuration, then build this one (passing the MAPM separate include directory - absent now - to dtoa includes), and then link both libraries with the resulting program?

IIUC you #include dtoa-aux.c and strtod-aux.c into dtoa-fast.c and strtod-fast.c resp.,
so are they needed as separate units, and shouldn't they be .h instead of .c?

I were never sure what belongs to .c and .h :grin:
The original goal of the project were to ease understanding of David Gay's dtoa.c,
a BIG file including everything, even arbitrary precision math calculations.
Breaking up the code into units only to keep things simple.
  I found many dtoa.c bugs, in 2016. https://www.netlib.org/fp/changes
BTW, BF96_FAST were David Gay's idea, added to dtoa.c on April 29, 2016
His version had BF96 as default, but can be turned off by #define NO_BF96
I saved his email to me, dated May 5, 2016 (below only a snippet)
I've now updated

    http://www.ampl.com/netlib/fp/dtoa.c

to incorporate the above bug fixes and to add my rather different
way of using 64-bit integer arithmetic, using a large array of
powers of 10 in a sort of 96-bit floating-point encoding.  The
resulting dtoa() and strtod() routines are sometimes faster and
sometimes slower than your routines, but usually much faster
than my earlier dtoa.c.  There's also a new dtoa() variant, dtoa_r(),
which adds two trailing arguments, char *buf and size_t blen.  When
buf is not NULL and blen (the size of buf) is large enough, buf is
returned containing the desired string.  I've updated
http://www.ampl.com/netlib/ampl/solvers/printf.c to use dtoa_r()
rather than dtoa().  It's nonstandard in that %.0g gives the mode
0 conversion.  (Many years ago, I urged this upon the C standards
committee, but my suggestion fell on deaf ears.)

And why dtoa-fast.c includes strtod-fast.c?

I guess using #include to pull pieces together is not optimal, but I were lazy to create makefiles.
For applications that need the code, I just do #include "dtoa\dtoa-fast.c"
As to why dtoa_fast() require strtod_fast(), think edges.
For rare cases, the edge might require less digits to represent the number.
However, edge might not round-trip back to the same number.
Example: strtod_fast("7.9999999999999993e+023", NULL) =+0x1.52d02c7e14af6p+79
The edge, "8E23" is much shorter, but can only be used if it round-trip back to above number
Luckily it does, which can be check by their hex-floats.
+0x1.52d02c7e14af6p+79 = 7.99999999999999932891136E+23
+0x1.52d02c7e14af68p+79 = 8e+23, with half-way-to-even, round-back to above.
To speedup calculations, dtoa_fast() does not actually convert the 54-bits hex-float.
Instead, it call strtod_fast() to convert "8E23", and check for round-trip.

The MAPM library used here includes dtoa-* files that likely belong here, not to MAPM?

MAPM libray also uses dtoa-*.h files for MAPM numbers to IEEE Double (see mapm_dbl.c)
Instead of 2 copies of same header files to different projects,  I leave it to MAPM.
This meant MAPM is self-contained, with makefile to create its static library.
(Hopefully, you can just download it, and simply run "make")
On the other hand, dtoa-fast is not self-contained, even with the header files moved here.
It require arbitrary precision math package to break ties, for close to half-way case.
I had setup defaults so that dtoa-fast uses MAPM static library, libmapm.a, for this.
(to use GMP instead, add #define USE_MAPM 0)

MAPM libray also uses dtoa-*.h files for MAPM numbers to IEEE Double (see mapm_dbl.c)
Instead of 2 copies of same header files to different projects, I leave it to MAPM.
This meant MAPM is self-contained, with makefile to create its static library.
(Hopefully, you can just download it, and simply run "make")

Understood; but then the files likely should be named mapm_*, and referenced from dtoa as such, to make things clear? Also any dtoa-specific bits should be removed from there (like declaration of functions exported from dtoa)?

Understood; but then the files likely should be named mapm_*, and referenced from dtoa as such, to make things clear? Also any dtoa-specific bits should be removed from there (like declaration of functions exported from dtoa)?

Done.
DTOA : added declarations-only dtoa-fast.h (actually copied from your include directory)
MAPM: renamed the shared header file as mapm-dtoa.h


YOUTUBE API ERROR: ClientException: XMLHttpRequest error.,

I am trying to play a youtube video but it gives me errors.

How to solve the cors issue in production?


Redundant code in test.py

Hi Chengan,
Please check out this line:
https://github.com/vita-epfl/CrowdNav/blob/64f4f412245ddb4de138de5708075d97d02b5034/crowd_nav/test.py#L78
I don't think you will need it (checking if it is robot.visible).
Best,
Tessa

Hi Tessa,
This is actually not redundant. The invisible case breaks the reciprocal assumption and that's why I added safety space to make up for that. The value of the safety space in the invisible case is supposed to be non-zero, and users can tune this value. Somehow I set it to zero, but I will add one comment to make it clear.
Thanks,
Changan


Use rollup.

This change does the following:

Convert to rollup for bundling. Remove browserify.
Use "extend: true" to extend a global "tf" object. There is now no "tf_converter".
Ship es6 in a dist-es6 folder as well as es5.
Remove all index.ts files. They are not needed and just introduce another layer of indirection.

Converter here (rollup for all other packages has been released):
https://storage.googleapis.com/dljs-test/rollup/tf-converter.js

This change is

At the bottom of the README file. https://github.com/tensorflow/tfjs-converter/blob/master/README.md,
instead of generate for commonjs
node_modules/protobufjs/bin/pbjs -t static-module -w commonjs -o src/data/compiled_api.js --no-create --no-encode --no-verify --no-convert --no-delimited --no-beautify src/data/api.proto
replace with es6
node_modules/protobufjs/bin/pbjs -t static-module -w es6 -o src/data/compiled_api.js --no-create --no-encode --no-verify --no-convert --no-delimited --no-beautify src/data/api.proto
Is the clean up of the index.ts required for rollup, can that change be delayed to a separate PR to reduce the complexity?

Review status: 0 of 50 files reviewed at latest revision, all discussions resolved, some commit checks failed.

src/executor/graph_executor.ts, line 21 at r1 (raw file):

operations

This is the wrong import, it should be operations.executeOp. It happened on other locations as well.

Comments from Reviewable

Rollup was having issues with the multiple layers of exports, especially through the commonjs module.
Got the build working now.
When I run "yarn test" locally, only 1 test runs -- is this a bug?

Review status: 0 of 55 files reviewed at latest revision, 1 unresolved discussion, some commit checks broke.

src/executor/graph_executor.ts, line 21 at r1 (raw file):
Previously, pyu10055 (Ping Yu) wrote…

operations

This is the wrong import, it should be operations.executeOp. It happened on other locations as well.

Hmm.. not sure how this is wrong. I removed the index, so I'm importing executeOp directly from operation_executor.

Comments from Reviewable

I signed it!

This is ready for review now!

thanks for fixing this!

Review status: 0 of 55 files reviewed at latest revision, 1 unresolved discussion, some commit checks broke.

rollup.config.js, line 28 at r2 (raw file):

json

remove?

demo/package.json, line 9 at r2 (raw file):

tensorflow

change local file dependency to npm version, maybe also update core to 0.11.0

scripts/build.sh, line 26 at r2 (raw file):

compiled_api

for es6 it should copy the compiled_api.js in the data/es6 directory, it also need compiled_api.d.ts
I add this change in the package.json build script section, maybe it should be calling this to reduce the duplication.

src/executor/frozen_model.ts, line 21 at r2 (raw file):

proto

should this be reverted back to import tensorflow directly from compiled_api?

src/executor/graph_executor.ts, line 21 at r1 (raw file):
Previously, nsthorat (Nikhil Thorat) wrote…
Hmm.. not sure how this is wrong. I removed the index, so I'm importing executeOp directly from operation_executor.

It was importing from creation_executor, I changed it in one of my submit.

Comments from Reviewable

Review status: 0 of 55 files reviewed at latest revision, 5 unresolved discussions, some commit checks broke.

Comments from Reviewable

couple minor comments, please fix those before merge. Thanks.

Review status: 0 of 55 files reviewed at latest revision, 5 unresolved discussions, some commit checks broke.

Comments from Reviewable

Review status: 0 of 55 files reviewed at latest revision, 5 unresolved discussions, some commit checks broke.

rollup.config.js, line 28 at r2 (raw file):
Previously, pyu10055 (Ping Yu) wrote…

json

remove?

Done.

demo/package.json, line 9 at r2 (raw file):
Previously, pyu10055 (Ping Yu) wrote…

tensorflow

change local file dependency to npm version, maybe also update core to 0.11.0

Done.

scripts/build.sh, line 26 at r2 (raw file):
Previously, pyu10055 (Ping Yu) wrote…

compiled_api

for es6 it should copy the compiled_api.js in the data/es6 directory, it also need compiled_api.d.ts
I add this change in the package.json build script section, maybe it should be calling this to reduce the duplication.

Deleted this file since you added it to package.json.

src/executor/frozen_model.ts, line 21 at r2 (raw file):
Previously, pyu10055 (Ping Yu) wrote…

proto

should this be reverted back to import tensorflow directly from compiled_api?

Done.

Comments from Reviewable


Bug pagination UI

all tables in tomomaster

Same a fixed issue in tomoscan: https://github.com/tomochain/tomoscan/issues/689


[Update term] More interlinking for containerisation for Japanese

Describe your changes

Applied the #2909 changes to the Japanese version as well

Related issue number or link (ex: resolves #issue-number)
no issue
Checklist before opening this PR (put x in the checkboxes)

[x] This PR does not contain plagiarism

don’t copy other people’s work unless you are quoting and contributing it to them.

[x] I have signed off on all commits

signing off (ex: git commit -s) is to affirm that commits comply DCO. If you are working locally, you could add an alias to your gitconfig by running git config --global alias.ci "commit -s".

The target pages on the deploy preview site:

コンテナ化: https://deploy-preview-2964--cncfglossary.netlify.app/ja/containerization/
コンテナオーケストレーション: https://deploy-preview-2964--cncfglossary.netlify.app/ja/container-orchestration/
継続的デリバリー: https://deploy-preview-2964--cncfglossary.netlify.app/ja/continuous-delivery/


Awesome biohacking

Hi,
I would suggest this repository to the list:
https://github.com/HeitorG/awesome-biohacking/

https://github.com/sindresorhus/awesome/blob/master/create-list.md

Thanks


typos

i have fixed some typos

@cdwijs There are other countries in the world than USA. I'm Australian, hence the Australian spelling.


NETOBSERV-1616: collect flow metric object from netobserv ns

Release note:
Add flowmetric object to MG collection
Note: PR assumes flowmetric object is in the same NS defined in fc object

/approve


Car following project

Hello,
I am working on a project and i want to simulate a leader car in front of my car and i want to know how i can do this with carla
Thanks

Hi @hamdii02
Better to take a look at the scenario runner project. It has an example called FollowLeadingVehicle. You can get how to run, from its docs.
Good luck.

@hita-bm ok thanks for responding and i will take a look on it

@hamdii02 You're welcome. I recommend using Carla 0.9.6 and Scenario Runner 0.9.6 because the latest version (0.9.7) still has some bugs and needs to be completed. Trust me!

@hita-bm  that's exactly what am using  and i have already tried the 0.9.7 version and i've had a lot of bugs but the 9.6.0 is good for now


Adding db impl for webhooks

Overview
This change adds the db implementation for webhooks
Description
Webhooks are defined by having a noun verb pairing instead of id. This way one can easily see all webhooks accosiated with a noun verb pairing at any time. Also, since these noun-verb look ups will happen on every request it makes it a constant lookup on each request instead of scanning
Routes
PUT
http://localhost:8080/v1/webhooks
{
  "noun": "Manifest",
  "verb": "Create",
  "url": "https://www.tbd.website/"
}

Response
{
    "webhook": {
        "noun": "Manifest",
        "verb": "Create",
        "urls": [
            "https://www.tbd.website/"
        ]
    }
}

GET
http://localhost:8080/v1/webhooks/Manifest/Create
{
    "webhook": {
        "noun": "Manifest",
        "verb": "Create",
        "urls": [
            "https://www.tbd.website/"
        ]
    }
}

Note that if you create another webhook with the same noun verb pairing and a different URL, and then do a GET you will get this type of response object. (One webhook object with many urls in array)
GET
http://localhost:8080/v1/webhooks/Manifest/Create
{
    "webhook": {
        "noun": "Manifest",
        "verb": "Create",
        "urls": [
            "https://www.tbd.website/",
            "https://www.otherwebsite.com/"
        ]
    }
}

Since these noun-verb look ups will happen on every request it makes it a constant lookup on each request instead of scanning db.
Currently I have implement DID.CREATE and CREDENTIAL.CREATE, CREDENTIAL.DELETE
Here is the output from a small test nodejs service that recieves the webhook fired by the SSI-Service:
{
  noun: 'DID',
  verb: 'Create',
  url: 'http://host.docker.internal:4321/',
  data: {
    did: {
      '@context': 'https://www.w3.org/ns/did/v1',
      id: 'did:key:z6Mkt11tgxpEBtFPde1T2nBZ7YzR21FejGfHKWEdtRc4MW57',
      verificationMethod: [Array],
      authentication: [Array],
      assertionMethod: [Array],
      keyAgreement: [Array],
      capabilityDelegation: [Array]
    },
    privateKeyBase58: '3cQuZzFFnv2By6VyZvihmDpGLjsEBBdZpnapGuRjEPXV7JppPvFquVVNzvESvfqMtcSaocLorAeShyEvtM5Fk729',
    keyType: 'Ed25519'
  }
}

Codecov Report

Merging #298 (06d9464) into main (1cadf75) will increase coverage by 0.00%.
The diff coverage is 4.16%.

@@           Coverage Diff           @@
##             main     #298   +/-   ##
=======================================
  Coverage   23.96%   23.97%
=======================================
  Files          33       33
  Lines        3325     3357   +32
=======================================
+ Hits          797      805    +8
- Misses       2408     2432   +24
  Partials      120      120

Impacted Files
Coverage Δ

pkg/server/router/credential.go
3.46% <0.00%> (-0.05%)
:arrow_down:

pkg/server/router/did.go
7.76% <0.00%> (-0.16%)
:arrow_down:

pkg/server/router/webhook.go
7.69% <0.00%> (+7.69%)
:arrow_up:

pkg/server/server.go
80.69% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Updated so that a new webhook middleware is configured on a per route basis:
Example Flow:
Create new webhooks
PUT - http://localhost:8080/v1/webhooks
{
  "noun": "DID",
  "verb": "Create",
  "url": "http://host.docker.internal:4321/"
}

{
  "noun": "DID",
  "verb": "Delete",
  "url": "http://host.docker.internal:4321/"
}

This basically is configuring the SSI-Service to post the response body to the configured url (http://host.docker.internal:4321/) when one of these events happen. (In this case when a DID is created or deleted)
now when someone creates a did:
PUT - http://localhost:8080/v1/dids/:method
My test nodejs service that is accepting requests from http://host.docker.internal:4321/ gets a POST from the SSI-Service in the form of:
{
  noun: 'DID',
  verb: 'Create',
  url: 'http://host.docker.internal:4321/',
  data: '{"did":{"@context":"https://www.w3.org/ns/did/v1","id":"did:key:z6MkuN2g4DAEYjnTQwHYMYftszfFRJrTK9rxzHeehAYogZDb","verificationMethod":[{"id":"#z6MkuN2g4DAEYjnTQwHYMYftszfFRJrTK9rxzHeehAYogZDb","type":"Ed25519VerificationKey2018","controller":"did:key:z6MkuN2g4DAEYjnTQwHYMYftszfFRJrTK9rxzHeehAYogZDb","publicKeyBase58":"FumdTxuoDCHzJSSqfyi42u7FbjabuGccJGjirtanmLSD"}],"authentication":[["#z6MkuN2g4DAEYjnTQwHYMYftszfFRJrTK9rxzHeehAYogZDb"]],"assertionMethod":[["#z6MkuN2g4DAEYjnTQwHYMYftszfFRJrTK9rxzHeehAYogZDb"]],"keyAgreement":[["#z6MkuN2g4DAEYjnTQwHYMYftszfFRJrTK9rxzHeehAYogZDb"]],"capabilityDelegation":[["#z6MkuN2g4DAEYjnTQwHYMYftszfFRJrTK9rxzHeehAYogZDb"]]},"privateKeyBase58":"3p6wbbJ3egoDXreF3MRsRWBNUguFoKSakxWdUgb2w8JsCnqZuZT1Nv5kT2EopHoqjEmQHqPTXBQjPgJbJUxZY5ub","keyType":"Ed25519"}'
}


[docs] Adding helm-docs step in CI and README.md

What is the purpose of the change
Auto-generate documentation from helm charts into markdown files.
Brief change log

Added README.md done with helm-docs
Added CI step to verify helm-docs changes are needed.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changes to the CustomResourceDescriptors: no
Core observer or reconciler logic that is regularly executed: no

Documentation

Does this pull request introduce a new feature? no

Hi! Could you please explain why you think this is a good addition? In general we should try to keep the README small and link the docs instead.

Hi! Thank you for your question!
In the current project there is no README from Helm charts and it's needed to deep into the values.yaml from the Helm Chart to check the different values or check the official documentation, for instance the table located here
Using helm-docs tool would auto-generate documentation from helm charts into markdown easily and the CI step would avoid to push Helm changes without update the documentation.
I think this change will increase the Helm project documentation with an easy step and it will allow to explore documentation near to the Helm Chart code. But I understand your point about link the docs

It would be great to integrate this into the regular docs but the problem I see is that this is not actually “documentation “ .
it only lists the possible config parameters but not what they do.
Even if we don’t always document new parameters at least the current docs explain what they do :)

The description could be added to the current values.yaml based on the helm-docs documentation and it would be pretty similar to the table description here.
If you think it would be valuable to add a description for each parameter in the README to take this PR into account I can add comments to the values.yaml to have the parameters description in the Helm Chart and in the README.
Happy to collaborate if that makes sense to you

I think this makes sense but what about the following way:

Add the current parameter descriptions to the values yaml from the docs.
Remove current docs table
Generate docs table with the plugin
Add link to generated docs from main readme

Also I was wondering is this plugin safe to use with the Apache GitHub credentials?

@kaskol10 could you please address the above comments from @gyfora?


Tooltips: Should not overlap

What is the expected behavior?
I'm not sure if this is a bug or expected behavior. I'm proposing that tooltips don't overlap if you happen to have your mouse positioned onscreen and use keyboard to activate tooltip.
What is the current behavior?
Currently you can get tooltips to overlap by using a mouse to set focus and the keyboard focus at the same time. Example is shown with buttons.
What are the steps to reproduce?
https://stackblitz.com/edit/angular-c5w9uu .

Set your mouse on one of the move buttons, then use keyboard to tab through buttons. Notice the tooltips overlap.
Which versions of Angular, Material, OS, TypeScript, browsers are affected?
Angular 7.2.15
Material 7.3.7
Is there anything else we should know?
This may or may not be a bug.

While it would be nice for this not to happen, addressing it isn't straightforward; we'd either have to only show one tooltip at a time or make the tooltips avoid each other (super complex). Based on the cost-to-fix vs. how often it happens, I feel comfortable saying we won't be able to do anything about this one.

Thanks for taking a look.


TenCrop and FiveCrop refactored

Refactor #261 following #240.

Had to rename parameter vflip because it was conflicting with function.

Thanks!


Enable Http2?

This should make things faster.. Let's give it a try!

This is already a thing with traefik


Allow for badly behaved plugins

Hi there,
Thanks for all your hard work, keep it up!
I have been fiddling around with the tsify plugin and noticed whenever i messed up my syntax badly watchify would crash.
Turns out plugins pipelining multiple errors makes bad things happen.
What would the expected behaviour of the plugin be in this case? Should it not be emitting multiple errors in the first place? Should it be pumping these errors out elsewhere? If so I'll have a go at fixing this over in tsify.
My node is weak so I apologise if the following is a complete travesty. Either way it has been an interesting learning exercise.
thanks again.

Hmm. I've had this problem before with multiple error events being emitted, but you still need to end the stream. How about just writing the first error only? Like this:
if (!didError) {
        didError = true;
        writer.end('console.error(' + JSON.stringify(String(err)) + ');');
}

My bad for not understanding the semantics. With the stream ended I had to catch the associated error on the writer when it comes to packing time. My error messaging sucks a bit so please suggest alternative wording.

Ok, I'm following you I think. How about this then? Writer errors should still bubble. But we catch the case where we cause one ourselves by trying to write when we have already .ended?
thanks for your patience.

no problem. I don't know if I'm missing something but I just don't think the case in https://github.com/notmodified/watchify/commit/025e3fcd8dd4d5015995cd69e45dcd7eaf44d1f7 can ever happen. Or can it?
wb is the b.pipeline wrapped to be a read-only stream. Any error in pack would have surfaced through the pipeline if it happened before it became "readable" - and at that point it would never become "readable". If an error did occur after it became "readable", then https://github.com/notmodified/watchify/commit/025e3fcd8dd4d5015995cd69e45dcd7eaf44d1f7 still won't happen because that block is wrapped in a once (not an on).
Also, keep in mind that 99.99% of all errors will be emitted in the deps or syntax pipeline step ("deps" if it has a require, "syntax" if it doesn't). And no data will flow after sort unless all modules have been seen.

It's probably me that's missing something. The case I aim to solve with  https://github.com/notmodified/watchify/commit/025e3fcd8dd4d5015995cd69e45dcd7eaf44d1f7 is not to catch errors from wb but to catch the case where writer has already been ended by writer.end.
I think the reason I am seeing this behaviour might well the way tsify deals with its syntax errors. By doing b.pipeline.emit('error', '<something>'); and not using the pipeline as intended would that allow pack to become readable even though things had gone wrong further up?
I am happy to chop that bit out and see if I can fix the problem over on the tsify end if you think it's not something watchify should have to deal with.

Got it. Ok, then leaving the didError check in the pack's on "readable" makes sense. The error output from watchify is relied on by many tools, so just don't add the "Browserfication failed..." message. Don't forget to squash and don't remove space that's not in your changeset :smile:

Ok. Message removed. Commits squashed. Sneaky white space back where it belongs.
Thanks again for your patience, it's appreciated.

np! thanks for the PR! published as<EMAIL_ADDRESS>


Use condition argument of cv.wait instead of loop

See changes.

So they have same meaning, just a style enhancement?

It can avoid spurious awakenings.


Vector format export to Office 365

Is your feature request related to a problem? Please describe.

Diagrams embedded using the Office 365 App are rasterized when shared as .docx (not in vector format)
Imported drawing using Office 365 app are rasterized when exported as PDF and looks terrible
Imported SVG from draw.io using Office 365 are broken due to lack of full SVG 1.1 support in Office
Poor print quality from diagram embedded with Office 365 app

Describe the solution you'd like
True vector export to Office Suite, either by simplified "office compatible" SVG export with text-to-path and avoidance of CSS to style text + backgrounds (would require "polyfills" in many cases).
Alternatively WMF export.
Describe alternatives you've considered
Please do let me know if I'm missing something obvious here. I've been unable to find an alternative method that provides the professional print/PDF quality that is needed.
Additional context
I know this is much to ask, but with draw.io being absolutely fantastic at drawing elegant and professional looking drawings, not be able to import them correctly into Word (or even Inkscape) in vector format without breaking the layout is limiting draw.io's usefulness for me and perhaps others.
Demo diagram imported via Office 365 app and exported to PDF with minimum size publishing. Terribly pixelation and JPEG artifacts:

Same diagram to PDF in Word with "standard publishing". Better, but still pixelated.

SVG exported from draw.io inserted into Word is mangled:

Original in draw.io or even SVG in browser (even Edge) looks absolutely professional, but I can't find any way to make it look good embedded in Word/PDFs.
Print quality when printed from Word with Office 365 app has fuzzy edge dithering due being scaled raster:

Print quality when printed from draw.io is perfect:

Imported SVG from draw.io using Office 365 are broken due to lack of full SVG 1.1 support in Office
The add-ins shows the same behavior, also I noticed objects from official disapears once imported.
@larsch , my current workaround is to use advanced export, png and boost to 400 DPI
Oli

Yes, I would fully support an "office compatible" SVG export! Maybe even just exporting to SVG 1.0 as an option.
How it currently looks in the latest Word and PowerPoint (Version 2102, build 13801):

Please implement something like this. I need a vector format, that can be easily imported (drag and drop) in Microsoft Office. EMF would work great for MS Office.


The link to blog post has broken

I am experiencing an issue while attempting to access the content through the following link: blog. It appears that the resource is currently unavailable, resulting in a 404 error.
In addition, I would like to inquire about the status of the matter referenced as Issue #79. Is there any progress or update available on this particular concern?

I think we could probably link directly to https://github.com/src-d/hercules
but I'm not sure if this project is still alive – maybe not worth keeping the link

Thank you for your insightful response.
I agree with your assessment regarding the status of the hercules project. It's crucial to note that this project still possesses substantial resources that could be beneficial. Also, your provided link indeed offers valuable and detailed information.


Obfuscated js throws error: "Uncaught SyntaxError: Illegal return statement"

I am using javascript-obfuscator 0.10.0-beta.8, and I have been obfuscating my js from the command line, with good success, for a few months now. But as of the past week or so, many of my resultant obfuscations error out inside my Chrome extension. I haven't touched javascript-obfuscator, haven't updated it, nothing. Also I haven't changed the way I'm calling it from cmd-line, nor changed any params.
Here is the error I have been getting: Uncaught SyntaxError: Illegal return statement. And here is the obfuscated block of code that triggers it:
if (_0x4423('0x7') === 'mJc') {
     zingLog(_0x4423('0x8'));
} else {
      return !![];
}

The return !![]; seems to be throwing the error. Both Chrome and JSNice point at the same line. When I encountered this 3-4 days ago, I had no idea what to do about it. I ended up re-obfuscating my files 2 or 3 more times, and after that -- PRESTO -- the error was gone. I wish I had taken a diff so I could know what changed. But that was another puzzling thing: how can I get different obfuscated results after changing nothing, no cmd-line params, no source code, etc? It seems as though the obfuscator generates different results each time it runs??
Here is how I'm calling the obfuscator:
node "C:\javascript-obfuscator.js" "myfile.js" --output "C:\obfuscated.js" --compact true --controlFlowFlattening true --controlFlowFlatteningThreshold 0.75 --deadCodeInjection true --deadCodeInjectionThreshold 0.4 --mangle false --rotateStringArray true --selfDefending true --stringArray true --stringArrayThreshold 0.75 --stringArrayEncoding false --unicodeEscapeSequence false
That always executes with no errors, and as mentioned, in the past the resulting obfuscations work great, and ran fine in Chrome. I am currently using Chrome Version 61.0.3163.79 (Official Build) (64-bit). That's the latest update as of 9/12/2017, but the previous version I was on (60.xxx - before today) threw the same error.
Is this a bug in the obfuscator? A bug in the browser? Can changing a cmd-line option solve this? My entire obfuscated file is over 1500 lines so it seems ill-advised to post all of it here, but I can if necessary. Please help.
Thanks in advance.

At first - can you update to 0.11.0 version?

I apologize, but how do I update from cmd-line? It took me awhile to figure out how to just get it installed and working, but I never updated before.

Did you install obfuscator globally? If so - npm i -g javascript-obfuscator

I ran that, and it seemed to work. But I can't tell if it really upgraded because:
(1) After re-obfuscating and running my extension in Chrome, I still get the same error.
(2) The version inside .\javascript-obfuscator\package.json is still 0.10.0-beta.8, as before. (How can I determine the installed version?)

.\javascript-obfuscator\package.json
show me full path

Anyway i reproduced error.

Fixed in https://github.com/javascript-obfuscator/javascript-obfuscator/commit/d9612d1bf4032bcb868a2a3771aea5de03dbdcb3
Will be out with first 0.12.0 beta release in next few weeks.

I'm not sure I understand. Am I able to get the fixed files somehow? I clicked the link, but don't know how to get that fix. Thanks! (I can't wait a few weeks because it's fairly urgent.)

I'll release first beta version of 0.12.0 in next few weeks.

And i should release 0.11.1 bugfix version on thos week.

Hi again. Check new<EMAIL_ADDRESS>version with this fix.

That works great now. Thanks so much!


Google Auth

When using google auth what do we use as the redirect url?

Sorry I got it
http://localhost:3000/api/auth/callback/google


Publish SixLabors.Memory

Reverenced System.Memory 4.5.1 and System.Buffers
Copied MemoryAllocator, ArrayPoolMemoryAllocator, SimpleGcMemoryAllocator and related types from the ImageSharp repository
Image processing-specific stuff like Buffer2D<T> and static extension classes have not been moved
Moved unit tests
Added docs
Changed Guard.NotNull() and DebugGuard.NotNull() to be generic

Let's take the time to properly review the API shape and namings!
After merging this PR I plan to PR the adaptation of the new core types in ImageSharp

Also looks good to me think we can merge this.

Damn, your review got lost because I touched the code. I didn't know it works like this.

Imma gonna merge this. Exciting!


Please update the require dependencies

Hi,
I was trying the example : https://github.com/reactphp/whois/blob/master/examples/query.php after installing react/whois
php example.php
PHP Fatal error:  Uncaught Error: Class 'React\EventLoop\Factory' not found in /home/hari/experiments/php/whois/example.php:5
Stack trace:
#0 {main}
  thrown in /home/hari/experiments/php/whois/example.php on line 5

Fatal error: Uncaught Error: Class 'React\EventLoop\Factory' not found in /home/hari/experiments/php/whois/example.php:5
Stack trace:
#0 {main}
  thrown in /home/hari/experiments/php/whois/example.php on line 5

After trying to install react/event-loop it looks dependencies cannot be resolved.
 ✘  ~/experiments/php/whois  composer require react/event-loop
Using version ^0.4.2 for react/event-loop
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - react/socket v0.3.4 requires react/event-loop 0.3.* -> satisfiable by react/event-loop[0.3.x-dev, v0.3.0, v0.3.3, v0.3.4, v0.3.5] but these conflict with your requirements or minimum-stability.
    - react/socket v0.3.4 requires react/event-loop 0.3.* -> satisfiable by react/event-loop[0.3.x-dev, v0.3.0, v0.3.3, v0.3.4, v0.3.5] but these conflict with your requirements or minimum-stability.
    - react/socket v0.3.4 requires react/event-loop 0.3.* -> satisfiable by react/event-loop[0.3.x-dev, v0.3.0, v0.3.3, v0.3.4, v0.3.5] but these conflict with your requirements or minimum-stability.
    - Installation request for react/socket (locked at v0.3.4) -> satisfiable by react/socket[v0.3.4].

Installation failed, reverting ./composer.json to its original content.

Have you tried a composer install before running the examples? This should install all dependencies including react/event-loop.

@jsor Hm... Interesting. That was actually an error in my part. The query.php was using '../' , but I was trying something like below.
mkdir dns
cd dns
composer require react/whois
wget https://raw.githubusercontent.com/reactphp/whois/master/examples/query.php
php query.php

Closing for this is resolved. Sorry for the hassle.

Interestingly there is a loop going on
Warning: stream_select(): No stream arrays were passed in /home/hari/experiments/php/dns/vendor/react/event-loop/React/EventLoop/StreamSelectLoop.php on line 141
PHP Warning:  stream_select(): supplied argument is not a valid stream resource in /home/hari/experiments/php/dns/vendor/react/event-loop/React/EventLoop/StreamSelectLoop.php on line 141

Warning: stream_select(): supplied argument is not a valid stream resource in /home/hari/experiments/php/dns/vendor/react/event-loop/React/EventLoop/StreamSelectLoop.php on line 141
PHP Warning:  stream_select(): supplied argument is not a valid stream resource in /home/hari/experiments/php/dns/vendor/react/event-loop/React/EventLoop/StreamSelectLoop.php on line 141

Any idea about it ?

This is probably better suited for an issue over at reactphp/event-loop. Be sure to include more context, ideally a reproducible test case.

The whois component hasn't been updated in a while. It actually uses the React\Socket\Connection class (https://github.com/reactphp/whois/blob/master/src/React/Whois/ConnectionFactory.php#L20) which is actually not supported (https://github.com/reactphp/socket/pull/13) and may result in the above error.
I guess this component could use an update :+1:


Release v3.0.0

This PR contains changes for the next major release: v3.0.0.
Being a major release, there were some hiccups with using an exact generator/API definitions hash. Primarily, I had to make some changes in the Apex generator that, at the time of this writing, are not yet merged. The closest commits at this time though are:

Codegen: 8dc2602de0f9555623a4721e51c48622a14cfdf5
API definition: 5d64e0df520d5620ab7bfd6cae9da917734918d7

This version comes with a lot of changes in the form of renamed getters/setters, due to naming convention changes in how we're generating the SDK. Things that were previously marked as deprecated were also removed and both Conversation and Language Translator v2 are no longer present.
Full migration details can be found in the migration guide.

Can you add the migration guide as part of this PR?

Yep, I'm writing it now and will add it once it's done.

@germanattanasio The migration guide can be found here and will be linked in the release.

Please make sure you do a quick "salesforce" search in our backlog and validate there are no more issues we can fix with this major release.
Good job here 👍

@mediumTaj everything should be good to review now. The last changes I added were the fixes for the Discovery parameter issue and the generated createClassifier()/updateClassifier() code. The migration guide has been updated accordingly.


[5.3] Consistent mail sending return values

As pointed out by @Mevrael, and many others, the current situation is pretty crap.
We are literally not allowed to return anything but an integer here if we want to correctly work with swiftmailer. All but our smtp driver send mail asynchronously, so I'm just going to plug the number of recipients sent to, in order to correctly implement the interface.

Why return anything?

Tweaked this.


Add translation files for Portuguese

sphinx language identifiers for Portuguese:
https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-language

pt – Portuguese
pt_BR – Brazilian Portuguese
pt_PT – European Portuguese

sphinx-book-theme supports only pt as language identifier:
https://github.com/executablebooks/sphinx-book-theme/tree/master/src/sphinx_book_theme/assets/translations

It's fine, but I can only help with the pt_BR, so the official translation to Portuguese will be Brazilian flavored ~the right and widely spoken anyway xD~.

I raised an issue at https://github.com/executablebooks/sphinx-book-theme/issues/600 about missing support for language variants.

My suggestion is to start with pt as language identifier for now, when we arrive at the point at which we want both an European and Brazilian flavored Portuguese translation, we can still move towards pt_PT and pt_BR as language identifiers.

Now available at https://hosted.weblate.org/projects/fortran-lang/fpm/pt/.


multiplication and division

Current problem

2 + 4 / 2 = 4 (expected)
2 + 4 / 2 = 3 (unfortunately it's happening)


⚠ Bofrost.es has degraded performance

In d9e0733, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1552 ms

Resolved: Bofrost.es performance has improved in be8eb31 after 11 minutes.


Doc and cleanup

Updated the docs to represent the current state of the code.

remove unused file This file has be rendered useless after the build upgraded to
cmake.

@trbehera addressed the review comments. Please re-review the PR.


adds availability zone aware node-pools

This PR adds functionality to explicitly specify an availability zone for node pools.
It includes:

Migration of existing node pools. Defaults are not queryable via API. Therefore this migration needs to be hardcoded. It is unfortunately CCloudEE specific. ☹ We can turn this migration into a NOOP once it is rolled out.
Makes availabilityZone a required field on node pools
Disallows changes to the availabilityZone via API
Creates additional StorageClasses cinder-zone-a, cinder-zone-b, ... Keeps cinder-default. This will choose an AVZ randomly of all existing nodes.

Needs to be released in lock-step with Elektra due to the now required availabilityZone field.
Fixes #258

Awesome work! Merge at will


About the result of test set

Nice Work! But I have some question about the result of test set.
I try to create a new test_evaluator instead of SegMetric,And I upload the test set result with tta on the codalab,it get a score about 66.7.
I wonder if the pretrained model provided by the repository is trained on the train-valid set.
Or maybe the method I'm using when reproducing is not accurate enough,I am a beginner in the mm3d framework.
Here is my  test_evaluator :
# Copyright (c) OpenMMLab. All rights reserved.
import os.path as osp
import tempfile
from typing import Dict, Optional, Sequence
import yaml
import mmcv
import numpy as np
from mmengine.evaluator import BaseMetric
from mmengine.logging import MMLogger
import mmengine
from mmdet3d.evaluation import seg_eval
from mmdet3d.registry import METRICS
import pdb
@METRICS.register_module()
class SemantickInferMertric(BaseMetric):

    def __init__(self,
                 collect_device: str = 'cpu',
                 prefix: Optional[str] = None,
                 pklfile_prefix: str = None,
                 submission_prefix: str = None,
                 result_path:str = None,
                 result_start_index:int = 0,
                 conf:str = None,
                 **kwargs):
        self.pklfile_prefix = pklfile_prefix
        self.submission_prefix = submission_prefix
        self.result_path = result_path
        self.result_start_index = result_start_index
        self.current_start_index = self.result_start_index
        self.limit=[921,1061,3281,631,1901,1731,491,1801,4981,831,2721]#the length of every test set seq
        self.limit_id = 0 #count
        self.scene_id = 0 #seq id
        super(SemantickInferMertric, self).__init__(
            prefix=prefix, collect_device=collect_device)
        self.conf = conf
        if self.conf :
            with open(self.conf) as f:
                self.conf = yaml.safe_load(f)
    def process(self, data_batch: dict, data_samples: Sequence[dict]) -> None:
        """Process one batch of data samples and predictions.

        The processed results should be stored in ``self.results``,
        which will be used to compute the metrics when all batches
        have been processed.

        Args:
            data_batch (dict): A batch of data from the dataloader.
            data_samples (Sequence[dict]): A batch of outputs from
                the model.
        """
        self.results.append((0, 0))

        # label inv
        # pdb.set_trace()
        # pdb.set_trace()
        pred = data_samples[0]['pred_pts_seg']['pts_semantic_mask'] #labels
        map_inv = self.dataset_meta['learning_map_inv'] #inv mapping
        pred[pred == 19] += 99 #unlabeled or ignore
        pred += 1 #[0,18] -> [1,19]
        for i in map_inv:
            pred[pred==i] = map_inv[i]+1000  #avoid the pred label in [1-19] be mapped twice
        pred[pred!=119]-=1000
        pred[pred==119] = 0 #unlabel
        pred.cpu().numpy().astype(np.int32).tofile(f'/mnt/storage/dataset/semanticKitti/dataset/FRNet/sequences/{self.scene_id+11}/predictions/{self.limit_id:06}.label')
        print(f'finsh the {self.scene_id+11}/predictions/{self.limit_id:06}.label')
        self.limit_id+=1
        if self.limit_id == self.limit[self.scene_id]: #next sequence
            self.scene_id += 1
            self.limit_id = 0
        # pdb.set_trace()

        # output to label file
    def format_results(self, results):
        r"""Format the results to txt file. Refer to `ScanNet documentation
        <http://kaldir.vc.in.tum.de/scannet_benchmark/documentation>`_.

        Args:
            outputs (list[dict]): Testing results of the dataset.

        Returns:
            tuple: (outputs, tmp_dir), outputs is the detection results,
                tmp_dir is the temporal directory created for saving submission
                files when ``submission_prefix`` is not specified.
        """

    def compute_metrics(self, results: list) -> Dict[str, float]:
        """Compute the metrics from processed results.

        Args:
            results (list): The processed results of each batch.

        Returns:
            Dict[str, float]: The computed metrics. The keys are the names of
            the metrics, and the values are corresponding results.
        """
        ret_dict = dict()

        return ret_dict

and the result of test set :

Hi @Xavier-wa, thanks for your interest in our work. The checkpoint provided in the repo is trained only on the train set. For the test set, you may try to train on the train-val set, model ensembling, TTA, etc. to reproduce our results.

Hi @Xavier-wa, thanks for your interest in our work. The checkpoint provided in the repo is trained only on the train set. For the test set, you may try to train on the train-val set, model ensembling, TTA, etc. to reproduce our results.
Thanks for your reply,I will try it.


Fix the frontend build errors (see the frontend build logs)

We get errors like these in the frontend build logs:
#24 132.6 n [Error]: Dynamic server usage: Route /help/scores-faq/ couldn't be rendered statically because it used `cookies`. See more info here: https://nextjs.org/docs/messages/dynamic-server-error
#24 132.6     at l (/app/front_end/.next/server/chunks/9085.js:1:58682)
#24 132.6     at d (/app/front_end/.next/server/chunks/6005.js:24:49081)
#24 132.6     at a (/app/front_end/.next/server/app/(main)/accounts/change-email/route.js:1:5638)
#24 132.6     at d (/app/front_end/.next/server/app/(main)/accounts/change-email/route.js:1:6641)
#24 132.6     at p (/app/front_end/.next/server/app/(main)/accounts/change-email/route.js:1:7115)
#24 132.6     at f.getBulletins (/app/front_end/.next/server/chunks/409.js:1:1417)
#24 132.6     at g (/app/front_end/.next/server/chunks/409.js:1:1671)
#24 132.6     at /app/front_end/.next/server/chunks/409.js:1:24705
#24 132.6     at eh (/app/front_end/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:134657)
#24 132.6     at e (/app/front_end/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:12:137542) {
#24 132.6   description: "Route /help/scores-faq/ couldn't be rendered statically because it used `cookies`. See more info here: https://nextjs.org/docs/messages/dynamic-server-error",
#24 132.6   digest: 'DYNAMIC_SERVER_USAGE'
#24 132.6 }

Should be fixed.

The issue was mostly caused by error boundary which didn't ignore DYNAMIC_SERVER_USAGE error (when it should). Should be fine now.


fix ibm.biz url references

Fixes https://github.com/IBM-Bluemix/docs/issues/1153
References to 'ibm.biz' URLs should be prefaced with "http://" to ensure that they are not interpreted as links relative to the current doc page.

I see this fixed out in production now. Closing PR.


Allow larger PGP key ids

Right now, the website only accepts short key ids. Basically that regexp needs to be tweaked to allow keys of the size being entered now, or twice that size :)

This is a duplicate of #24.


Upgrade test.check to 0.9.x

This is needed for #72

@nathanmarz Hi, I was just going through issues opened on specter and I found this ticket. I forked and tried to bump test.check to 0.9.0 and ran the tests locally. It ran fine. Is there something else you want to do for upgrading test.check version?

Tested on commit
commit acf2801cbcf74904706b7e531aab0b61dda8d9e2
Author: nathanmarz<EMAIL_ADDRESS>Date:   Wed Mar 1 16:07:21 2017 -0500

    update readme

If you want I can share a patch with the latest version. Thanks.

Hmm, I was under the impression there were more breaking changes from 0.7 -> 0.9. Took another look and the only updates needed on Specter were the namespace changes for test.check's cljs implementation. This is now complete on master.


 The point is we'd get caught if we ever did this!!)
Contributing Agreement

[x] I have read and understood the Optimism Contributing Guide and Code of Conduct and am following those guidelines in this pull request.

This is failing in the CI because of  Fatal: Error starting protocol stack: unable to start syncservice: Error resolving addresses: Cannot resolve canonical transaction chain: no contract code at given address
This error happens when the CTC deploy height configuration option for Geth is too early. By any chance was a change made that changed the number of contracts being deployed? It could also happen if the contract deployment failed, but we do not wait for receipts so it is impossible to know for sure
There are two open PRs that address this:

https://github.com/ethereum-optimism/contracts-v2/pull/99
https://github.com/ethereum-optimism/contracts-v2/pull/51

#51 is no longer viable based on Karl's comment re: not waiting for each receipt in order failing deployment on Goerli. I usually deploy using a Ledger, so I don't observe this because the signing time is long enough between txs to not notice the problem. Also we need to be mindful of not changing the output of the deployer such that it breaks downstream services that depend on addresses.json

Right now the CTC deploy height is hard coded as 8. It would be better if we could access a receipt from addresses.json and pass along the deploy height to remove the need to hard code it.
https://github.com/ethereum-optimism/optimism-integration/blob/4f51df04866b805d6a74b5f2335e80bc59018b10/docker-compose.env#L29

Problems mentioned above solved by https://github.com/ethereum-optimism/contracts-v2/pull/132
See https://github.com/ethereum-optimism/contracts-v2/issues/133


Threads blocked with IN_NATIVE state inside Netty

Expected behavior
All the threads managed by Netty need to co-run to complete all Network I/O initiated from multiple application threads that use Netty.
Actual behavior
We noticed that the system is unable to move further (i.e, JVM Hang) to process new resource creation requests via HTTP RESTful API, as all such calls ended up not having threads to be used to complete the request.
Across 3 thread-dumps spaced between 10-15 seconds , Out of 750 threads available on the system during a Jstack dump, atleast 1/3rd of the JVM threads have blocked on Netty (207 threads).
Thread 25375 - threadId:Thread 25375 - state:IN_NATIVE
stackTrace:

io.netty.channel.epoll.Native.epollWait0(int, long, int, int) @bci=0 (Compiled frame; information may be imprecise)
io.netty.channel.epoll.Native.epollWait(int, io.netty.channel.epoll.EpollEventArray, int) @bci=10, line=109 (Compiled frame)
io.netty.channel.epoll.EpollEventLoop.epollWait(boolean) @bci=117, line=232 (Compiled frame)
io.netty.channel.epoll.EpollEventLoop.run() @bci=65, line=256 (Compiled frame)
io.netty.util.concurrent.SingleThreadEventExecutor$2.run() @bci=13, line=112 (Interpreted frame)
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run() @bci=4, line=145 (Interpreted frame)
java.lang.Thread.run() @bci=11, line=748 (Compiled frame)

Steps to reproduce
We have not come up yet with the shortest way to reproduce the problem.
Minimal yet complete reproducer code (or URL to code)
Netty version
3.10.6.Final
JVM version (e.g. java -version)
java version "1.8.0_141"
Java(TM) SE Runtime Environment (build 1.8.0_141-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.141-b15, mixed mode)
OS version (e.g. uname -a)
Linux cic-2.domain.tld 4.4.0-78-generic #99~14.04.2-Ubuntu SMP Thu Apr 27 18:49:46 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Would you check your Netty version again, Netty 3.10.6 would use the org.jboss.netty package, not io.netty.

@vivek-konnect like @johnou already said the thread-dump shows Netty 4.x and not 3.x (also note that 3.x is EOL for a long time). That said have threads stuck in  io.netty.channel.epoll.Native.epollWait0  is normal as it just tells you that there is no IO to handle at the moment and we wait for the Kernel / OS to wakeup and tell us we need to process some IO events. Why you think there is an issue ?

We will reconfirm the Netty version.
We think there is definitely issue because we have more than 1/3rd of the threads in the production system are locked up in IN_NATIVE state (precisely 207 threads out of 750 threads)  and the production system freezes.  Two incidents of such a freeze both point to this same Jstrack Thread Trace.
Are there some other things we can continously monitor in production system so that we can attach more information here for further support ?

From within Karaf of OpenDaylight Controller, the version is showing up for us as 3.10.6.Final.
opendaylight-user@root>bundle:list | grep Netty
214 | Active   |  80 | 3.10.6.Final                       | Netty
opendaylight-user@root>
Is there a better way to confirm the version of the Netty.jar?
Shall we do:
unzip -p netty.jar META-INF/MANIFEST.MF

We got the required information..
Two versions of Netty are being run under our Karaf container but the one our software uses is 4.0.37.Final
Can you please advise further?
opendaylight-user@root>bundle:list | grep -i netty
165 | Active   |  80 | 4.0.37.Final                       | Netty/Buffer
166 | Active   |  80 | 4.0.37.Final                       | Netty/Common
167 | Active   |  80 | 4.0.37.Final                       | Netty/Transport
168 | Active   |  80 | 4.0.37.Final                       | Netty/Codec
169 | Active   |  80 | 4.0.37.Final                       | Netty/Codec/HTTP
170 | Active   |  80 | 4.0.37.Final                       | Netty/Handler
171 | Active   |  80 | 4.0.37.Final                       | Netty/Transport/Native/Epoll
172 | Active   |  80 | 0.5.4.SNAPSHOT                     | org.opendaylight.controller.netty-config-api
173 | Active   |  80 | 0.5.4.SNAPSHOT                     | org.opendaylight.controller.netty-event-executor-config
174 | Active   |  80 | 0.5.4.SNAPSHOT                     | org.opendaylight.controller.netty-threadgroup-config
175 | Active   |  80 | 0.5.4.SNAPSHOT                     | org.opendaylight.controller.netty-timer-config
214 | Active   |  80 | 3.10.6.Final                       | Netty
opendaylight-user@root>

@vivek-konnect again I think there is nothing wrong with have threads sitting in this state... Its expected if there is no IO ready to be handled atm. The number of threads really depends on how many *EventLoopGroups you have but that's up to you.

Let me close this as I think its not an actual issue (if its an issue then its one because you or the library you are using is creating to many EventLoops). Please re-open if you still think there is an issue and provide more info why you think so.


fixed broken link

Hi @mairaw, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

It looks like you're a Microsoft contributor (Maira Wenzel). If you're full-time, we DON'T require a Contribution License Agreement. If you are a vendor, please DO sign the electronic Contribution License Agreement. It will take 2 minutes and there's no faxing! https://cla.microsoft.com.

TTYL, MSBOT;

Closing since this issue was fixed in the private repo later.


Option to specify an editor for svelte-inspector

Describe the problem
There's a lot of text editors out there. I switch editors for different tasks using cmd + tab on macOS or alt + tab on Windows.
Describe the proposed solution
Is there a way to help people configure default file types?
This Visual Studio Code system default was particularly useful:
defaults write com.apple.LaunchServices/com.apple.launchservices.secure LSHandlers -array-add '{LSHandlerContentType=public.plain-text;LSHandlerRoleAll=com.microsoft.VSCode;}'

from Chris Ferdinandi's blog: https://gomakethings.com/changing-the-default-text-editor-on-macos/

Alternatives considered
Configure/advise which editor the inspector opens.
Importance
would make my life easier

We use vite to open files which in turn uses launch-editor. You should be able to specify your editor with EDITOR or VISUAL env.
This https://github.com/yyx990803/launch-editor/pull/39 is going to help in cases where multiple editors are open and it picks the wrong one

This needs to be handled upstream in launch-editor


Add bolt's journey with pymc-marketing

Add Bolt's journey with PyMC-marketing on pymc-marketing.io

:books: Documentation preview :books:: https://pymc-marketing--359.org.readthedocs.build/en/359/

I think we should put this in mmm #resources like in this PR? https://github.com/pymc-labs/pymc-marketing/pull/249

I think we should do both.

If we want to add it on the index page, we should add a new section? Maybe something along the lines of Who is using PyMC-Marketing?

@SangamSwadiK Can you make those changes?

thanks!


[frameit] Change framefile.json load path to support Screengrab and Supply

Issue https://github.com/fastlane/fastlane/issues/16289
When using screengrab and frameit, frameit doesnt load properly
the config file because it matches the current iOS setup,
to be able to load the android properly we need to fully extend the
path.

Checklist

[x] I've run bundle exec rspec from the root directory to see all new and existing tests pass
[x] I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
[x] I've read the Contribution Guidelines
[x] I've updated the documentation if necessary.

Motivation and Context
This PR fixes this issue: https://github.com/fastlane/fastlane/issues/16289
I noticed that frameit expects the Framefile.json to be in a specific folder location, but when using it with screengrab and supply it doesn't support the folder structure, I saw the in the PR ea4d8a2
the method create_config was created but isn't considering the actual support to supply/screengrab, the screenshots have added the frame but it doesn't add the background.
Description
I've just extended the path to be 4 levels deep.
Since I couldn't see much value in add a test for this specific method I tested loading from different paths, but if you guys think that might be cool I can add it.
Testing Steps
Just created a new test in the config_parser_spec to simulate it to be and tested in a local project.
I can add the test if it's ok.

@googlebot I signed it!

hey @joshdholtz can you have a look when you have some time?
Cheers

@joshdholtz no worries, looking forward to open my second PR screengrab related


Push PySoundFile into Debian

As soon as 0.6.0 is released, I'll ask the Debian people to add it to their repository.
I guess this will eventually also become available for Ubuntu, but I don't know if/how we can speed this up.

There is some activity: http://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-April/043822.html

http://anonscm.debian.org/cgit/pkg-multimedia/pysoundfile.git/

The package is now in the "NEW" queue ...

http://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-May/043986.html
https://ftp-master.debian.org/new/pysoundfile_0.7.0-1.html

Is there going to be a python2-soundfile as well as python3-soundfile ?

currenlty there are no plans for a python2 Debian package.
do you need it? is upgrading your project to python3 an option?

It's not an option since Shoebot implements Nodebox1, which is python2 ... if I make it python3, it won't be implementing Nodebox1 any more.
I might make a Shoebot3 in the future, but quite possibly not as it won't be able to run many existing bots.

tell the Nodebox1 implementors, that python2 EOL has been announced, so they should start migrating to python3.
in the meantime: i'll probably add python2 support to the Debian package once PySoundFile has been accepted in the archives (which might take some time)

Taking some time is fine. .. I'm hoping to be able to do a release in the next year or so.
I think all the Nodebox1 implementors are working on newer versions now that are quite different - NBX1 == graphics programming, NBX 3 is a graphical-programming-environment that is incompatible.    Shoebot is to run those NBX1 programmes ... if I port it there won't be much to run on it at all.

pysoundfile is now in Debian. this issue can be closed.

Thanks a lot @umlaeute for your work!
For reference: https://packages.debian.org/python3-soundfile

and for those depending on python2: python-soundfile is currently awaiting approval into the Debian archives.

and since today Debian has both python-soundfile (for python2.x) and python3-soundfile (for python3.x)

Awesome, great work :)


Bundle size?

Hey this is just a quick question for somebody knowledgeable about the ecosystem... when I put graphiql in my app it balloons bundle size by about 2mb. Culprits are: codemirror + graphql-language-service-interface+graphql-language-service-utils + graphiql +graphql and I'm not sure I need -ALL- that in my frontend? Just looking for tips on how to cut the bundle. thank you!

@sw-yx Hi there!
A short answer is yup, you'd need all those things. Just to give a short rundown:

graphql-js: The ref implementation of GraphQL - this powers every GraphQL-related needs.
graphql-language-service-*: The core of the GraphQL online syntax parser/validator
codemirror and codemirror-graphql: A series of wrappers around graphql-language-service in CodeMirror editor. This provides the CodeMirror mode/hint/lint functions for query/variables editor, and other cool stuffs such as onHover tooltip implementations

I did a simple du -hs for the libraries listed above, and codemirror wins with being 3.2mb of the size. Now I understand this may not be the exact size of how much CodeMirror's size is as graphiql.min.js may have a different proportions between libraries bundled in - my point is I think GraphiQL is as compact as it could be for now.

@sw-yx: It's definitely big, and something that you probably want to use webpack bundle splitting to ensure that it doesn't get baked into the bundle for the rest of your app.

very cool guys, thank you so much! loving it (of course) and will look to see where I can contribute.

bundle size has dropped precipitously, with plans afoot to optimize even further, hold my kombucha:
https://github.com/graphql/graphiql/pull/1001


Delete symbols that don't exist

I think this fixes the issue with #95.
I found the script that you mentioned in #95 (in #57) and it confirmed my guess that there was "custom" symbols from the script author there that don't seem to exist in standard Typst, which was causing the issue. Thankfully, the script puts them all in the same place, so it's fairly easy to fix. (Though partial seems to be supported in Typst now.)

Thanks for the update!


Question about the space.

Hi authors,
Congratulations on a great work!
I did not go over every line of code but could you please help to point out how you implement the source or target space in Fig.2 of your paper?
Thanks.

Download
https://www.mediafire.com/file/o50xaz6wgtazqnx/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Thanks for your kind words.
You can check the implementation code in src/utils/llm_rep_utils.py and src/utils/vm_rep_utils.py.


Add fenglixa from IBM

Provide PRs:
kubeflow/kfserving#566
kubeflow/fairing#427
kubeflow/kfserving#582
kubeflow/fairing#431
kubeflow/fairing#433
kubeflow/pipelines#2752
Logged issues:
kubeflow/kfserving#564
kubeflow/kfserving#558
kubeflow/kfserving#565
kubeflow/fairing#430
Discuss/Fixed issues:
kubeflow/examples#681
kubeflow/kfserving#563
kubeflow/fairing#335
kubeflow/fairing#369
kubeflow/pipelines#2713

This change is

/cc @jinchihe

wow! Thanks for the efforts and great contributions! @fenglixa
/ok-to-test

/lgtm
/cc @jlewi @abhi-g
Could you please approve? Thanks!

/approve

Thanks for your contributions @fenglixa


Use native typeof instead

Closes #55
This is going away in 3.0.0 but I think it's worth releasing in the current major

:+1: Would love to see this released in the 2.x series, since Ember CLI 2.13 and Babel 6 make it so easy to compile to native async/await with targets.


Rate limit options for HTTPS behind network load-balancers

Do you want to request a feature or report a bug?
Feature
What did you expect to see?
In addition to the IP and header options for the RateLimit middleware, I would like a way rate limit connecting clients while behind a load-balancer or proxy, where it's not possible to see the clients IP.
I'm thinking would there be a way of using the connecting client's public key or signature to identify them as unique?  I don't know enough about the workings of TLS to know if this is doable, or if anything unique lives on beyond each HTTP request.
Our situation is that the HTTPS connection is terminated by Traefik, and all it sees is the IP of the load-balancer.

@Sarke most modern load balancers should support Proxy Protocol, which can be enabled when configuring your entrypoint in Traefik. When you configure the rate limiter for services assigned to that entrypoint it will, by default, use the actual client IP for stickiness criteria as long as the Proxy Protocol headers were sent by your upstream proxy.
https://docs.traefik.io/routing/entrypoints/#proxyprotocol
If you have any additional questions or trouble configuring this, please create a thread on our community forums and feel free to ping me there. Thank you!

Thank you for your suggestion, but the Rackspace load balancer doesn't support Proxy Protocol.
However, as this problem appears to be solved by using newer tech I doubt it will be implemented, so closing.


Using getlist for files is never empty?

I want to get file list from form by using:
photos = request.files.getlist("photo")
However if i haven't select anything, the photos value is never be [] like other fields.
So the way to check for empty field is failed:
if not photos:
   pass

The result for empty form is:
 [<FileStorage: '' ('application/octet-stream')>]
How do i get rid of it?

Duplicate of #3130

request.files always has whatever is submitted. You can do request.files["secondfile"].filename == ""`, or look at its size. But it was still submitted, just with an empty filename and no data. Again, that's just what HTML forms do.

https://github.com/pallets/flask/issues/3130#issuecomment-476765447


load sklearn pre-processing model

I'm trying to implement Transformer for a custom xgboost model. While training I'm using sklearn StandardScaler to scale the inputs, which I save to a pickle file to be used to scale test inputs.
I want to use this pickle object when overloading the preprocess method and then pass the scaled inputs to overloaded predict method.
How do I load this pickle model? I'm guessing there would be some setting in my InferenceService yaml file, like using storageUri for loading the xgboost model

@swapkh91 On transformer you can use the storageUri env to achieve the same thing, check this example https://github.com/kubeflow/kfserving/tree/master/docs/samples/v1beta1/transformer/torchserve_image_transformer#create-the-inferenceservice

@yuzisun  I was just trying without external model at the moment and running into the below problem
The transformer deployment is failing. Below are the logs
For xgb-transformer-predictor-default-00001-deployment, this is the last log so I'm assuming predictior is working
Uvicorn running on http://<IP_ADDRESS>:8080 (Press CTRL+C to quit)
For xgb-transformer-transformer-default-00001-deployment, I get these repeating errors
aggressive probe error (failed 202 times): dial tcp <IP_ADDRESS>:8080: connect: connection refused
timed out waiting for the condition
This is my yaml file:
kind: InferenceService
metadata:
    name: xgb-transformer
spec:
  transformer:
    containers:
    - image: swapkh91/xgb-transformer:latest
      name: kfserving-container
      env:
        - name: STORAGE_URI
          value: gs://kfserving-custom-models/xgb-iris
  predictor:
    xgboost:
      protocolVersion: "v2"
      storageUri: gs://kfserving-custom-models/xgb-iris

__main__.py
import json
import kfserving
from .transformers_test import KFServing_Custom_XGBoost_Model

DEFAULT_MODEL_NAME = "xgb-transformer"

parser = argparse.ArgumentParser(parents=[kfserving.kfserver.parser])
parser.add_argument(
    "--predictor_host", help="The URL for the model predict function", required=True
)

args, _ = parser.parse_known_args()

if __name__ == "__main__":
    model_names = [DEFAULT_MODEL_NAME]
    models = []
    for model_name in model_names:
        transformer = KFServing_Custom_XGBoost_Model(model_name, predictor_host=args.predictor_host)
        models.append(transformer)
    kfserving.KFServer(
        workers=1
    ).start(models=models)

transformers_test.py
from typing import List, Dict
import kfserving
import logging

logging.basicConfig(level=kfserving.constants.KFSERVING_LOGLEVEL)

class KFServing_Custom_XGBoost_Model(kfserving.KFModel):
    def __init__(self, name: str, predictor_host: str):
        super().__init__(name)
        self.name = name
        self.predictor_host = predictor_host
        logging.info("MODEL NAME %s", name)
        logging.info("PREDICTOR URL %s", self.predictor_host)
        self.timeout = 100

    def preprocess(self, inputs: Dict) -> Dict:
        """Pre-process activity of the Image Input data.
        Args:
            inputs (Dict): KFServing http request
        Returns:
            Dict: Returns the request input after converting it into a tensor
        """
        return inputs

    def postprocess(self, inputs: List) -> List:
        """Post process function of Torchserve on the KFServing side is
        written here.
        Args:
            inputs (List): The list of the inputs
        Returns:
            List: If a post process functionality is specified, it converts that into
            a list.
        """
        return inputs```

Well, the same problem occur in my tf codes, have you resolved this problem?

@yuzisun I got the transformer issue working and bot deployments are now running
These are the files, for anyone else's reference:
__init__.py
from .transformers_test import XGBoostModel
from .transformer_model_repository import XGBoostModelRepository

__main__.py
import argparse
import sys
import logging
import kfserving
from xgbserver import XGBoostModel, XGBoostModelRepository

DEFAULT_MODEL_NAME = "custom-xgb"
DEFAULT_NTHREAD = 1
MODEL_DIR = '/mnt/models'

parser = argparse.ArgumentParser(parents=[kfserving.kfserver.parser])  # pylint:disable=c-extension-no-member
parser.add_argument('--model_dir', default=MODEL_DIR,
                    help='A URI pointer to the model directory')
parser.add_argument('--model_name', default=DEFAULT_MODEL_NAME,
                    help='The name that the model is served under.')
parser.add_argument('--nthread', default=DEFAULT_NTHREAD,
                    help='Number of threads to use by XGBoost.')
args, _ = parser.parse_known_args()

logging.basicConfig(level=kfserving.constants.KFSERVING_LOGLEVEL)

if __name__ == "__main__":

    try:
        model = XGBoostModel(args.model_name, args.model_dir, args.nthread)
        kfserving.KFServer(registered_models=XGBoostModelRepository(args.model_dir, args.nthread))\
        .start([model] if model.ready else [])
    except Exception:
        ex_type, ex_value, _ = sys.exc_info()
        logging.error(f"fail to load model {args.model_name} from dir {args.model_dir}. "
                      f"exception type {ex_type}, exception msg: {ex_value}")
        # model.ready = False

transformers_test.py
import kfserving
import xgboost as xgb
import numpy as np
from xgboost import XGBModel
import os
from typing import Dict
import joblib
import logging

BOOSTER_FILE = "model.bst"
SCALER_FILE = "scaler.joblib"

logging.basicConfig(level=kfserving.constants.KFSERVING_LOGLEVEL)

class XGBoostModel(kfserving.KFModel):
    def __init__(self, name: str, model_dir: str, nthread: int,
                 booster: XGBModel = None):
        super().__init__(name)
        self.name = name
        self.model_dir = model_dir
        self.nthread = nthread
        if booster is not None:
            self._booster = booster
            self.ready = True
        self.load()

    def load(self) -> bool:
        model_file = os.path.join(
            kfserving.Storage.download(self.model_dir), BOOSTER_FILE)
        scaler_file = os.path.join(
            kfserving.Storage.download(self.model_dir), SCALER_FILE)
        self._scaler = joblib.load(scaler_file)
        self._booster = xgb.Booster(params={"nthread": self.nthread},
                                    model_file=model_file)
        self.ready = True
        return self.ready

    def preprocess(self, inputs: Dict) -> Dict:
        """Pre-process activity of the Input data.
        Args:
            inputs (Dict): KFServing http request
        Returns:
            Dict: Returns the request input
        """
        logging.info("Request body: {}" \
                       .format(inputs))

        scaled_req = self._scaler.transform(np.array(inputs["data"]))
        logging.info("Scaled request body: {}" \
                       .format(scaled_req))
        return {'data': scaled_req}

    def predict(self, request: Dict) -> Dict:
        try:
            logging.info("Pre-processed request body: {}" \
                       .format(request))
            # Use of list as input is deprecated see https://github.com/dmlc/xgboost/pull/3970
            dmatrix = xgb.DMatrix(request['data'], nthread=self.nthread)
            result: xgb.DMatrix = self._booster.predict(dmatrix)
            return {"predictions": result.tolist()}
        except Exception as e:
            raise Exception("Failed to predict %s" % e)

transformer_model_repository.py
from kfserving.kfmodel_repository import KFModelRepository
import os
from kfserving.kfmodel_repository import KFModelRepository, MODEL_MOUNT_DIRS
from xgbserver import XGBoostModel

class XGBoostModelRepository(KFModelRepository):
    def __init__(self, model_dir: str = MODEL_MOUNT_DIRS, nthread: int = 1):
        super().__init__(model_dir)
        self.nthread = nthread

    async def load(self, name: str, ) -> bool:
        model = XGBoostModel(name, os.path.join(self.models_dir, name), self.nthread)
        if model.load():
            self.update(model)
        return model.ready

transformer.yaml
apiVersion: serving.kubeflow.org/v1beta1
kind: InferenceService
metadata:
  name: custom-xgb
spec:
  transformer:
    containers:
    - image: swapkh91/custom-xgb2:latest
      name: kfserving-container
      imagePullPolicy: IfNotPresent
      env:
        - name: STORAGE_URI
          value: gs://kfserving-custom-models/xgb-iris/model

  predictor:
    xgboost:
      protocolVersion: "v2"
      storageUri: gs://kfserving-custom-models/xgb-iris/model
      image: swapkh91/custom-xgb2:latest
      imagePullPolicy: IfNotPresent

Initial all steps same as to deploy a custom xgbserver.
Keep "data" or something as key for your POST req (check preprocess func), "instances" or "inputs" will give error - Expected "instances" or "inputs", those are expected to be a list, but the inputs are a 2D array in this case
@yuzisun this looks fine to me but do you have any suggestions to improve?

I do not think you need the transformer in this case as you are doing both preprocess and predict in the model.


CBoardArray::GetCell(int row, int col) argument order

Typical cartesian coordinate system use orders its arguments as (x, y), where x is the horizontal and y the vertical coordinate.  However, CBoardArray::GetCell(int row, int col) orders its arguments as (y, x).  I have looked at the GetCell() implementation, and I am reasonably sure that the argument names match their use, so it's not a matter of incorrect argument labels.
What is the reason for ordering the arguments this way?  I regret to say it, but this order causes me cognitive dissonance.

What is the reason for ordering the arguments this way? I regret to say it, but this order causes me cognitive dissonance.

All the calls that spec out row and column use the ordering of C arrays (row major). That ordering always felt natural for such things. I've never bumped into a foobar(col, row) pattern (although I really didn't go search for them either.) This convention is as old as the code base is.
I'm aware this conflicts with the Cartesian view that x and y are ordered such a way with the independent variable first. But in general they should be in the same context.
So it's a matter of choosing one's favored dissonance. Generally I don't think there has been any confusion. I would say the only complaint CB received with reversing these was in one of the board creation dialogs where rows and cols were specified and the resulting y and x dimensions shown that felt out of order.
-Dale