Cython/PYX compilation failure tracebacks are excessively long.

Note: This is a very minor, quality-of-life annoyance.
I'm working on a relatively complex project that uses several cython modules, with additional pyxbld mediated c++ source files.
Right now, when I'm working on the C++ sources, cython compilation failures yield massively long tracebacks. The common failure output is 100+ lines.
This wouldn't be too much of a problem, if cython did not currently output the C++ compilation failure, and then the multiple python tracebacks it caused. As it is, basically all the information I need to actually fix the issue is inevitably scrolled off the screen.
Here is an example failure:

Error compiling Cython file:
------------------------------------------------------------
...

                # Queue structure:
                # [USB Thread] -> [Parse Thread] -> [Log Thread]

                self.usb_thread.insert_output_queue(self.parse_thread.get_queue_reference())
                self.parse_thread.insert_output_queue(self.log_thread.get_queue_reference())
                  ^
------------------------------------------------------------

Hardware/interface_wrapper.pyx:138:19: Object of type 'BatParserThread' has no attribute 'insert_output_queue'

Error compiling Cython file:
------------------------------------------------------------
...

                # Queue structure:
                # [USB Thread] -> [Parse Thread] -> [Log Thread]

                self.usb_thread.insert_output_queue(self.parse_thread.get_queue_reference())
                self.parse_thread.insert_output_queue(self.log_thread.get_queue_reference())
                                                                          ^
------------------------------------------------------------

Hardware/interface_wrapper.pyx:138:75: Cannot convert 'ReaderWriterQueue[UsbMessageRecord] &' to Python object
building 'Hardware.interface_wrapper' extension
arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I. -I/usr/include/ -I/usr/lib/openmpi/include/ -I/usr/include/hdf5/serial -I/usr/include/python3.4m -c /home/pi/.pyxbld/temp.linux-armv7l-3.4/pyrex/Hardware/interface_wrapper.cpp -o /home/pi/.pyxbld/temp.linux-armv7l-3.4/home/pi/.pyxbld/temp.linux-armv7l-3.4/pyrex/Hardware/interface_wrapper.o -std=c++11 -O3 -Wall -Werror -lrt -lusb-1.0 -DLINUX
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/home/pi/.pyxbld/temp.linux-armv7l-3.4/pyrex/Hardware/interface_wrapper.cpp:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
 #error Do not use this file, it is the result of a failed Cython compilation.
  ^
Traceback (most recent call last):
  File "/usr/lib/python3.4/distutils/unixccompiler.py", line 116, in _compile
    extra_postargs)
  File "/usr/lib/python3.4/distutils/ccompiler.py", line 909, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "/usr/lib/python3.4/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/usr/lib/python3.4/distutils/spawn.py", line 162, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'arm-linux-gnueabihf-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyximport.py", line 216, in load_module
    inplace=build_inplace, language_level=language_level)
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyximport.py", line 192, in build_module
    reload_support=pyxargs.reload_support)
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyxbuild.py", line 102, in pyx_to_dll
    dist.run_commands()
  File "/usr/lib/python3.4/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.4/dist-packages/Cython/Distutils/old_build_ext.py", line 185, in run
    _build_ext.build_ext.run(self)
  File "/usr/lib/python3.4/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/usr/local/lib/python3.4/dist-packages/Cython/Distutils/old_build_ext.py", line 193, in build_extensions
    self.build_extension(ext)
  File "/usr/lib/python3.4/distutils/command/build_ext.py", line 503, in build_extension
    depends=ext.depends)
  File "/usr/lib/python3.4/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/usr/lib/python3.4/distutils/unixccompiler.py", line 118, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command 'arm-linux-gnueabihf-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "acquire_only.py", line 23, in <module>
    import Hardware.acq_hardware
  File "/home/pi/BatLogger/Interface/Hardware/acq_hardware.py", line 11, in <module>
    from . import interface_wrapper
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyximport.py", line 445, in load_module
    language_level=self.language_level)
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyximport.py", line 232, in load_module
    raise exc.with_traceback(tb)
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyximport.py", line 216, in load_module
    inplace=build_inplace, language_level=language_level)
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyximport.py", line 192, in build_module
    reload_support=pyxargs.reload_support)
  File "/usr/local/lib/python3.4/dist-packages/pyximport/pyxbuild.py", line 102, in pyx_to_dll
    dist.run_commands()
  File "/usr/lib/python3.4/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.4/dist-packages/Cython/Distutils/old_build_ext.py", line 185, in run
    _build_ext.build_ext.run(self)
  File "/usr/lib/python3.4/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/usr/local/lib/python3.4/dist-packages/Cython/Distutils/old_build_ext.py", line 193, in build_extensions
    self.build_extension(ext)
  File "/usr/lib/python3.4/distutils/command/build_ext.py", line 503, in build_extension
    depends=ext.depends)
  File "/usr/lib/python3.4/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/usr/lib/python3.4/distutils/unixccompiler.py", line 118, in _compile
    raise CompileError(msg)
ImportError: Building module Hardware.interface_wrapper failed: ["distutils.errors.CompileError: command 'arm-linux-gnueabihf-gcc' failed with exit status 1\n"]

I don't see why the later tracebacks couldn't just re-raise the initial error, preventing the additional clutter. As it is, I tend to do my test-building python3 script_name.py 2>&1 | head -n 50, but shell hacks seem kind of crude.
Alternatively, why not pack the error messages from the compilation failure into the error? It already contains lines from further down the build process:
ImportError: Building module Hardware.interface_wrapper failed: ["distutils.errors.CompileError: command 'arm-linux-gnueabihf-gcc' failed with exit status 1\n"]

It seems like if the g++ stderr was attached to the python error and percolated up through the handler chain, it could at least be output at the end of the traceback block, rather then before (for that matter, if you have a custom error with a overridden __repr__() and __str__(), does the interpreter use those when it's printed to the console)?

I do not recommend using pyximport as part of a build (but I guess that's just your example).
If you call cythonize() before setup(), the C compiler will not be run at all if the translation fails, so the error output will end with Cython's own output instead of the multi-step exception tracebacks above.

Apart from that, I agree that an exception trace is not helpful at the end of a failed Cython compilation. The exception should be caught instead and printed without the traceback.

I do not recommend using pyximport as part of a build (but I guess that's just your example).

Uh.... what? How do you use cython files without building them?
Basically, I have a project that's distributed as source.  You run the project, and if needed, it builds the pyx files, the c++ files, and glues them together using a pyxbld config file, and then executes.
The point is, there's no needed separate build step. which is both very convenient (from a development perspective), and means you never have to worry about having an out-of-date version of the cython/c++ code.
I suppose I could have the cython stuff be built by a setuptools command, but that'd be annoying with no benefit. Your comments about calling cythonize() is irrelevant unless you do that.

On Sat, Aug 26, 2017 at 4:35 AM, Connor Wolf<EMAIL_ADDRESS>wrote:

I do not recommend using pyximport as part of a build (but I guess that's
just your example).
Uh.... what? How do you use cython files without building them?
Basically, I have a project that's distributed as source.  You run the project, and
if needed, it builds the pyx files, the c++ files, and glues them together
using a pyxbld config file, and then executes.
The point is, there's no needed separate build step. which is both very
convenient (from a development perspective), and means you never have to
worry about having an out-of-date version of the cython/c++ code.
I suppose I could have the cython stuff be built by a setuptools command,
but that'd be annoying with no benefit. Your comments about calling
cythonize() is irrelevant unless you do that.

Using a setup.py with setuptools is more robust (and well tested) than
pyximport, as well as being the standard way to do things.

there's no needed separate build step. which is both very convenient (from a development perspective),

pyximport is ok for testing and developing. That's its main use case, IMHO. Pull requests that improve pyximport are welcome. It has not changed in years and is not up to date with the less old improvements in Cython's build support.
IMHO, pyximport should not be imposed on users of Cython code. Definitely not by libraries. Applications are different as they can control their runtime environment, but even there it still has the drawback of making imports more complex and less controllable by end users.

and means you never have to worry about having an out-of-date version of the cython/c++ code.

People commonly include the generated (and tested) .c/.cpp files in the sdist that they publish on PyPI. That avoids a dependency on Cython and prevents users from accidentally using a different version of Cython to compile your code, which makes it more predictable what they get. If your code starts failing to build with a newer version of Cython, or if the Cython version that they happen to have installed has a bug that affects your code, users will be on their own to resolve it when they try to install an existing version of your code.

I suppose I could have the cython stuff be built by a setuptools command, but that'd be annoying with no benefit.

People commonly distribute binary wheels these days, which are very fast to install and do not depend on compilers being installed on user side, be it Cython or a C compiler. I do not know about the specific constraints of your project, but this is a proven huge benefit for the large majority of users I heard of.

In this case, the project is some custom hardware, C++ that talks to it, and python that manages the runtime.
Development hasn't even reached the point where I'd vaguely considered trying to abstract everything up into a library that gets versioned separately from the entire hardware stack.
I'd assume that the great majority of people writing cython code are not writing libraries, and as such pyximport is probably more broadly used/convenient then depending on distutils.
Note that there are probably more people using cython code via distutils then pyximport, but that's separate from people writing cython code.

I'd assume that the great majority of people writing cython code are not writing libraries, and as such pyximport is probably more broadly used/convenient then depending on distutils.

While that is an unbacked opinion, I fully appreciate people who extend the applicable range of tools beyond what they where initially designed for.
That being said, I think this ticket is getting out of scope. I'd like to restrict it the initial intention of improving the error output of pyximport.

While that is an unbacked opinion, I fully appreciate people who extend the applicable range of tools beyond what they where initially designed for.

Hah, well, I did say I assume, so I'd happily admit I have little basis.

That being said, I think this ticket is getting out of scope. I'd like to restrict it the initial intention of improving the error output of pyximport.

Ok.


Support: i've disabled mediarouter  (afik)

with these patches
https://u.teknik.io/AuwIC.xz

This seems like it breaks casting, which some users use. I don't see how Media Router violates any of ungoogled-chromium's goals, so I'm not merging this unless proven otherwise.


xorg.xauth: 1.1.0 -> 1.1.2

Changes:

1.1.1: https://lists.x.org/archives/xorg-announce/2021-November/003118.html
1.1.2: https://lists.x.org/archives/xorg-announce/2022-April/003158.html

Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

applied to https://github.com/NixOS/nixpkgs/pull/208900
thanks

Another option would be to merge it to master sooner to shrink the commit queue you have.

this would still had to have targeted staging because 501+ rebuilds so i just applied it to my PR insted of you having to rebase this

Aha, sounds reasonable.


feat(core): use focus-visible instead of focus

What/Why?
Use focus-visible instead of focus, so that we retain our focus styles for accessibility purposes, but do not style elements when focused via actions like click or touch events.
Testing
Ensured no focus styling is visible when using click or touch events, but remains when navigating via keyboard.

🍹 Not sure if you meant to add apps/core/components/header/index.tsx

Impressive catch, had modified but not saved the file, thanks.

Can you expand on why we don’t want focus to show on click?


provide an implementation of Stdlib.Semaphore.Counting and Stdlib.Semaphore.Binary

If you stick to the same interface, then people can easily switch between the two implementations.
https://v2.ocaml.org/api/Semaphore.Counting.html
https://v2.ocaml.org/api/Semaphore.Binary.html
You said you can be much more efficient, right?

You said you can be much more efficient, right?

No.  I don't think I said quite that.
What I did say is that the Stdlib semaphores aren't very fast, and that they don't work nicely with effects based schedulers, and that we already have lock-free queues written in OCaml that are faster than the Stdlib semaphores.
I also gave an example of how one could implement a semaphore using Kcas.  Here is the example for future reference:
module Binary_semaphore : sig
  type t
  val make : bool -> t
  val release : t -> unit
  val acquire : ?timeoutf:float -> t -> unit
  val try_acquire : t -> bool
end = struct
  type t = [ `Available | `Unavailable ] Loc.t
  let make available =
    Loc.make ~padded:true (if available then `Available else `Unavailable)
  let release semaphore = Loc.set semaphore `Available
  let acquire ?timeoutf semaphore =
    Loc.modify ?timeoutf semaphore @@ fun state ->
    if state == `Available then `Unavailable else Retry.later ()
  let try_acquire semaphore =
    Loc.compare_and_set semaphore `Available `Unavailable
end

I gave that example to demonstrate that you can use Kcas for synchronization like you use traditional synchronization primitives.  I also explicitly said that the semaphore implementation might not be the best possible.  I said that because I know that there are many trade-offs related to the implementation of such synchronization primitives.  The simple implementation I gave likely isn't too bad under low contention, but it will definitely be possible to implement semaphores with (both) lower overhead and better contention handling.
However, I didn't give that example of a semaphore to argue that you should implement semaphores using Kcas.  That would basically be an example of abstraction inversion.  That was not my point.  The point was:

STM makes it possible to implement the communication and synchronization patterns that the problem at hand requires.

There aren't (m)any problems where a semaphore, in particular, is required.  Yes, you can solve many problems with semaphores, but you can also solve those same problems using different abstractions.  As an example, see The dining philosophers problem solved using Mvars instead of (binary) semaphores.  And, no, I'm not suggesting that you should use Mvars to solve every problem.  As another example, see The sleeping barbers problem that is currently not in main as it awaits a proper bounded queue implementation.  And, TBH, I generally am not a big fan of toy examples like that, but I can't write examples based on actual problems people haven't yet encountered and asked about.
More to the point, while you definitely can implement replacements for traditional synchronization primitives with Kcas, I would not recommend that.  Also, I would not recommend adding such replacements to the Kcas or Kcas_data libraries.  The main reason for that is that traditional synchronization primitives do not generally compose in a way that would allow them to work nicely with other transactional abstractions.  I talk about fundamentally the same issue in my blog post on Kcas here.
OTOH, providing examples of traditional synchronization primitives implemented using Kcas is fine, IMO, as long as one mentions the caveats (e.g. they are not solving an actual application level problem, they do not compose, you can implement them better without the abstraction inversion).

Picos comes with replacements for the Stdlib Mutex, Condition, and Semaphore modules (amond some other Stdlib replacements).  Those implementations are currently optimized for memory use (i.e. minimize memory use) and low contention use cases and in those cases they outperform the Stdlib versions.  The implementations are also specifically designed for effects based schedulers.  In high contention scenarios the current implementations in Picos will not perform ideally.  For such use cases I would recommend A) rethinking the architecture to avoid contention (locks will not give you parallel scalability — at best you won't degrade as badly under contention), B) using high performance lock-free data structures (including Kcas) instead of locks, C) implementing separate locking primitives optimized for high contention.


[Java] added *.properties

*.properties
Reasons for making this change:
TODO
Links to documentation supporting these rule changes:
TODO
If this is a new template:

Link to application or project’s homepage: TODO

@xxammuxx any links to StackOverflow discussions or other documentation about this change?

The Spring project, Java's most popular web application framework (see https://dzone.com/articles/most-popular-java-web-frameworks for example), relies on those heavily ; they should definitely not be ignored. See : https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html

@Tristan971 thanks for the link. And these paths can be safely ignored from version control? Users won't expect these files to be committed and tracked?

@xxammuxx given the link from @Tristan971 above, does it still make sense to add this rule?

No they indeed cannot be ignored and should 100% be version controlled.
Unfortunately I do not believe this PR is good in the long term.

@Tristan971 is right about it. This can be ignored.


Fix typo for number of total queries

Changes

fixed small typo in second onboarding task doc

Good catch. Can you please consolidate into #2277 ?


[FEATURE] Support Filesystem expansion for encrypted volumes

Is your feature request related to a problem? Please describe.
We currently utilize the longhorn-manager to expand the filesystem on the block device.
This should be moved into the csi driver + share-manager for rwx volumes.
Additional context
PV encryption is implemented in https://github.com/longhorn/longhorn/issues/1859
csi expansion is tracked at https://github.com/longhorn/longhorn/issues/2794

@joshimoo We need to backport this to 1.2.x. Please help create the backport issue and related PRs. Thanks.

We will also need to backport https://github.com/longhorn/longhorn/issues/2794
as well to have encryption expansion support.
@innobead Will make the necessary backports, we currently don't have a branch for v1.2.3 on the longhorn repro yet, so I waited with the chart PR to not modify the master head. Can you create the appropriate release branch on longhorn/longhorn

We will also need to backport #2794 as well to have encryption expansion support.
@innobead Will make the necessary backports, we currently don't have a branch for v1.2.3 on the longhorn repro yet, so I waited with the chart PR to not modify the master head. Can you create the appropriate release branch on longhorn/longhorn

Just created :)

@joshimoo Please help backport to 1.2.x then we can move this to ready for testing. Thanks.

Created https://github.com/longhorn/longhorn/issues/3240 for backport.
Validation - Pass on master-head 11/11/2021
Validated the expansion with below scenarios

RWO encrypted volume created using Longhorn UI.
RWX encrypted volume created using Longhorn UI.
RWO and RWX encrypted volume created using storage class.
Restored encrypted volume expansion.

Logs:
csi.log


Resource Id's mismatch between the original FHIR resource and the corresponding resource in the parquet file.

During the process of transforming FHIR resources into parquet files, an issue arises where the ID of the resource in the transformed parquet files does not match the original FHIR resource id.
patient resource:
{
    "resourceType": "Patient",
    "id": "6f866a1c-c1af-40a0-94c6-cab20b7f33c2",
    "meta": {
        "versionId": "1",
        "lastUpdated": "2023-07-04T04:54:55.663+00:00",
        "source": "#7xtul0v7pPE2VN0v"
    },
    "text": {
        "status": "generated",
        "div": "

PATIENT TEST

Identifier	P8654

Address	Abc

Date of birth	15 July 1997

"
    },
    "identifier": [
        {
            "system": "http://example.com/identifiers/patient-card",
            "value": "P8654"
        }
    ],
    "name": [
        {
            "family": "TEST",
            "given": [
                "PATIENT"
            ]
        }
    ],
    "telecom": [
        {
            "system": "phone",
            "value": "+91-0070707070070"
        }
    ],
    "gender": "female",
    "birthDate": "1997-07-15",
    "address": [
        {
            "line": [
                "Abc"
            ]
        }
    ],
    "contact": [
        {
            "telecom": [
                {
                    "system": "phone",
                    "value": "+91-000707070777"
                }
            ]
        }
    ]
}

Content in the parquet file:
{
    "id": "82656",
    "meta": {
        "versionId": "1",
        "lastUpdated": "2023-07-04T04:54:55.000+00:00"
    },
    "text": {
        "status": "generated"
    },
    "identifier": {
        "array": [
            {
                "system": "http://example.com/identifiers/patient-card",
                "value": "P8654"
            }
        ]
    },
    "name": {
        "array": [
            {
                "family": "TEST",
                "given": {
                    "array": [
                        "PATIENT"
                    ]
                }
            }
        ]
    },
    "telecom": {
        "array": [
            {
                "system": "phone",
                "value": "+91-0070707070070"
            }
        ]
    },
    "gender": "female",
    "birthDate": "1997-07-15",
    "address": {
        "array": [
            {
                "line": {
                    "array": [
                        "Abc"
                    ]
                }
            }
        ]
    },
    "contact": {
        "array": [
            {
                "telecom": {
                    "array": [
                        {
                            "system": "phone",
                            "value": "+91-000707070777"
                        }
                    ]
                }
            }
        ]
    }
}

Thanks @vbothe23 for filing this issue; are you using the FHIR-search mode or direct DB access when fetching resources? Is it possible that in the source FHIR server, there is also a Patient resource with id 82656? This behavior is unexpected and the IDs should be preserved.

Thanks @vbothe23 for the clarification; yes this is a bug that we should fix. In the meantime you can use the FHIR search mode (i.e., avoid JDBC) and I believe that should return the original resource IDs.

This was fixed by #754 thanks @vbothe23.


Request // PDF Export Features

Describe the feature you'd like
Updated 2018.1206 to create a concise list of desired features pertaining to the PDF export function.

Print view for the WYSIWYG editor and MD editor preview pane; similar to MS Word's Print Layout (View > Print Layout, or default view). I think this would be better suited as a toggle in the editors because I do not believe that a simple print preview-esque feature would be sufficient as it wouldn't allow dynamic control over breaks.

Ability to adjust margins at either a page or global level; e.g. Word has narrow, wide, etc... custom not necessary.
Ability to dynamically insert/edit page breaks.

Option to set quality of PDF export (currently exports using low quality).
... any others?

Describe the benefits this feature would bring to BookStack users
With the goal of using Bookstack for client project documentation, sometimes hard copies are required for planning meetings/discussions. Having control over page breaks greatly improves printed presentation as it helps avoid content spanning printed pages; e.g. tables, lazyetc.
Additional context
None.

For pagebreak try this code:

While that can be used to specify a page break location, that doesn't give a real-time print view of where the break is on the page based on how it is going to view when printed.
The best way to further describe the request would be analogous to editing a MS Word doc in View > Print Layout (default view).

@ssddanbrown Apologies if the issue title and content update causes confusion but I wanted to have a single issue to place enhancement requests pertaining to PDF export instead of filing them separetly.

Hmm, @ssddanbrown I found the disable-smart-shrinking option but when I try to add it to bookstack/config/snappy.php I get the following;
InvalidArgumentException
The option '0' does not exist.

Here is my bookstack/config/snappy.php
<?php

return [
    'pdf' => [
        'enabled' => true,
        'binary'  => file_exists(base_path('wkhtmltopdf')) ? base_path('wkhtmltopdf') : env('WKHTMLTOPDF', false),
        'timeout' => false,
        'options' => ['disable-smart-shrinking'],
        'env'     => [],
    ],
    'image' => [
        'enabled' => false,
        'binary'  => '/usr/local/bin/wkhtmltoimage',
        'timeout' => false,
        'options' => [],
        'env'     => [],
    ],
];

I tried enclosing disable-smart-shrinking with {} and () and this did not work either. I'm not an expert here (especially with PHP), so any help would be greatly appreciated.

@derek-shnosh I believe the option would be provided like so:
        'options' => [
            'disable-smart-shrinking' => true,
        ],

@ssddanbrown Thanks, that format was accepted... however it doesn't seem to help.
The reason I was looking for this option is because the fonts are a bit skewed after an export, below is an example with the Work Sans, Muli, and Fira Code fonts; it seems their upper right corners are a bit hacked.

If I open the PDF with Acrobat Standard I can see the fonts are assigned properly. Also, when editing the text field, new characters typed are quite clear.

Now that I know how to assign the options properly, I'm going to dig through them to see if there is soemthing to help with this.

As discussed here https://github.com/linuxserver/docker-bookstack/issues/32 it would be good if the PDF options would be either produce a nicer output, meaning that the output works better on an A4 page, or that there options for the .env file because customizing the snappy.php file means that we're editing the source code directly.

I'm going to close this off since this has had relatively little demand while being a feature that is quite open and would be a relative pain to implement & maintain. If needed, we have since made it easier to customize exports using header CSS or the visual theme system.


chore: Added default value for Path fill type (same with android API)

@DimitrisRK is it intentional to be a draft?

@DimitrisRK is it intentional to be a draft?

Yes, I leave it as is so that we can commit possible improvements.


return to home not consistently available

I guess this is a shortcoming of the selected template for the website, but there isn't any consistently placed / available link on the subordinate pages to jump back to the website home page.
The Previous Workshops page includes a "Return to Home" link in its content. However, since the link is under a heading that also reads Return to Home, you get a Return to Home item in the TOC list at left that's non-functional, which is somewhat confusing.

Each page has a (or at least should have a) return to home "section" that includes a title ("return to home") and a link that actually returns you to home. The Table of Contents lists the section headers. So you click on 'return to home' in the sidebar and it brings you to the section "return to home" where you have to click on the actual link to get returned to home. Suggestions welcome on how to do it differently.

OK, I see that. It's a bit unconventional, but unless the theme offers some way to add a home page link to the header so something appears there consistently, there's probably not much else that can be done.


Add test coverage or status checks on upload

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.20%. Comparing base (abaa460) to head (111ffa7).
Report is 30 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #19   +/-   ##
=======================================
  Coverage   94.20%   94.20%
=======================================
  Files           6        6
  Lines         328      328
=======================================
  Hits          309      309
  Misses         19       19

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[Fix] Switch: Implement optionnal text

The text is mandatory currently, we must update the UIKit component to have an optional text

PR:

https://github.com/adevinta/spark-ios/pull/445
https://github.com/adevinta/spark-ios/pull/455

Hey @robergro , only for UI kit I noticed a desynchro between toggles when you do this specific journey :

when arriving on the page, turn the demo switch off
Then turn the "is On" switch on --> then it's desynchronising

https://github.com/adevinta/spark-ios/assets/142396647/fc260bc6-ebc8-4b0e-8e9c-032685d6f29a

I reproduced it this morning too. After some investigation, the problem is just on our demo.
We need to simplify the UIKit component demo and fix some troubles like this...
Sorry ...!

A ticket is dedicated to this bug --> https://github.com/adevinta/spark/issues/1496


 We should be using them. Off the top of my head:

[x] pep8
[x] pylint
[ ] pyflakes
[ ] mypy

We've been using pep8 for a while. MyPy is the only one of these where integration may be a bit tricky; I like the idea behind it, but the particulars of the type system they've constructed rule out a lot of very reasonable python idioms, and it's a bit awkward when your dependencies don't use it.
Pylint is very noisy, and integrating it is probably going to mean turning off some warnings at least at first, and turning them back on as we fix things.

pep8 -> pycodestyle.

@zenhack what do you think of coveralls? It will force users to write tests every time new code is added. Though, we can make it optional and not required for making the merge button green.

Adding something that does test coverage to this list seems like a good idea to me. I haven't used coveralls before, so can't comment on it as a specific tool, but would not be opposed.

do we really need pyflakes along with pylint?

Probably not, the list was more brainstorming than anything else. I'm fine skipping that one.


No internet search option with the One-click Installers?

I installed the latest One-click Installer (using windows11)
I can see most features but the one I really interested in is the web search option like in the website https://gpt.h2o.ai/

Running locally I dont see that option.
Nov 05, 2023:
h2oGPT GPU-CUDA Installer (1.9GB file)
Is that only available in the latest build from scratch option ?
Or is there a way to enable it?

It's part of windows install via this package:
https://github.com/h2oai/h2ogpt/blob/bede54b665b1c630a43ff09a918e3d17020dcac1/windows_installer.cfg#L99
One requires to do things mentioned here:
https://github.com/h2oai/h2ogpt/blob/9560e4401d18255a1ef370df6aa80f82fdc604c5/docs/README_SerpAPI.md
i.e. need API key.  They give free searches up to some limit.

Can the feature work without api? I think it would benefit users to fetch the results locally.
🙏

No way I'm aware of.

No way I'm aware of.

Ive been playing around with the huggingface chat-ui (https://github.com/huggingface/chat-ui)
It also had only api option for search queries, but they added to fetch it locally without api recently, maybe you can look at how they implemented it?


Pricing in orders

added pricing to the constants file
implemented individual pricing to each item added
raise price for each item based on the quantity
@pattnaik-soumitri

@pattnaik-soumitri


Remove CSW target account ID

The latest version of CSW will not need target account ID.
Related PRs:

https://github.com/alphagov/csw-client-role/pull/14
https://github.com/alphagov/govuk-aws-data/pull/358

Just ran tf fmt and force pushed


Completion support

What would you like to have changed?
Having completion support like arc completion [zsh|bash|fish]
Why is this feature a useful, necessary, and/or important addition to this project?
Useful to install completion with ease and make command usage easier.
What alternatives are there, or what are you doing in the meantime to work around the lack of this feature?
N/A

This would be cool, yeah.
However, as I don't have plans/time to make a command for v4 (unless sponsorships motivate it specifically), I'll close this issue; but I've labeled it with "command" so we can remember this if someone does end up contributing a command later. Thanks!


sql: delete index if there is an error when it's added to the registry

Related to https://github.com/src-d/gitbase/issues/411
This is the simplest solution. If we want to avoid create the index if it can't be added to the registry, we must mix a little bit the create index and add index logic to rely on the IndexRegitry.AddIndex method to create the index instance. WDYT @src-d/data-retrieval ?

Ok, it may work because in pilosa we have frames/fields as a combination of expression AND id

Lets add test for this case

@mcarmonaa - I think there is no need to add it to gitbase, but
https://github.com/src-d/go-mysql-server/blob/master/sql/index_test.go#L138
just tries to add exactly the same index. I thought about low level test, where you try to add an another index with different id but the same expression(s). When it fails the index will be deleted but should still have an access to the previous one.

@kuba--  https://github.com/src-d/go-mysql-server/blob/master/sql/index_test.go#L131 is adding an instance of a new index with a different id

I would prefer do not create a valid index folder when Create() method is called. We can create the .processing file at that moment. Then, we will throw an error saying that another index is using the same expression. After that, if we restart gitbase, the invalid index will be automatically deleted. WDYT?

@mcarmonaa - ok, you're right (my bad). so maybe we can keep the old instance and again try to load the index.
@ajnavarro - so far .processing is created in Save. It's not a problem to move it to Create, but it means:

All created indexes at the beginning will be invalid (til you save something).
Processing will not be atomic operation, because we will create .processing in one function (Create) and remove it in other function (Save) after saving.

so maybe we can keep the old instance and again try to load the index.

:+1:

All created indexes at the beginning will be invalid (til you save something).

Is it an index useful if you don't save something on it?

@mcarmonaa - it's just an empty index (no data)
I just wondering if it's possible no to create any index, I mean just compare expressions in potential new index with existing indexes expressions. Instead of first create and later validate

Which solution will we go ahead with finally? the .processing file one?

@mcarmonaa If it is possible to compare before the Save() method if the expression is already indexed, do it.
If not, create the processing file on Save().
I would prefer the first approach.

What if we modify func (r *IndexRegistry) validateIndexToAdd(idx Index) error  to sth, like:
func ValidateIndex(db, table string, expr []sql.Expression) error and move it from sql/index.go to sql/plan/create_index.go.
After that we can call it in func (c *CreateIndex) RowIter(ctx *sql.Context) (sql.RowIter, error) instead of  c.Catalog.AddIndex(index)
c.Catalog.IndexRegistry.ValidateIndex(db, table, expr)

WDYT?

Generally we can split current functionality c.Catalog.AddIndex(index) by Validate and Add
We can even (just in case) leave current validation in AddIndex but add/make public a new one (lower level) which we can call before AddIndex

...but if it's too complicated, lets go with .processing approach. The only thing which I would suggest to add is,  maybe we can add sth. to the file (append content) like a progress counter, so we'll know if it's just created index, saving or failed during saving, etc.

I have implemented something similar to what you say @kuba--, the problem I see is that the process validate -> create -> add should be atomic, shouldn't be it?
So we need something like:
func (r *IndexRegistry) CreateAndAddIndex(
	driver IndexDriver,
	db, table, id string,
	expressions []Expression,
	config map[string]string,
) (idx Index, created chan<- struct{}, ready <-chan struct{}, err error)

WDYT @ajnavarro @kuba-- ?

IndexRegistry shouldn't create any Index.

imo, each of them are atomic operation (just to test it out) and CreateAddIndex is just a wrapper on top of it.

Ok, let's with the .processing file approach then


install_docs

Purpose/implementation Section
What changes are being implemented in this Pull Request?
What was your approach?
What GitHub issue does your pull request address?
Tell potential reviewers what kind of feedback you are soliciting.
New Content Checklist

[ ] New content/chapter is in an Rmd file with this kind of format and headers.

[ ] New content/chapter contains Learning Objectives and are in the correct format.

[ ] Bookdown successfully re-renders and any new content files have been added to the _bookdown.yml.

[ ] Spell check runs successfully in Github actions style-n-check).

[ ] Any newly necessary packages that are needed have been added to the Dockerfile and image.

[ ] Images are in the correct format for rendering.

[ ] Every new image has alt text and is in a Google Slide.

[ ] Each slide is described in the notes of the slide so learners relying on a screen reader can access the content. See https://lastcallmedia.com/blog/accessible-comics for more guidance on this.

[ ] The color palette choices of the slide are contrasted in a way that is friendly to those with color vision deficiencies.
You can check this using Color Oracle.

Not sure why it failed. Any suggestion to fix. @cansavvy

⚠ There are spelling errors that need to be addressed. Read this guide for more info.
Download the errors here.
Comment updated at 2022-02-10 with changes from 6b57d77

Take a look at the comment and guide from above, there’s some spelling errors,

Thank you. I have added some words in the dictionary

Thank you. I have added some words in the dictionary

Looks like it passed! 🎉


[Bug]: commit-and-sync doesn't pull when there is no files to commit

Describe the bug
This occurs when I run the commit-and-sync with the pull enabled. When there is changes to commit all goes all, but for me it seems that when there is no new changes to push the pulling part of the operation is aborted
Relevant errors (if available) from notifications or console (CTRL+SHIFT+I)
No response
Steps to reproduce

Make changes in the repository (via another Obsidian instance or via file)
Go to Obsidian and do the commit-and-sync action

Expected Behavior
Even that there is nothing to commit, the sync part of the command should occur anyways
Addition context
No response
Operating system
Windows
Installation Method
None
Plugin version
2.27.0

Not pulling, when no commit happened, was the expected behavior. It suited the previous phrasing "Backup", but for the new phrasing "commit-and-sync" I think you are right. So it now tries to pull on commit-and-sync as well. Though, just to mention it, there is and was the extra interval for pulling as well.

@dkbarn Thanks for the info!
This is now released in 2.28.0 :tada:


ilmControl: add support for saving screenshot as png file

To save as png file, the file name should have ".png" as extension.
Signed-off-by: Ramzi BEN MEFTAH<EMAIL_ADDRESS>

@rbmeftah Just noticed, there are 2 very similar commits 7e3d04c15ec3265ef3a0d42f4ab2767a70072def and ae75d7ccfb2cfc8a2af8b7b1b36fe77d5729bb1f. One of them should be dropped. And the merge commit e545276fd4566e536f2e011339aeb3280b49070e seems to be redundant.

forced push of a new single commit

@rbmeftah looks like rad and blue color components are swapped in the .png dumps , just take a dump with some colors which are not black and white, e.g. EGLWLMockNavigation will show it. maybe some BGR to RGB miss match.

0001-patch-v2.patch.txt
attaching the diff file for the last commit


Try It examples where IDs or tokens cause 400 bad request errors

Repro:
https://docs.microsoft.com/en-us/graph/api/event-get?view=graph-rest-1.0&tabs=http#request-1
In our demo environment that is used for anonymous (non signed in GE) these snippets will not work. The sample is
https://graph.microsoft.com/v1.0/me/events/AAMkAGIAAAoZDOFAAA=?$select=subject,body,bodyPreview,organizer,attendees,start,end,location
The event-id is the AAMkAGIAAAoZDOFAAA= part. In some cases in docs it is also {event-id}.
When you Run Query these will fail with 400 bad request because the  AAMkAGIAAAoZDOFAAA=  is not a full length id like AAMkAGVmMDEzMTM4LTZmYWUtNDdkNC1hMDZiLTU1OGY5OTZhYmY4OABGAAAAAAAiQ8W967B7TKBjgx9rVEURBwAiIsqMbYjsT5e-T7KzowPTAAAAAAENAAAiIsqMbYjsT5e-T7KzowPTAACNM9xPAAA=.
In the docs they are shortened for readability. They also wouldn't be useful anyway as if user is signed in the ids would be different. And we don't have all content in for each workload even in our demo environment. E.g. intune wouldn't return anything in our demo environment as not even configured, so users need to sign in.
The docs v-team state that this is A LOT of work and that for now the plan would be to address this issue based on current state.
We are aware of this experience issue and auto-suggest will help here when we get to this feature. As it can reverse engineer knowing it needs a event-id and call /me/events and pick the first result and use its id. This should be tracked as an additional work item.
In the near term, can we at least investigate pattern matching any tokens in request urls e.g. {event-id} and format them in the textarea rendered on the screen. To make it obvious that they need to supply an event-id. E.g. make the background of {event-id} or AAMkAGIAAAoZDOFAAA= text in bright yellow and bold.
Can we also if they do Run Query, that we show a helper box that tells them that the reason they got this is because they need to update the url (or request body) to ids and values that are in their environment. We could provide the get event example as explanation in that text.

We could also disable the Run Query button if there are {event-id} type things in the url.

Other examples
{id} two diff ids {teams-id} (channel-id} - https://docs.microsoft.com/en-us/graph/api/channel-get?view=graph-rest-1.0&tabs=http
{id | userPrincipalName} https://docs.microsoft.com/en-us/graph/api/user-get?view=graph-rest-1.0&tabs=http#example-3-users-request-using-select
{idOrUserPrincipalName} https://docs.microsoft.com/en-us/graph/api/drive-get?view=graph-rest-1.0&tabs=http#http-request-1
{school-id} https://docs.microsoft.com/en-us/graph/api/educationschool-get?view=graph-rest-1.0&tabs=http#request
AAMkAGIAAAoZDOFAAA=  - not a token a sample id (i don't know how we'd find these) - https://docs.microsoft.com/en-us/graph/api/event-get?view=graph-rest-1.0&tabs=http
We could actually get real smart
# [HTTP](#tab/http)
<!-- {
  "blockType": "request",
  "name": "get_educationschool",
}-->
```msgraph-interactive
GET https://graph.microsoft.com/v1.0/education/schools/{school-id}

and add a new attribute to the comments below the blockType and name here and get the docfx team to send this through to Graph Explorer. We could then get the doc writer to provide the api call to get things like the school-id based on a graph query or even just provide helper text.
like
"helpText":"To get a {school-id} you can run https://graph.microsoft.com/v1.0/education/schools",
"autoComplete":{ "school-id", "https://graph.microsoft.com/v1.0/education/schools" }

@thewahome The idea is to discover instances in docs where we have IDs shared, then replace that with {Insert Id here} or any other tip to show there's more data required to run that query.


ListCollection example crashes

Project
ListCollection
Environment
Please include at least the following details:

Xcode version: Version 12.0 beta 2 (12A6163b)
iOS version: iOS 14
Device or simulator used to reproduce the bug: simulator

Describe the bug
Check out CodeExamples. Open the ListCollection example and run it. This will result in the following:
DiffableDataSource[68322:3360678] -[UICollectionViewListCell _updateCurrentSystemTypeAccessoryViews]: unrecognized selector sent to instance 0x7fa823f26f40
-[UICollectionViewListCell _updateCurrentSystemTypeAccessoryViews]: unrecognized selector sent to instance 0x7fa823f26f40

Odd, I'm not seeing any problem.

Yeah, that is weird. I am also using Catalina 10.15.5 (19F101).

Me to, didn't upgrade to Big Sur. I wonder what is different?
What is the error partially hidden at the top of the console? Is it complaining about finding several versions of the framework? I wonder if it is somehow linking against the wrong SDK?

2020-07-09 14:47:38.146576+0200 List[97404:3977423] libMobileGestalt MobileGestaltCache.c:38: No persisted cache on this platform.
objc[97404]: Class VCWeakObjectHolder is implemented in both /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneOS.platform/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS.simruntime/Contents/Resources/RuntimeRoot/System/Library/PrivateFrameworks/AVConference.framework/Frameworks/ViceroyTrace.framework/ViceroyTrace (0x130730e40) and /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneOS.platform/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS.simruntime/Contents/Resources/RuntimeRoot/System/Library/PrivateFrameworks/AVConference.framework/AVConference (0x12ff0a3f0). One of the two will be used. Which one is undefined.
2020-07-09 14:47:39.453690+0200 List[97404:3968754] -[UICollectionViewListCell _updateCurrentSystemTypeAccessoryViews]: unrecognized selector sent to instance 0x7fb29d741080
-[UICollectionViewListCell _updateCurrentSystemTypeAccessoryViews]: unrecognized selector sent to instance 0x7fb29d741080
(null)

I don't see that in my build log and who knows if it is related? There's a two year old thread in the forums which mentions it:
Class VCWeakObjectHolder is implemented in both
Not sure what to suggest other than trying the magic incantation to the Xcode gods:

reset simulators
clean build folder
delete derived data

Yeah, that didn't work either. I will try again with the next Xcode beta.

Frustrating. I'll leave this open, maybe someone else is also having the same issue

Seems to be fixed in Xcode Version 12.0 beta 3 (12A8169g)


Some optimizations

cc #79, now this should have significant wins.
r? @anthrotype

I like the idea of disabling notifications while the objects are being loaded.

That has significant overhead. disableNotifications() has a cost, and then it's still gonna do _get_dispatcher() which in some cases is a chain of 6 function calls and lookups/weakref creation (see glyph guidelines impl), and it's still gonna do postNotification() which will do quite some things (see the code) before figuring out "oh this is disabled".
What's the point of all this when we know we're loading the data and we don't even need to look it all the way. And at that point we're still eating most of the price (actually send the notification when you have no subscriber is not a costly thing to do). disableNotifications() will only relieve you of the cost of subscribers (which is nil when loading the font), not of the notifications machinery itself.
But the core of the problem is ufoLib naively does glyph.anchors = anchors, glyph.guidelines = guidelines and each of these in the defcon APIs take each element one-by-one, calls a new function, does the notifications thing, does additional safety checks even though ufoLib already does way more checking than is necessary.. We need to reuse the data structures made by ufoLib and not rescrutinize every single item when ufoLib is meant to do it so that we don't have to.

By the way here is my perf document: https://gist.github.com/adrientetar/936be611f7b805590c80ebe0e273c2dc

I dont know, I dont like all the duplicate code...
A _isLoading tag is very welcome and handy, but I guess this can be simply handle in:
def postNotification(self, notification, data=None):
    if hasattr(self, "_isLoading") and self._isLoading:
        # dont send any notifications
        return
    ...

and Im also wondering if most is not already captured by an object not having a parent and no dispatcher at all, so no postNotification...
could you post your test cases you mentioned here? https://gist.github.com/adrientetar/936be611f7b805590c80ebe0e273c2dc

yeah, there is indeed some duplication. It would be nice if we can avoid that

could you post your test cases you mentioned here?

https://github.com/google/roboto/tree/master/src/v2/Roboto-Regular.ufo

and Im also wondering if most is not already captured by an object not having a parent and no dispatcher at all, so no postNotification...

The slowdown isn't just notifications though (for containers it calls a function for each element making additional checks but we know ufoLib data is sounds) and sometimes even _get_dispatcher ends up taking a long time.

The test suite should be pretty robust in guaranteeing the additional code paths are correct, earlier it catched an error with handling of identifiers for instance.


Raf teal compile

Provided samples and code snippets to support SDK TEAL compile. Examples for Contract Account and Account Delegation now have , compile a sample TEAL program, Use the results in the Make Logic Sig c all and use a string argument. Java, JavaScript, Go and Python.

replaced with https://github.com/algorand/docs/pull/171 (just samples here in 171)


Cloud Policies missing

I am attempting to setup Kubescape Operator in an EKS environment but have been unsuccessful in determining what AWS permissions are required when specifying the cloudProviderMetadata.awsIamRoleArn section of the Helm chart. I found the documentation lacking for all Cloud Providers but was able to find this: https://hub.armosec.io/recipes/setup-aws-iam-authorization-of-in-cluster-installation-of-kubescape-in-eks. However, it appears this may be outdated as the Helm chart requires only a single ARN and the recipe creates two service accounts with their own policies. Additionally, the recipe specifies the --set createKubescapeServiceAccount=false parameter when doing the Helm install which does not exist in the current Helm chart.
Is it possible to get documentation on what is required for creating the AWS accounts/permissions needed for use with Kubescape Operator?

@matthyx this seems like a documentation bug. Please add it to the board.


Travis should use nodejs 14

Should fix the CI error:

error<EMAIL_ADDRESS>The engine "node" is incompatible with this module. Expected version "^8.10.0 || ^10.13.0 || >=11.10.1". Got "10.4.0"

not that there are a lot of tests in this package though 🤪


:image_optim is deprecated. Please use :imageoptim instead.

Even using :imageoptim, it's still showing this message.

Anyone got this to work? I was having the same issue.

Use this
gem "middleman-imageoptim", :git => "https://github.com/plasticine/middleman-imageoptim", :branch => "master"
bundle install again

This is still happening for me with the git source above

+1

+1

+1

same for me. Tried everything possible.

+1

+1

I'm having the same warning.. Are there any news?

any fix to this?

and? Any fix?

For me too.

Same here!

+1 middleman 4.3.4

+1
GIT   remote: https://github.com/plasticine/middleman-imageoptim   revision: fe3dac51a3a1180acebc38c0223e92e3e981380e   branch: master   specs:     middleman-imageoptim (0.3.0)       image_optim (~> 0.25.0)       image_optim_pack (~> 0.2.1)       middleman-cli       middleman-core (>= 3.1)
    middleman (4.3.5)       coffee-script (~> 2.2)       haml (>= 4.0.5)       kramdown (~> 1.2)       middleman-cli (= 4.3.5)       middleman-core (= 4.3.5)
    image_optim (0.25.0)       exifr (~> 1.2, >= 1.2.2)       fspath (~> 3.0)       image_size (~> 1.5)       in_threads (~> 1.3)       progress (~> 3.0, >= 3.0.1)     image_optim_pack (0.2.3)       fspath (>= 2.1, < 4)       image_optim (~> 0.19)

I've created PR #62 to silence this warning and bump the image_optim dependencies in the process.
A few people to test would be great:
gem 'middleman-imageoptim', :git => 'https://github.com/alexsanderson/middleman-imageoptim', :branch => 'master'


Update primary field argument passed to ElectrostaticSphere

Use an array with 3 elements instead of a float as the primary_field argument for the ElectrostaticSphere. Geoana errors out if a float is passed instead.

Thanks Santi! It looks like this now runs, but the results for the electric field and the current density look off
e.g. currently on the site
https://em.geosci.xyz/content/maxwell2_static/fields_from_grounded_sources_dcr/electrostatic_sphere.html#electric-field

and now

This might be a bug down in geoana. The potentials look correct though

I was hoping this would be obvious, but it will take some more time to figure out where things went sideways here. What do you think of pinning to an older version of geoana for now so that we can get #577 in, @santisoler?

Sure! Let's pin it. We can solve this later on.


S7 sometimes no start

I have 3 instances. 2 for 1st generation without any issue. The 3rd is an roborock s7. If I push data point "start" most times it works, but sometime it do not work with following debug log. If I press it again after 10 seconds it works again. Quite strange I think.
Screenshots & Logfiles
If applicable, add screenshots and logfiles to help explain your problem.
2021-08-13 10:35:30.687 - info: mihome-vacuum.2 (22163) starting. Version 3.2.2 in /opt/iobroker/node_modules/iobroker.mihome-vacuum, node: v14.16.1, js-controller: 3.3.15
2021-08-13 10:35:30.794 - info: mihome-vacuum.2 (22163) IOT enabled, create state
2021-08-13 10:35:31.099 - info: mihome-vacuum.2 (22163) select standard vacuum protocol....
2021-08-13 10:35:31.387 - info: mihome-vacuum.2 (22163) Map not selected delete states...
2021-08-13 10:35:31.611 - info: mihome-vacuum.2 (22163) settest next timer: not available
2021-08-13 10:35:31.648 - info: mihome-vacuum.2 (22163) change states from State control.fan_power
2021-08-13 10:35:31.693 - info: mihome-vacuum.2 (22163) create states for water box
2021-08-13 10:35:31.695 - info: mihome-vacuum.2 (22163) create states for water box mode
2021-08-13 10:35:31.696 - info: mihome-vacuum.2 (22163) create states for mop
2021-08-13 10:35:31.740 - warn: mihome-vacuum.2 (22163)
Versions:

Adapter version: 3.2.2
JS-Controller version: 3.3.15
Node version: 14.6.1
vacuum cleaner: roborock s7

Hopefully I give all information you need. If not please let me know what you need. Thanks!

thats NOT  a DEBUG log. please post a Debug log When it dont work.

I'm facing the same issue - but it takes a while to occur.
I started debug-log on 2021-09-15 08:51:30 and the adapter crashed on 2021-09-16 03:31 (as you can see).
Debug log says nothing special - only that it seem to loose connection:
2021-09-16 03:30:45.403  - debug: mihome-vacuum.0 (30278) Message= {"id":4833,"method":"get_sound_volume"}
2021-09-16 03:30:45.604  - debug: mihome-vacuum.0 (30278) MIIO RECIVE: {"id":4833,"result":[90],"exe_time":101}
2021-09-16 03:30:45.604  - debug: mihome-vacuum.0 (30278) Message= {"id":4834,"method":"get_carpet_mode"}
2021-09-16 03:30:45.907  - debug: mihome-vacuum.0 (30278) MIIO RECIVE: {"id":4834,"result":[{"enable":1,"current_integral":450,"current_high":500,"current_low":400,"stall_time":10}],"exe_time":101}
2021-09-16 03:31:05.908  - debug: mihome-vacuum.0 (30278) get params for stock Vacuum
2021-09-16 03:31:05.908  - debug: mihome-vacuum.0 (30278) Message= {"id":4835,"method":"get_status"}
2021-09-16 03:31:06.039  - debug: mihome-vacuum.0 (30278) MIIO RECIVE: {"id":4835,"result":[{"msg_ver":2,"msg_seq":3391,"state":8,"battery":100,"clean_time":5470,"clean_area":79137500,"error_code":0,"map_present":1,"in_cleaning":0,"in_returning":0,"in_fresh_state":1,"lab_status":1,"water_box_status":1,"fan_power":102,"dnd_enabled":1,"map_status":3,"is_locating":0,"lock_status":0,"water_box_mode":203,"water_box_carriage_status":1,"mop_forbidden_enable":1,"adbumper_status":[0,0,0],"water_shortage_status":0,"dock_type":1,"dust_collection_status":0,"auto_dust_collection":1,"mop_mode":300,"debug_mode":0}],"exe_time":100}
2021-09-16 03:31:06.039  - debug: mihome-vacuum.0 (30278) setGetStatus {"msg_ver":2,"msg_seq":3391,"state":8,"battery":100,"clean_time":5470,"clean_area":79137500,"error_code":0,"map_present":true,"in_cleaning":false,"in_returning":0,"in_fresh_state":1,"lab_status":1,"water_box_status":1,"fan_power":102,"dnd_enabled":true,"map_status":3,"is_locating":0,"lock_status":0,"water_box_mode":203,"water_box_carriage_status":1,"mop_forbidden_enable":1,"adbumper_status":[0,0,0],"water_shortage_status":0,"dock_type":1,"dust_collection_status":0,"auto_dust_collection":1,"mop_mode":300,"debug_mode":0,"error_text":"No error"}
2021-09-16 03:31:06.040  - debug: mihome-vacuum.0 (30278) Message= {"id":4836,"method":"get_network_info"}
2021-09-16 03:31:06.240  - debug: mihome-vacuum.0 (30278) MIIO RECIVE: {"id":4836,"result":{"ssid":"WLAN_NAME","ip":"192.168.1xx.xxx","mac":"b0:4a:39:xx:xx:xx","bssid":"3c:37:xx:xx:xx:xx","rssi":0},"exe_time":101}
2021-09-16 03:31:06.241  - debug: mihome-vacuum.0 (30278) Message= {"id":4837,"method":"get_sound_volume"}
2021-09-16 03:31:06.543  - debug: mihome-vacuum.0 (30278) MIIO RECIVE: {"id":4837,"result":[90],"exe_time":101}
2021-09-16 03:31:06.543  - debug: mihome-vacuum.0 (30278) Message= {"id":4838,"method":"get_carpet_mode"}
2021-09-16 03:31:06.845  - debug: mihome-vacuum.0 (30278) MIIO RECIVE: {"id":4838,"result":[{"enable":1,"current_integral":450,"current_high":500,"current_low":400,"stall_time":10}],"exe_time":101}
2021-09-16 03:31:26.846  - debug: mihome-vacuum.0 (30278) get params for stock Vacuum
2021-09-16 03:31:26.847  - debug: mihome-vacuum.0 (30278) Message= {"id":4839,"method":"get_status"}
2021-09-16 03:31:28.849  - debug: mihome-vacuum.0 (30278) MIIO Receive Timeout for "get_status" <<<
2021-09-16 03:31:28.850  - debug: mihome-vacuum.0 (30278) ERROR at setGetStatus: MESSAGE TIMEOUT
2021-09-16 03:31:28.850  - debug: mihome-vacuum.0 (30278) Message= {"id":4840,"method":"get_network_info"}
2021-09-16 03:31:30.850  - debug: mihome-vacuum.0 (30278) MIIO Receive Timeout for "get_network_info" <<<
2021-09-16 03:31:30.851  - debug: mihome-vacuum.0 (30278) Error at getSetNetwork: MESSAGE TIMEOUT
2021-09-16 03:31:30.851  - debug: mihome-vacuum.0 (30278) Message= {"id":4841,"method":"get_sound_volume"}
2021-09-16 03:31:32.851  - debug: mihome-vacuum.0 (30278) MIIO Receive Timeout for "get_sound_volume" <<<
2021-09-16 03:31:32.852  - debug: mihome-vacuum.0 (30278) ERROR at setGetSoundVolume: MESSAGE TIMEOUT
2021-09-16 03:31:32.852  - debug: mihome-vacuum.0 (30278) Message= {"id":4842,"method":"get_carpet_mode"}
2021-09-16 03:31:34.854  - debug: mihome-vacuum.0 (30278) MIIO Receive Timeout for "get_carpet_mode" <<<
2021-09-16 03:31:54.855  - debug: mihome-vacuum.0 (30278) get params for stock Vacuum
2021-09-16 03:31:54.856  - debug: mihome-vacuum.0 (30278) Message= {"id":4843,"method":"get_status"}
2021-09-16 03:31:56.858  - debug: mihome-vacuum.0 (30278) MIIO Receive Timeout for "get_status" <<<
2021-09-16 03:31:56.858  - debug: mihome-vacuum.0 (30278) ERROR at setGetStatus: MESSAGE TIMEOUT
2021-09-16 03:31:56.858  - debug: mihome-vacuum.0 (30278) Message= {"id":4844,"method":"get_network_info"}
2021-09-16 03:31:58.860  - debug: mihome-vacuum.0 (30278) MIIO Receive Timeout for "get_network_info" <<<
2021-09-16 03:31:58.860  - debug: mihome-vacuum.0 (30278) Error at getSetNetwork: MESSAGE TIMEOUT
2021-09-16 03:31:58.861  - debug: mihome-vacuum.0 (30278) Message= {"id":4845,"method":"get_sound_volume"}
That the connection is lost becomes obvisiously when hitting the start button:
2021-09-16 07:15:57.637  - debug: mihome-vacuum.0 (30278) Message= {"id":7097,"method":"get_carpet_mode"}
2021-09-16 07:15:59.638  - debug: mihome-vacuum.0 (30278) MIIO Receive Timeout for "get_carpet_mode" <<<
2021-09-16 07:16:05.525  - debug: mihome-vacuum.0 (30278) command: start parent: control
2021-09-16 07:16:05.526  - debug: mihome-vacuum.0 (30278) sendTo "startVacuuming" to system.adapter.mihome-vacuum.0 from system.adapter.mihome-vacuum.0
2021-09-16 07:16:05.527  - debug: mihome-vacuum.0 (30278) start Cleaning: 5 MObj: {}
2021-09-16 07:16:05.527  - info: mihome-vacuum.0 (30278) trigger cleaning all
2021-09-16 07:16:05.528  - debug: mihome-vacuum.0 (30278) Message= {"id":7098,"method":"app_start"}
2021-09-16 07:16:05.531  - debug: mihome-vacuum.0 (30278) We are in onMessage:{"command":"startVacuuming","message":null,"from":"system.adapter.mihome-vacuum.0","_id":54404041}
2021-09-16 07:16:05.531  - debug: mihome-vacuum.0 (30278) Message= {"id":7099,"method":"app_start"}
2021-09-16 07:16:07.529  - debug: mihome-vacuum.0 (30278) MIIO Receive Timeout for "app_start" <<<
2021-09-16 07:16:07.529  - warn: mihome-vacuum.0 (30278) Cant send command please try again "start"
2021-09-16 07:16:07.532  - debug: mihome-vacuum.0 (30278) MIIO Receive Timeout for "app_start" <<<
2021-09-16 07:16:07.533  - error: mihome-vacuum.0 (30278) Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch().
2021-09-16 07:16:07.533  - error: mihome-vacuum.0 (30278) unhandled promise rejection: undefined
2021-09-16 07:16:07.533  - error: mihome-vacuum.0 (30278) undefined
2021-09-16 07:16:07.631  - error: mihome-vacuum.0 (30278) Socket Close
2021-09-16 07:16:07.631  - info: mihome-vacuum.0 (30278) terminating
2021-09-16 07:16:07.631  - debug: mihome-vacuum.0 (30278) Plugin sentry destroyed
2021-09-16 07:16:07.631  - warn: mihome-vacuum.0 (30278) Terminated (UNCAUGHT_EXCEPTION): Without reason
2021-09-16 07:16:08.335  - error: host.ganymed instance system.adapter.mihome-vacuum.0 terminated with code 6 (UNCAUGHT_EXCEPTION)
2021-09-16 07:16:08.335  - info: host.ganymed Restart adapter system.adapter.mihome-vacuum.0 because enabled
After that the adapter restarts automatically - reinitializes and gets back to work (at least for a while).
Hope the log helps!

Kann ich die von github installieren? Im latest hab ich noch keine 3.3.0

@MeisterTR

please test 3.3.0

just installed and givin' it a try for a couple of days...

@MeisterTR
Looks good to me. Robot just started fine without restart of adapter. So for me it seems to be solved. Going over to long term test now.


[Chore] Setup Unit testing

What happened 👀
Closes https://github.com/tyrro/react-survey/issues/6
Jest and React Testing Library are setup as unit testing utilities
Insight 📝
N/A
Proof Of Work 📹

The same small question I added on the previous PR, any plan to add the code coverage report? If there's a plan maybe you'll want to merge the report from UI and unit test, you can take a look at the way we did on the Varun project 😄

done at https://github.com/tyrro/react-survey/pull/48/commits/453739d04c1070363c45776e745bf5c26a11668b 👍


filepath.SplitList doesn't quite do what I wnat

Ah sorry!

Nah not your fault. I naively assumed it would be a drop in replacement but it does some filepath stuff returning me "." for a empty string.


[BUG] DeletionListener not triggering consistently

Description
It seems that DeletionListener doesn't always trigger for the explicit deletion scenario (through the Delete(key) function). While it seems like a bug, it could be me not using the feature correctly.
To Reproduce
Steps to reproduce the behavior:
Using this working test as a base, simply modify the "size" to 100 (or other values), like so:
func TestCache_Delete_Modified(t *testing.T) {
	size := 100
	var mutex sync.Mutex
	m := make(map[otter.DeletionCause]int)
	c, err := otter.MustBuilder[int, int](size).
		InitialCapacity(size).
		WithTTL(time.Hour).
		DeletionListener(func(key int, value int, cause otter.DeletionCause) {
			mutex.Lock()
			m[cause]++
			mutex.Unlock()
		}).
		Build()
	if err != nil {
		t.Fatalf("can not create builder: %v", err)
	}

	for i := 0; i < size; i++ {
		c.Set(i, i)
	}

	for i := 0; i < size; i++ {
		if !c.Has(i) {
			t.Fatalf("key should exists: %d", i)
		}
	}

	for i := 0; i < size; i++ {
		c.Delete(i)
	}

	for i := 0; i < size; i++ {
		if c.Has(i) {
			t.Fatalf("key should not exists: %d", i)
		}
	}

	time.Sleep(time.Second)

	mutex.Lock()
	defer mutex.Unlock()
	if len(m) != 1 || m[otter.Explicit] != size {
		t.Fatalf("cache was supposed to delete %d, but deleted %d entries", size, m[otter.Explicit])
	}
}

Which results in:
=== RUN   TestCache_Delete_Modified
    cache_test.go:198: cache was supposed to delete 100, but deleted 92 entries
--- FAIL: TestCache_Delete_Modified (5.01s)

Tracking 92 of the 100 entries
Expected behavior
DeletionListener should be called for all explicitly deleted entries
Environment
otter 1.2.1
go 1.22
Additional context

Increasing the sleep time even to a large value like 30s doesn't seem to help.
Curiously, the test works for 256, 512, 1024, 2048 and so on, so I'm guessing it's related to the operation of some buffer?

Hi. Yeah, this is because of the buffer that is used to apply operations to eviction and expiration policies (some link).
In real life, this shouldn't affect you very much. Maybe, except for various scenarios in which the deletionListener is used to close connections and an instant reaction to this is very important to you.
P.S. Too many issues are created for this behavior... I will have to try to abandon prefetching in favor of processing one entry at a time.

Hello, author.
May I ask, if I'd like to use DeletionListener as a delay queue to call a delay function.
Is there any risk if i change the buffer size to 1?
I think change to 1 will disable the buffer, so it will reaction instantly.
Maybe we can expose the buffer size?

Hello, author.
May I ask, if I'd like to use DeletionListener as a delay queue to call a delay function.
Is there any risk if i change the buffer size to 1?
I think change to 1 will disable the buffer, so it will reaction instantly.
Maybe we can expose the buffer size, so we can change it outside?

@maypok86 I have the same question, hope you can reply soon.

Hi, I tried to fix it and it seems to have worked. I will try to check again tomorrow and merge the pull request, but so far without publishing a new release.

Maybe we can expose the buffer size, so we can change it outside?

I don't think adding implementation specifics to the API is a good idea.

@maypok86
Hello author, I tried your code, but found an issue.
I use this code below. It sets a key with ttl=12, and open a loop to get the key every 3 seconds.
func TestGetExpired(t *testing.T) {
	c, err := MustBuilder[string, string](1000000).
		CollectStats().
		DeletionListener(func(key string, value string, cause DeletionCause) {
			fmt.Println(cause)
		}).
		WithVariableTTL().
		Build()
	if err != nil {
		t.Fatal(err)
	}
	c.Set("test1", "123456", time.Duration(12)*time.Second)
	for i := 0; i < 5; i++ {
		c.Get("test1")
		time.Sleep(3 * time.Second)
	}
	select {}
}

Which result in:
=== RUN   TestGetExpired
Explicit

I think it should output expired instead of explicit.
And after I read your code, it push a delete to the writebuffer in get.

I'd like to contribute, maybe i can add a new case here, to fix this issue?

Oh, I totally forgot about this if. Yes, this should be enough, but it's worth adding n.Die() to stop various possibilities for race conditions. aka
if n.HasExpired() {
    n.Die()
    c.writeBuffer.Push(newExpireTask(n))
    c.stats.IncMisses()
    return nil, false
}

ok

I open a PR.
https://github.com/maypok86/otter/pull/104

I will publish a release with the latest changes in a few hours.

The new release has been published


Driver/ats9360

This should replace #293
Changes proposed in this pull request:

make Alazar driver work for ATS9360 as well as ATS9870

add acquisition controllers for arraylike alazar parameter for use with ATS9360 (but easily adaptable to other alazar cards)

add example notebook

add aux.io functionality to alazar driver

[x] I've written it such that the get_sample_rate function doesn't work for the ATS9360 and i'm not sure what the best way to implement this such that it works is (the issue is that the byte to value dict  of the sampling rate should be different depending on the value of another parameter which sets how it works out the sample rate, not sure the 'nice' way to write this')

[x] how should different acq controllers be organised? all in one big file seems messy but also the way I've done it seems overkill, maybe just have two files in the master branch: the existing one by @damazter and one which works for the ATS9360 and returns an array

[x] currently the two acq controllers rec_samp and rec_buf don't work, they are designed to return a 2d dataset in one acquisition, I didnt try super hard to get them to work but also not sure why they don't, @alexcjohnson any ideas?

[x] where should filters live considering that they are a quite general tool, also how do I deal with the whole import scipy thing...

@alexcjohnson @giulioungaretti @WilliamHPNielsen

@nataliejpg
A quick response to your message above
It would be great if this Alazar driver gets extended to work with other alazar cards as well!
I  have not taken the time yet to read all the changes proposed here. (I would like to check the changes to ATS.py)

I've written it such that the get_sample_rate function doesn't work for the ATS9360 and i'm not sure what the best way to implement this such that it works is (the issue is that the byte to value dict of the sampling rate should be different depending on the value of another parameter which sets how it works out the sample rate, not sure the 'nice' way to write this')

I am not sure that I know what you mean, but with the ATS 9870 you had to take the decimation into account which is done in the get_sample_rate function in ats.py does this solve your problem or would a similar approach solve your problem?

how should different acq controllers be organised? all in one big file seems messy but also the way I've done it seems overkill, maybe just have two files in the master branch: the existing one by @damazter and one which works for the ATS9360 and returns an array

and

currently the two acq controllers rec_samp and rec_buf don't work, they are designed to return a 2d dataset in one acquisition, I didnt try super hard to get them to work but also not sure why they don't, @alexcjohnson any ideas?

My original idea was to not put any acquisition controllers in qcodes as it might by to personal from person to person. But I had to put at least one their to sort of explain the idea of the driver (making sure that ATS_WORKS.ipnb works)
The reason that the example does not return an array is that qcodes loops do not play nicely with these kinds of array parameters yet, so I decided to have an example which is not hacky in any way, but would not be really useful as a practical acquisition_contoller. (It only saves a single point per acquisition which is not that useful).

Hi @damazter, thanks for your comments.
The reason the get_sample_rate function doesn't work for me is that when setting the sample rate the byte to value dict you use in the ATS9870 driver works fine if you aren't using a 10MHz reference signal to generate a sample rate from (which isn't an option for the ATS9870). In the 9360 however if you use the reference signal then you can set any sample rate you want and don't use a byte to value dictionary so unfortunately there can be two ways to map the same sample rate depending on if you use a reference signal or not. Effectively if you use a reference signal you don't want to use a byte to value dict you just send the int rate to the alazar. So no, what you suggest won't solve it but if I sit down and think a bit I can probably come up with a nice way to write it I just didn't have time yet so I left it as a to do.
In terms of acquisition controllers the arraylike parameter works fine for us so far including int he loop (check out the Rohde Schwarz VNA driver whcih has been around a while by now). There are a few plotting bugs but the acquisition part itself is fine. Hence I think it's definitely worth having at least one acq controller which gets an array in qcodes for people to build off. Maybe just the 'basic_controller' I've written gives the idea and together with yours lets people write their own. Let me know what you think when you've had a chance to look. The ones I was having trouble with were the ones which are meant to return a data structure which is (samples * records) or (records * buffers), the ones which return an array of samples or an array of records are both there :)

@nataliejpg

In terms of acquisition controllers the arraylike parameter works fine for us so far including int he loop (check out the Rohde Schwarz VNA driver whcih has been around a while by now). [...] The ones I was having trouble with were the ones which are meant to return a data structure which is (samples * records) or (records * buffers), the ones which return an array of samples or an array of records are both there :)

I'll check it out!

Hence I think it's definitely worth having at least one acq controller which gets an array in qcodes for people to build off. Maybe just the 'basic_controller' I've written gives the idea and together with yours lets people write their own.

Yeah exactly, that one dataextractor example can definitely be improved upon. But having just basic_acquisition_controller as you suggested will prevent an entire mess of acquisition controllers which are all similar but not the same.

The reason the get_sample_rate function doesn't work for me is that when [...] Effectively if you use a reference signal you don't want to use a byte to value dict you just send the int rate to the alazar.

I will also think about that, maybe the ats9870 has the same option and the same problem. I'd have to look that up

@damazter
acq conroller-wise:
I think we should have at least 1 which just works out the box for ATS9360 and one for ATS9870 and one (could be one of these) which has an arraylike param. I think we're agreeing I just want to be sure :P
get_sample_rate-wise:
I did look it up and unless I'm drastically misremembering the ATS9870 doesn't have that function and hence doesn't have the problem.

@nataliejpg
get_sample_rate-wise:
That solves a problem for me ;)
acq conroller-wise:

I think we're agreeing I just want to be sure :P
I think so, I would just like it to be a single one, working with the 9870, 9360 and return an array like reuslt which works with the loop and dataset, such that the ATS_WORKS examplenotebook still works

@damazter
The only problem with having just one acq controller is that from looking at yours I think the samples are arranged differently in the buffer. Not sure if that's an optional setting which differs between our setups or an ATS9870 feature but it would definitely be nice if people could just run the example notebook on their ATS card without having to edit the acq controller to do a basic test. Would be nicer not to have separate example notebooks but it seemed simpler, I guess I could check out trying to merge them and get yours to work for me (although for the data arranging reasons mentioned above I'd have to use one of my acq controllers rather than yours...).
I'll leave it all up there for the moment and do a clean up and update when we have some feedback from one of the core devs and probably delete quite a bit of acq controller stuff at least. :)

I've left only the basic_controller which works with the ATS9360 and returns a trace of unprocessed data (averaged by records and buffers) and deleted the other acq controllers, maybe worth renaming the acq controllers to make it clear what each of them does, any strong feelings @damazter
I've done a temporary fix so that get_sample now works for the ATS9360 at the expense of being able to set the clock source to be anything other than 10MHz reference signal. If anyone has an elegant way to fix this I'd be very interested, otherwise it's enough to work for our setup now and shouldn't break anything for the 9870 guys.

realised all other questions were specific to the acq controllers I've deleted...

Notes to self:
Should probably revert https://github.com/QCoDeS/Qcodes/pull/400/commits/e8190ec03f5e98b824fe09d4d4d4e1ecbfca33b1 that seems to be unrelated and should if needed go in on a different branch

This should be probably closed ?

Yes closed as replaced by #491


Enable restricting storage accounts to a virtual network even though App Service Plan is in Standard

If App Service Plan is eligible to Regional VNet Integration, it appears that we can restrict storage accounts to a VNet. But, the document says the restriction only work in Premium plans. I think it's more appropriate if the sentence covers Standard plans which support Regional VNet Integration.
https://docs.microsoft.com/en-us/azure/azure-functions/functions-networking-options#restrict-your-storage-account-to-a-virtual-network
"This feature currently only works with Windows Premium plans. "
Also, technically Premium plans consists of 2 kinds of plans (Premium plan or Dedicated plan's Premium plan). It may be simple that it says "This feature currently only works with Windows plans which support Regional VNet Integration."

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bc0b34e2-0eb8-fe89-300c-67e6c212ed31
Version Independent ID: 5d01d8ce-f835-c88c-801c-1d569f24fd9f
Content: Azure Functions networking options
Content Source: articles/azure-functions/functions-networking-options.md
Service: azure-functions
GitHub Login: @cachai2
Microsoft Alias: cachai

Hello @YusukeTobo , thanks for taking time to share your valuable feedback! We are taking a look into this and will get back to you soon.

@YusukeTobo , Please be informed that  this feature works for Windows Premium plans and Standard. It does not work for Consumption. I will update the same in the Docs.
Let me know if this clarifies your doubt.

@YusukeTobo, just checking in to see if the above response answers your query.

@HarshitaSingh-MSFT, sorry for being late because of my days off. I agree on the above commit. It works in Standard and Premium plans unless the sku in Flex stamp. Also, the consumption plans are not supported in this scenario because it doesn't have vnet integration.


Grinder should use the same parameter value from upstream job

Describe the bug
Trying to run grinder from test job, but grinder trying to download "nightly" jdk from API VS original upstream job is copying jdk from build artifact
This could give the false test result if jdk is not the one from last build.
Since we have some problems to download the latest ones from jdk19/20 therefor grinder failed,
But for other jdk versions, grinder wont failed to download jdk so the test result might not be accurate.
To Reproduce
original upstream job https://ci.adoptopenjdk.net/job/Test_openjdk20_hs_sanity.functional_x86-64_alpine-linux/11/consoleFull
click link from console to run grinder https://ci.adoptopenjdk.net/job/Grinder/5564/console
Expected behavior
grinder pre-fill value

because in original upstream job it does not set SDK_RESOURCE, then grinder fall back to the first item from dropdownlist "nightly"
Screenshots

both jobs set correct upstream job information

It is upstream in the template. https://github.com/adoptium/aqa-tests/blob/master/buildenv/jenkins/testJobTemplate#L46.
It is the way how the Test_Job_Auto_Gen is configured. @llxia any concern if we set sdk_resource to be upstream by default in the job https://ci.adoptopenjdk.net/view/Test_grinder/job/Test_Job_Auto_Gen/configure?

I have changed the Test_Job_Auto_Gen SDK_RESOURCE to default to upstream.

Yes, it is ok to set SDK_RESOURCE to default to upstream in Test_Job_Auto_Gen. The correct way is to set the value in the pipeline. @sophia-guo you have already figured it out - https://github.com/adoptium/ci-jenkins-pipelines/pull/391.


42444 Updates the example app to use CDTReplicators properly.

The CDTTodoReplicator keeps a strong reference to the
CDTReplicator objects that it creates and sets itself as
their delegate. The CDTReplicators are released upon
completion or error.
The polling block has been removed.
The CDTTodoReplicator now uses the new CDTPush/PullReplication
configuration and the CDTReplicatorFactor -oneWay:error method
to build CDTReplicator objects.
Additionally, the CDTAppDelegate now keeps a persistent
CDTTodoReplicator object, which the CDTViewController uses to
fire off synchronization.
Adds a description method to CDTReplicator.

I think this is fine now. Your proposed message in 384795c looks good to me.


Use the same trigger for Py2 and Py3 PostCommit test suites.

Post-Commit Tests Status (on master branch)

Lang
SDK
Apex
Dataflow
Flink
Gearpump
Samza
Spark

Go

---
---
---
---
---
---

Java

Python

---

---
---
---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

Run Python Postcommit

R: @markflyhigh
Looks like the PR need to be submitted first for the change to take effect.

Run Seed Job

Run Python Postcommit

The Seed job is what we use to "commit" configuration of tests on jenkins.  The pitfall is that it updates configurations for everyone.

LGTM.

@pabloem , thanks. The tests will probably fail, but as long Py3 suite is triggered, feel free to merge this. Thank you!

should also update the trigger phrase in https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md

FYI - @RobbeSneyders , @Juta


Download goal: No renditions are included, but AEM publish does not trigger update_asset workflow to create them

https://github.com/wcm-io/io.wcm.maven.aem-global-parent/blob/4b08ec3a5fbcb047f24381f288575cdcc3dea7ac/pom.xml#L418
This makes sure to not include renditions in the SCM, which is of course very nice to have. When installing the package on the author environment, the renditions are automatically created because of the update_asset workflow launcher.
However, this launcher is only active on author (runmodes=author), due to which the assets will not have any renditions on the publish environment.
When activating assets from the author environment to the publish, the asset including all metadata & renditions will be activated to the publish environment.
In the end, it causes the assets to be available in the author environment with renditions, and available without renditions in the publish.
@stefanseifert  How are you handling this "feature" in your projects? How do you make the renditions available in your publish environment as well?

Any thoughts on this?

the mentioned line excludes only some of the auto-generated renditions, namly all the cq5dam.thumbnail in the name. this should by only those generated for usage in the author UI itself, usually those should not be used on live website? if you are actively using those on publish, you should probably remove that exclude and overwrite the "excludeFiles" section.


Make sure the sourcemaps build option is respected, even when providing a custom terser config

Closes https://github.com/ember-cli/ember-cli-terser/issues/306.
Closes https://github.com/ember-cli/ember-cli-terser/pull/308.

Made https://github.com/ember-cli/ember-cli-terser/pull/314 as a prerequisite, so we can merge stuff like this with more confidence.


Quantization failed for the yayi2 30b model

Yayi 30b k,v layer has [input_dims=7168, out_dims=112], so it failed to quantize due to error  all dimensions should be divisible by 32 for now.  FYI, here is the implementation of the yayi2 30b model's k,v layer -> https://huggingface.co/wenge-research/yayi2-30b/blob/main/modeling_yayi.py#L180

Yes that's a known limitation of our quantization at the moment. Will mark this as an ehnancment.
In the meantime one thing you could do as a workaround so you aren't blocked by this is to concatenate the K and V projections and do them as a single matmul (112 * 2 / 32 = 7).

Yes that's a known limitation of our quantization at the moment. Will mark this as an ehnancment.
In the meantime one thing you could do as a workaround so you aren't blocked by this is to concatenate the K and V projections and do them as a single matmul (112 * 2 / 32 = 7).

Hi Awni, thanks for the reply. Would you be able to give a bit more detail about the workaround? By the way, happy new year! :)

Yes so there are two projections, one for the keys and one for the values.
k = x @ Wk.T
v = x @ Wv.T

Instead you could do something like:
k, v = mx.split(x @ mx.concatenate([Wk, Wv], axis=0).T, 2)

Now you can pre compute the concatenated matrix and quantize it since its dimensions should be divisible by 32.

Thanks @awni , this workaround works. I have added it to the yayi2 example. :)

Thanks @awni , this workaround works. I have added it to the yayi2 example. :)

@mzbac not sure you saw, in 0.0.10 this should be fixed. We can add yayi2 to mlx-lm now 😃

@mzbac not sure you saw, in 0.0.10 this should be fixed. We can add yayi2 to mlx-lm now 😃

Nice, I will take a look. Hopefully, it just needs to be mapped to the llama arch and it will work.

Oh that would be amazing.

Please close this now as MLX supports non-32 dims quantization and I have tested it on the yayi2 model, and it works as expected.


Only fixed params of logistic model are used for calling the core method after fitting

Here the fixed parameters are taken although a model has been fitted and is then raising an error...
https://github.com/kusterlab/curve_curator/blob/b87b2126f27b5223ad395cc3720b11af5f04f139/curve_curator/models.py#L864
Example:
# data
line_x = np.linspace(-10.3, -4.3, 1000)
x = np.log10([0.1, 1.0, 10.0, 30.0, 100.0, 300.0, 1000.0, 3000.0, 10000.0, 30000.0]) - 9
y = pd.Series([1, 1, 4, 6, 7, 16, 55, 105, 147, 160])

# Define the logistic Model
logistic_model = LogisticModel()

# Fit the unrestricted model with ordinary least squares (ols)
logistic_model.find_best_guess_ols(x, y)
logistic_model.fit_ols(x,y)
logistic_model(line_x)

Error message:
TypeError: LogisticModel.core() missing 4 required positional arguments: 'pec50', 'slope', 'front', and 'back'

https://github.com/kusterlab/curve_curator/blob/b87b2126f27b5223ad395cc3720b11af5f04f139/curve_curator/models.py#L302

Quick solution is to actively pass the fitted params after fitting.
logistic_model(line_x, **logistic_model.get_all_parameters())

But I agree. It would be a nicer API call.

Idea: follow scikit-learn logic of fit and predict methods. Easy solution is a high level predict function that works after all parameters are available
# Define the logistic Model
logistic_model = LogisticModel()

# Fit the unrestricted model with ordinary least squares (ols)
logistic_model.find_best_guess_ols(x, y)
logistic_model.fit_ols(x, y)
logistic_model.predict(line_x)


[Improvement] Do not hardcode the "default" machine name, use DOCKER_MACHINE_NAME and default as fallback

First, thanks for this great software.
I would like to make a proposition of improvement.
Actually the machine name is hardcoded to "default".
There has been a patch (https://github.com/docker/kitematic/pull/992), but it don't work anymore (i tried it without any success) and found no more reference of it in the actual source code.
Actually the default machine name seems to be hardcoded:
https://github.com/docker/kitematic/blob/master/src/utils/DockerMachineUtil.js#L24
This is problematic because we don't use the name default in virtualbox (because we have others VM managing LXD, some Debians, etc.), and default mean nothing in the list of VM, it is not a clean name for developpers to easily spot the VM in charge of docker toolbox and can be prone error when cleaning VM. (actually we name our VM DockerToolbox)
So a solution of forcing naming docker VM "default" is not possible for us.
First question please: is it a way to change this default name ?
Second question: If not can an improvement be made on this ?
An easy solution would be to look for the environement variable DOCKER_MACHINE_NAME and if it exist take this one (this environment variable is used by docker-machine itself to determine the active machine). If this environement variable don't exist or is not set use default. This condition could be implemented here:
https://github.com/docker/kitematic/blob/master/src/utils/DockerMachineUtil.js#L24
Actually we can no more use Kinematic and switched to Portainer which handle well this. We would really like to able to use it again.
Thanks!

Up please, someone here ? Could the patch be implemented for next release ?

The behavior would need to first change in machine - default has always been the go-to for deployments and thus somewhat "reserved" for the Boot2docker provisioned VM.
What happens when the env var is set, but the machine doesn't exist?

Thanks @FrenchBen for your post.
Machinge get default machine name as default name, but it can be used with other names (docker-machine [command] [machine name]). Machine work great with other names.
Problem with Kinematic is it won't work at all with other names. default name is hardcoded on it and there is no way to indicate it to use another machine name. When Kinematic start if it don't find the default name machine, it even attempt to create it... : (
To answer to your question, i think if env var is set and machine is not found, it can do any of the following (by order of best preference):

send an error message (i don't think it is the purpose of Kinematic to create the VM... so for me it is the best, and it give the developper the possibility to create manually the machine)
create the machine with this name (like it does actually for default)
create a default machine and use it

But the real problem in this issue is Kinematic cannot be used with machine with other name than default, and it limit its usablility. Docker Toolbox do not force to use default. You can create any new VM name you want with docker-machine.
And the patch is so easy to implement, and will bring so more to Kinematic, that actually i don't see any raison to not implement it. We don't talk about big code change here.

Here's part of the complexity, say you're doing a lot of docker-machine things and to make your life easier, you set the ENV var to be machine.
In order for the ENV var to be "seen" by Kitematic, Kitematic would need to be started from the CLI (not very convenient for most users)
This VM isn't a boot2docker image, but it's what you use for X project.
You then decide to open Kitematic from CLI, as you've done before, with the changes you've mentioned above.
It wont connect.  You've now blown away your X project machine and may be mad at Kitematic (even though you told it to do it)
TL;DR - it's easier to tell people that default is the Kitematic preferred VM, and the one that should always be used for boot2docker things, rather than providing complex options as to what VM/Driver combo should be used.

I undestand better thanks for the explaination.
I was not aware that:

Kinematic cannot read Windows system/user environment variables
Kinematic has the job to manage the VM (create, etc.) (i thought Kinematic was only to view container in a docker instance)
Kinematic has a "delete VM" policie if it don't manage to connect to VM

I am just not agree on you last sentence "default is the Kitematic preferred VM, and the one that should always be used for boot2docker things". It is more right to say Kinematic actually can only be used with default, this is not a preferred VM as there is no choice (and this is purpose of this issue which could be resolved i hope). For boot2docker, it has never been mandatory to use default, using another name works very great, just Kinematic don't handle it. In fact developpers could even have several boot2docker VM, this is the puprose of docker-machine, be able to create/manage multipe machine and so multiple boot2docker. Forcing it to only one VM called default is reducting. For instance, what about swarm in this case ? Or testing local cluster environment, etc...
I think if Kinematic cannot read windows environment the old patch could be reincluded (kinematic read a config file). To prevent machine deletion, if the VM kinematic try to start is not default, it could just report "Cannot connect" and that's all. It should not have an option to destroy and recreate... It is to the user to do it so VM cannot be deleted accidentaly. And we can consider that an user using not the default name is more advanced than a user staying on default so he know how to handle the problem.
What do you think of this approach?
p.s: does your deletion exemple apply to default too ? If i create a "default" VM for some project (imagine a CentOS VM) and start Kinematic it can destroy it ? If yes, deletion problem is not particularly tied to not use the default name for the VM in this case but more an other global issue i think.

@Yivan there is a "choice", as the software is OSS and thus can be modified by you to do what you want.
The origin of the tools were as such and thus have remained. I understand your need, but also see the simplicity of keeping a Docker tool as the default target.
Even though it can now do much more, there's quite a lot that's ok to have as the default.
Kitematic was suppose to prevent the hard-entry into the Docker CLI by simplifying the creation and connection to the VM. If the default was taken by something else, than Kitematic would not be able to connect to it. It will then give you the option to destroy it in order to start fresh, but will not do it without you clicking the button.

I understand your reflexion and need to keep it simple. Thanks for taking time talking about this issue.
I hope somedays we could use kinematic with different host/VM name (like portainer does for instance).

i would vote for a reopen. having many virtualbox machines up and running on my system and  a machine which is named default inbetween them is really really unhandy.  quite a bad DX
i changed the env var to docker and assumed that at least after a host restart kitematic will be able to read my new set environment variable.
as docker toolbox introduced a env variable that is named DOCKER_MACHINE_NAME i would thinkg kitenmatic should obey this naming/config.
normally people would change it and if you provide some info in the readme then people who change the name, will know the risks.
just my 2 ct


Duplicated unified/split diff buttons

On PRs, it looks like Refined GitHub adds icon buttons for unified/split diffs, but doesn't remove the existing textual buttons:

With the extension disabled, the icon buttons don't appear.

I think it should be clarified that this is visible on a single commit view, and not in the context of a PR.
Example: https://github.com/sindresorhus/refined-github/commit/75624f2acf21abf14fa75ea512cfcab42eeb6a2c

The feature is supposed to replace those buttons with our “icon-only” version.
The future of the feature itself is being discussed in https://github.com/sindresorhus/refined-github/issues/2202