Texture compression support in PIXI.loader and PIXI.texture

Im helping devs of a specific app (I mentioned it in pixi-spine) that have really big number of textures and atlases, and it affects both loading time and amount of used videomemory.
It seems that we need to load different kind of textures based on which compression is supported by device. I know that there's some simple mechanism for retina textures, I dont know if the same way can be applied to compressed textures

No, compressed textures are not yet supported by PIXI, you need to modify PIXI and code your own class to handle compressed textures.
It can be done (we've done it) but it's a good amount of work.

The loader is able to understand binary datas so you can load your compressed textures into an arraybuffer without any modification to the code
You need a class/library to parse this array. Most of the code you will need can be found inside Toji's texture tester project here https://github.com/toji/texture-tester/blob/master/js/webgl-texture-util.js
You need to modify PIXI to tag your compressed textures (we use "isCompressed" parameter on the base texture source)  in order to hack the call to "gl.texImage2D" inside WebGLRenderer.prototype.updateTexture and use glCompressedTexImage2D instead.

Please note that every compressed texture must be a power of two, and if you want to use PVR on iOS, your textures needs to be square.
If you prefer to wait for PIXI to deals with compressed texture itself, you can follow this issue : https://github.com/pixijs/pixi.js/issues/1343

@JiDW even if you can't submit a PR (and I understand why not, the changes to PIXI must be vast) - if you could still share your source then I'm sure someone would be able to extract what they need from it. Saves re-inventing the wheel and all that.

@JiDW make it as PR or patch for pixi that re-defines some functions, PLEASE PLEASE PLEASE.

The changes in PIXI aren't that big, since we use an external class and don't mind about compatibility with further version of PIXI. It's why we can't push a PR too, it's not generic enough, it only deals with the formats/options we need, and it's linked to our own 3d engine too.
To be honest, most of the code is where I linked. Most of the work is about setting a working flow to integrate compressed images inside your project. WebGL lacks some tools to do it easily, we had to create our own tool in java to make it easier.
If it can help, here is our PIXI.js and class. Just ctrf+f on "compressed" and that should do the job.
https://game.celsiusheroes.com/pixi.js
https://game.celsiusheroes.com/compressedImage.js

@JiDW yeah, it seems that your game downloads @2x dds files for windows, that's nice. And I like that game!

Yes, It depends of what kind of compressed texture you can use : http://toji.github.io/texture-tester/

Is that game in production?

Currently in a soft launch (search for Celsius Heroes on Facebook or Android). Polishing stuffs, fixing some stability issue on android and making sure the servers can hold the load.
We looked at Spine in our early stage of development but we didn't find good examples of what Snipe can do with characters facing you, it was only from the sides (and it makes sense for me).
This place may not be the best way to talk about it anyway. I planed to post soon on html5dev forum to present the game and what we learned from 1yr and a half with PIXI, Cocoon and WebGL. I was just waiting for the full release.

@JiDW I just cant wait to read that post :)

I am looking for a way to reduce the load size of the textures and found this.
If I were to change my pixi class (which already has changes anyway) and add your compressedImage.js to my project, how would be the workflow to be able to do this? Do I need to change something in the source texture? If yes, how will old browsers and gpus (we want to support really old stuff, one of our test pcs has a 7600gs) work with it?
Thank you for the help.

Well, you need to generate the compressed textures, in 3 formats (DTX, ATC and PVR) + keep the PNG source too.
"Old" browsers won't have WebGL anyway, so the main compatibility issue is on the GPU. You don't know what compressed format they can read so you need to have all possibles formats ready to download.
In compressedImage.js you will find a getCompressedFormat function that will tell you what format to use. You can use this value to load a different batch of images in your PIXI loader.
Then you need to make all of your calls to PIXI.Sprite.fromImage with "CompressedImage.getNomFichierCompresse()" (sorry for the french, company code rule... It should be getCompressedFileName in english)
Please remember that compressed textures are lighter in the memory (by a 4x factor) but heavier on the disk. It is not too bad between a PNG and a DDS (1.4x), but if you compare with a JPG, it can be 15x time heavier to download. So it's all about finding the good balance.

Oh, then this is exactly the opposite of what I was looking for, I want to reduce the download size, the memory is less of an issue right now.
Thank you for the help anyway :)

I'm moving your code here: https://github.com/pixijs/pixi-compressed-textures

@JiDW done. Please look at github.
Also, demo is working
You can try using pixi.js and pixi-compressed-textures.js from that demo in your project.
Do I have to mention in the comments that I took CompressedImage code from you? Or did you take it from other source?

Also please join gitter: https://gitter.im/pixijs/pixi.js

Cool work Ivan ! ;)
Most of the texture parsing was done by Toji ( http://toji.github.io/texture-tester/ )
We integrated it inside PIXI. We didn't release it under MIT so officially it's still copyrighted (as it is a part of Celsius Heroes). However I can tell you that we accept this use of the CompressedTexture.js file and will later release this file under MIT licence.
We won't change our code to try yours just because we do not intend to update PIXI in this specific project so it would be pointless but we for sure try it for our next project.
There are some things that I could see done in another way regarding specification compliance (for example, .atc is a fake extension that we made up to distinguish between DXT and ATC compression but ATC is supposed to use a .dds extension (dds is only a container)) but I will comment about it when I get the time to do so and I will do it in your repo.

@JiDW did you have problems with pre-multiplied alpha? We are getting very strange things in production. Something wrong with blending and other stuff

Yes indeed, you need to use textures with premultiplied alpha only.
Premultiply your PNG and then compress them.

Ok, found some problems in png and dds files , its fixed. When will you try my library and pixi3.0.9-dev?

I don't know, not before being done with our current project, so I guess in two months.

Ok, everything is working in pixi-compressed-textures, and it also supports multiple resolutions, even stuff like @.5x . We can close the issue.


chore: updated sqlalchemy docs

This PR updates the SQLAlchemy documentation to reflect the latest changes.

This flaky test are addressed in pr #2957

Going to re-open this one as part of a larger bit of refactoring.


Change in behaviour of PDHG

Description
From @samtygier-stfc on Discord
I am upgrading from 23 -> 24, and came across some changes in the way we iterate the PDHG algorithm. Previously we created the PDHG instance with a large number for max_iterations, and then had our own loop to which called next() to run each iteration
algo = PDGH(..., max_iterations=1000000, ...)
for i in range(N):
    algo.next()
    ...

In 24.0 the next() method is gone, but I can do next(algo) instead. Also max_iterations is deprecated in the constructor, with the suggestion to use Algorithm.run(iterations). If I remove max_iteration from the constructor, then next() will give StopIteration straight away. I can get around the deprecation warning by doing
algo = PDGH(...)
algo.max_iterations = 1000000
for i in range(N):
    next(algo)
    ...

So far algo.max_iteration is used, although we would like to discourage it in favour of the algo.run method with callbacks.
Also, next has been removed (it was there for compatibility with Python2) but __next__ is still there.
The callback you need would be something like
from cil.optimisation.utilities import Callback

class MaxIterationCallback(Callback)
    def __init__(self, max_iteration):
        self.max_iteration = max_iteration

    def __call__(self, algorithm):
        if algorithm.iteration >= self.max_iteration:
            raise StopIteration

algo = PDHG(...)
algo.run(100, callbacks=[MaxIterationCallback(100000)] )

We might want to add it to the default callbacks?

For now you could use:
algo = PDGH(...)
algo.max_iterations = 1000000
for i in iter(algo):
    ...

but it might stop working in a future release.
Best is:
from cil.optimisation.utilities.callbacks import Callback, ProgressCallback

class EachIterCallback(Callback):
    def __call__(self, algo):
        ...  # do something with algo

algo = PDHG(...)
algo.run(1000000, callbacks=[ProgressCallback(), EachIterCallback()])

vis. https://tomographicimaging.github.io/CIL/v24.0.0/optimisation/#callbacks

I've switched to doing the progress bar updates from a callback:
https://github.com/mantidproject/mantidimaging/pull/2184/commits/5006fd30d5ae81a89c7904d32af9af71cb175585
It would be good if there was a mention in the release notes about the change.

I've switched to doing the progress bar updates from a callback: mantidproject/mantidimaging@5006fd3

lgtm!

It would be good if there was a mention in the release notes about the change.

Unfortunately it is mentioned in the highly inaccessible CIL@master:/CHANGELOG.md but for some unknown reason @TomographicImaging/cil-developers seem to dislike keeping the more standard CIL/releases readable/helpful/neat/up-to-date/in-sync 😠


adjust image recipes

@rojkov Please review and merge.

Successfully built the images for
Build Configuration:
BB_VERSION        = "1.33.3"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "fedora-25"
TARGET_SYS        = "i586-oe-linux"
MACHINE           = "qemux86"
DISTRO            = "nodistro"
DISTRO_VERSION    = "nodistro.0"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              = "master:eeb17d64ecae7442a627824ccacca83959cb32e0"
meta-oe
meta-python
meta-multimedia   = "master:cdb428e7c49899675ee7b7a43f6cecdb5b4c2546"
meta-ros          = "bulwahn-master-v1:0e470810360ef659fa25eee8437d15c30d2414c2"
meta-intel-realsense = "master:53e93ed3f9fad6aa96ca455172e4919b666dca3b"


compile failed for tf-gpu 1.4, cuda 9, cudnn 7, vc 2017, windows 10.

I would greatly appreciate if anyone could help me out with compiling.
I followed #5600 and #13962 in compiling the wheel, however when building i got into 6 types of in total 90 errors. Respectively c2070, c2059, c2064, c2001, c1057, c2146.
My build command was:
cmake ..
It manage to configure the build so I continue with:
MSBuild /p:Configuration=Release tf_python_build_pip_package.vcxproj
The last 9999 line of the failed log is here.
tf-compile-build-error-log.txt
Warmest Regards,
Colman

AFAIK CUDA 9 and cuDNN 7 aren't supported on Windows yet.

@colmantse It looks like something is failing in the RE2 build, apparently in test code. Since we don't need that code, I've created PR #14696 to disable it. Could you please try patching it in (or making the one-line change to tensorflow/contrib/cmake/external/re2.cmake) and let us know if that fixes things?

Thank you, by doing so the error is reduced from 90 to 1. It now gives:
"D:\projects\tensorflow\tensorflow\contrib\cmake\build\tf_python_build_pip_package.vcxproj" (default target) (1) ->
"D:\projects\tensorflow\tensorflow\contrib\cmake\build\pywrap_tensorflow_internal.vcxproj" (default target) (3) ->
"D:\projects\tensorflow\tensorflow\contrib\cmake\build\tf_core_gpu_kernels.vcxproj" (default target) (133) ->
(CustomBuild target) ->
  C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.CppCommon.targets(17
1,5): error MSB6006: "cmd.exe" exited with code 1. [D:\projects\tensorflow\tensorflow\contrib\cmake\build\tf_core_gpu_k
ernels.vcxproj]

    2629 Warning(s)
    1 Error(s)

Time Elapsed 01:25:22.60

Hi, thanks for the advice. I have implemented the change and the error reduced from 90 to 1 as follow:
"D:\projects\tensorflow\tensorflow\contrib\cmake\build\tf_python_build_pip_package.vcxproj" (default target) (1) ->
"D:\projects\tensorflow\tensorflow\contrib\cmake\build\pywrap_tensorflow_internal.vcxproj" (default target) (3) ->
"D:\projects\tensorflow\tensorflow\contrib\cmake\build\tf_core_gpu_kernels.vcxproj" (default target) (133) ->
(CustomBuild target) ->
  C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.CppCommon.targets(17
1,5): error MSB6006: "cmd.exe" exited with code 1. [D:\projects\tensorflow\tensorflow\contrib\cmake\build\tf_core_gpu_k
ernels.vcxproj]

    2615 Warning(s)
    1 Error(s)

Time Elapsed 01:25:40.39

I googled and it seems this is a path problem, i rebuild the config using this but the problem persist.
cmake .. -A x64 -DCMAKE_BUILD_TYPE=Release ^ -DSWIG_EXECUTABLE="C:/swigwin-3.0.12/swig.exe" ^ -DPYTHON_EXECUTABLE="C:/ProgramData/Anaconda3/python.exe" ^ -DPYTHON_LIBRARIES="C:/ProgramData/Anaconda3/libs/python36.lib" ^ -DPYTHON_INCLUDE_DIR="C:/ProgramData/Anaconda3/Include" ^ -DNUMPY_INCLUDE_DIR="C:/ProgramData/Anaconda3/lib/site-packages/numpy/core/include" ^ -Dtensorflow_ENABLE_GPU=ON ^ -DCUDNN_HOME="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.0" ^ -Dtensorflow_WIN_CPU_SIMD_OPTIONS=/arch:AVX
here is the last 9999 of the new error log.
tf-compile-build-error-log.txt

I figure this might be vc 2017 cuda 9 problem, i switched back to vc 2015 and cuda 8 and finally successfully compile it and get it installed. It will still be great if the issue with vc 2017/cuda9 could be resolved tho. Thank you very much for your help.

There are a lot of warnings about code page issues. For example:

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\include\cuda_runtime.h : warning C4819: The file contains a character that cannot be represented in the current code page (950). Save the file in Unicode format to prevent data loss (compiling source file D:\projects\tensorflow\tensorflow\core\kernels\slice_op.cc) [D:\projects\tensorflow\tensorflow\contrib\cmake\build\tf_core_kernels.vcxproj]

I'm not sure how the system code page interacts with the Visual C++ compiler—and there's no indication in the error message about exactly what character is causing problems—but it seems like it might be necessary to change your system settings to get this to build cleanly.

thanks for the direction, may you clarify a little bit more on changing system settings? I am not very sure where should I be looking at. Also, is compiler_arch_native_support failed an issue? I noticed after the upgrade from tf 1.3 to 1.4, my gpu utility (2 1080 ti) drop from 90% to 30 %, efficiency decreased as well by more than 3 quarters. This is why I try to build from source ( I previously use pip install tensorflow_gpu --upgrade). I notice my pc support AVX2 but I am not sure how to compile arch:AVX/AVX2 together, but nevertheless, the new build does not yield any performance gain. CPU remains excessively high in usage (abnormally) and gpu outrageously low utility. I am pretty desperate now, would be great to have your advice.

Additionally I am not able to compile with AVX2, despite when running tf 1.4 installed from pip, it says my pc supports so and would benefit from compiling with it. Below is the log, building was successful, but compiling yield below. Would be great if someone could help.
compile AXV2 error.txt

FYI I've been able to compile everything (without testing python bindings, and using AVX, not AVX2), see #14801. However I've discovered that despite everything ran fine, the VS2017 (until 15.4) distribution introduced a bug in the /WHOLEARCHIVE trick resulting in unfilled factories (session, device...) as mentioned in. Therefore I am stuck using vs2017 for linking my application with TF. @aluo-x

thank you for the pointer!

If you are seeing a failure similar to this one:
http://ci.tensorflow.org/job/tf-master-win-gpu-cmake/
This looks like an nvcc bug, and it is reported to NVIDIA. We are waiting for their investigation

thank you! however, I could not see the fail build log from your page. There seems a lot of experiment has been going on tho. Good to know!

@gunan Any updates on this? Can we link to an nvidia page / thread to help triage this issue?

https://bitbucket.org/eigen/eigen/pull-requests/351/win-nvcc/diff
Once the above change is merged, I will bump TF eigen dependency, and we should be all fixed.

@arunmandal53
Just change CUDA host compiler to v140 by the following CMake option (you can also find it in cmak-gui by checking "Advanced"):
-DCUDA_HOST_COMPILER="C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/amd64/cl.exe

NonCUDA code can still be compiled by v141 in VS 2017.
Reference: https://medium.com/@shiweili/building-tensorflow-c-shared-library-on-windows-e79c90e23e6e

@chaosink thank you. I will test it.


Request: Update text style

I added a custom text style and later wanted to change the font or color. I had to add a new style instead of updating a current style. It would be nice if it were possible to simply update an existing style.

You can update a style by right-clicking on it and selecting update style :) This will update the right-clicked style to whatever you currently have selected in the panel


Display absolute links to downloaded files via youtube-dl?

Hi,
I've set up youtube-dl on my shared Debian 3 host so I can access it via an URL to pass address to what I want to download so it would download it on my host.
I was wondering if there is a way to display URL links to file(s) downloaded by last request on a web page for easier download.
So for example, if you call domain.com/youtubedl.php?url=www.youtube.com/watch?v=000000 would download it and then display this on the page:
http://domain/download_directory/youtubefile.mp4
so you can easily download it by clicking on that link.
Thanks

I've set up youtube-dl on my shared Debian 3 host so I can access it via an URL to pass address to what I want to download so it would download it on my host.

youtube-dl does not provide such functionality. Ask support of the web front-end you are using.

I'm not using any frond-end. I've wrote a bunch of php files to handle my calls to youtube-dl.
Also if you var_dump output of youtube-dl, you can see the relative addresses of the files it's downloaded but it also has a lot of other info that I do not need and the paths are not absolute as well. So I was wondering if it's possible to achieve what I asked for with it.

all youtube-dl paths are relative to the current directory.


the connect customizations for the default implementations of the standard algorithms should be noexcept when they can be.

 Issue by ericniebler
Friday Jan 28, 2022 at 18:02 GMT
Originally opened as https://github.com/NVIDIA/stdexec/issues/438

None

 Comment by lewissbaker
Wednesday Feb 02, 2022 at 06:02 GMT

That seems like an overly broad requirement...
If my connect allocates memory can I not just declare it unconditionally noexcept(false)?
Am I missing some context here?

 Comment by ericniebler
Wednesday Feb 02, 2022 at 07:52 GMT

Sorry, I mean that the connect customizations for the default implementations of the standard algorithms should be noexcept when they can be.

I think the default implementations are all specified in terms of basic-sender which has appropriate noexcept clauses as far as I can tell.
Is there anything more to be done here, @ericniebler ?


Support call transcription

Add support for call transcription in the standard.
Multiple formats should be supported(eg. VTT, SRT).

We would be very interested in this have you started on this at all?

@kemmiso I have a local draft, I'll start a draft PR so we can start discussion on this.


Expose ServicePublishingStrategy option for kubevirt e2e

Currently the service publishing policy in the e2e suit is hard coded to the default Ingress, which utilizes usage of LoadBalancer. We would like to expose the option to use NodePort as well.
Signed-off-by: Oren Cohen<EMAIL_ADDRESS>
What this PR does / why we need it:
Which issue(s) this PR fixes (optional, use fixes #<issue_number>(, fixes #<issue_number>, ...) format, where issue_number might be a GitHub issue, or a Jira story:
Fixes #
Checklist

[ ] Subject and description added to both, commit and PR.
[ ] Relevant issues have been referenced.
[ ] This change includes docs.
[ ] This change includes unit tests.

currently this is not needed.
/close


23 stage total, compare fucntions , etc. from mm bot

worst total
best total
best total full mismatch


Affix issue with transition from bottom to floating mode

Hello guys! I'm using Affix plugin v3.3.5 on my site and found strange behaviour. Here is code that initializes plugin:
           // inside JQuery.ready
            var gap = 40, footerHeight = $('#site-footer').outerHeight(true) + gap;
            $('.question-list')
                .affix({
                    offset: {
                        top: 160,
                        bottom: footerHeight
                    }
                })

The issue is when you scroll page down to edge bottom position and then try to return to floating mode (scroll back to the center of the page) affixed panel will be scrolled top. After some investigation I've found that this happens due to jQuery.offset set position: relative for static positioned elements.
I double checked this issue on bootstrap site and found that there is fix of this bug inside CSS. Due to position: absolute set by default for .affix-bottom JQuery.offset will not set position: relative

Hope I was clear :) Thanks!

Experiencing this as well. Has there been any update?

Bootstrap 3 is no longer being officially developed or supported.
All work has moved onto our next major release, v4. As such, this issue or pull request is being closed as a "won't fix." For additional help and support, we recommend utilizing our community resources. Thanks for your understanding, and see you on the other side of v4!
<3,
@mdo and team


Generate ToString method for custom payload types

To improve the experience of using custom payload types this PR automatically generates a ToString() override following the convention of C# record types.
Specifically, for a payload RgbPayload with the following spec:
payloadSpec:
  Green:
    offset: 0
    description: The intensity of the green channel in the RGB LED.
  Red:
    offset: 1
    description: The intensity of the red channel in the RGB LED.
  Blue:
    offset: 2
    description: The intensity of the blue channel in the RGB LED.

The following method is generated:
/// <summary>
/// Returns a <see cref="string"/> that represents the payload of
/// the Rgb register.
/// </summary>
/// <returns>
/// A <see cref="string"/> that represents the payload of the
/// Rgb register.
/// </returns>
public override string ToString()
{
    return "RgbPayload { " +
        "Green = " + Green + ", " +
        "Red = " + Red + ", " +
        "Blue = " + Blue + " " +
    "}";
}

Fixes #59

I think this is also forwards compatible with https://github.com/harp-tech/protocol/issues/40 but just making sure that's the case before merging.

@bruno-f-cruz I was also thinking about possible edge cases. The heterogeneous structure case is actually fine since this will just iterate through all public fields in the struct. This is also a simpler case than the one we have for sgen because structs don't support inheritance, so we don't have to worry about complex inheritance trees.
The only case which might be interesting to consider in the future is a case of nested structs, e.g. something like:
public struct TrackingPayload
{
    public VectorPayload Position;
    public float Confidence;
}

In this case we have a struct made up of other structs, but even in this case it should be fine. Assuming that all structs implement the same formatting scheme, we would have:
VectorPayload { X = 0, Y = 0, Z = 1 }
KinematicsPayload { Position = VectorPayload { X = 0, Y = 0, Z = 1 }, Confidence = 0.5 }


Error upon startup

Hello, I am getting this error on startup. My config is below. This is installed on an rpi 3B+
[2020-11-04 18:19:06] [lib.camera.driveway     ] [ERROR   ] - Error starting decoder pipe! mmal: mmal_vc_port_info_set: failed to set port info (2:0): EINVAL
mmal: mmal_vc_port_set_format: mmal_vc_port_info_set failed 0x206a8b0 (EINVAL)
mmal: mmal_port_disable: port vc.ril.video_decode:in:0(H264)(0x206a8b0) is not enabled
mmal: mmal_port_disable: port vc.ril.video_decode:out:0(0x206abd0) is not enabled
mmal: mmal_port_disable: port vc.ril.video_decode:ctr:0(0x2065920) is not enabled
 Retrying in 5 seconds

`# See the README for the full list of configuration
cameras:

name: Driveway
mqtt_name: viseron_driveway
host: <IP_ADDRESS>
port: 554
username: XXXXXX
password: XXXXXXX
path: /h264Preview_01_main
width: 2560
height: 1920
fps: 30
motion_detection:
interval: 1
trigger_detector: false
object_detection:
interval: 1
labels:
- label: person
confidence: 0.9

MQTT is optional
mqtt:
broker: <IP_ADDRESS>
port: 1883
username: homeassistant
password: XXXXXXXXX
`

Can you show me your docker run/docker compose?

Hello, thanks for the help, this is what I used to run the container.

Have you increased the allocated GPU memory as indicated in the README?
Included the snippet below (Its under the Raspberry Pi 3b+ section in Getting Started)
To do this edit /boot/config.txt and set gpu_mem=256 and then reboot.

I changed the camera to the substream and increased the GPU memory and all is working now.  Thanks!


Make JAVA_HOME configurable on the devappserver child process

This was motivated by https://github.com/GoogleCloudPlatform/google-cloud-intellij/issues/927. These changes allow callers to specify a java_home directory in the devserver RunConfiguration.
Note that up until now, RunConfiguration has been used to only specify arguments to dev_appserver.py. This change would be the first environment variable that is added to the RunConfiguration interface - I think it's kind of nice to abstract that from the user a bit; I'm interested to hear other thoughts on that.

sorry. didn't mean to unassign everyone there. Some weird UI interaction with github.

@joaoandremartins @etanshaul

Have you tried running dev_appserver with --dev_appserver_log_level debug and checking that the called java binary is indeed the {$JAVA_HOME}/bin/java one?

@joaoandremartins it does work — it's the only workaround we found for the Java8/JSP problem https://github.com/GoogleCloudPlatform/google-cloud-eclipse/issues/590#issuecomment-241177482

Oh, OK. It should work, though I tried calling dev_appserver.py from a shell where JAVA_HOME was set and the wrong java binary was still being called. i.e., java from PATH, not JAVA_HOME.

Are there any other environment variables that may make sense to allow the user to modify?  I guess any other environment variables should be put in the appengine-web.xml or app.yaml?

@joaoandremartins From my testing, the JAVA_HOME environment variable was being respected. Did you export it when setting the variable for testing?
@briandealwis I'm not sure of any other env variables that it makes sense to add right now.  If we ever have a need, we can always add more using this mechanism in the future.

Ah, I might have forgotten to export yes.
LGTM


Can we implement CustomTestMethod(TestCategory,TestName,TestID) like that

Need info ,
Can we create the custom Test method as CustomTestMethod(Category,TestName,TestID) like that . Pls help me . If it is possible

Hi @vigneshbabuv! Sorry for the late answer, could you give us a little more details about your request?

I will move forward by closing this issue. Please feel free to ping back here or to open a new issue.


Make abbreviated type hints expandable

As a follow-up to https://github.com/clangd/clangd/issues/2268, if a type hint contains an abbreviated template argument list (<...>), we could use the new InlayHintLabelPart.command facility in LSP 3.17 to make the ... clickable, and react to the click by replacing it with the full template arguments.
The contents after clicking do not need to respect the type name length limit (since, by clicking, the user has given an explicit indication that they want to see the full type hint). However, if the arguments are themselves very long, we may still want to abbreviate any nested template argument lists inside them as <...> until those are also clicked.

See also https://github.com/clangd/clangd/issues/1535 which is about using the LSP 3.17 API additions to make class names in type hints also clickable (with the effect of triggering go-to-def on the class).


[Bug]: Upgrade button didn't work

Description
I have the version v4.0.0-beta.25 installed and I want to upgrade to the latest beta version.
When I click on the button to upgrade, it shows a success toast saying "Upgrading to next version..." but nothing happens.
Is it possible to upgrade manually?
Minimal Reproduction (if possible, example repository)
Upgrade Coolify version by clicking on upgrade button
Exception or Error
No response
Version
v4.0.0-beta.25

It is fixed now.
Yes. Just execute the installation script again. curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash

Not fixed for me.
I tried to update via the button and via your proposed solution.


Added new procedure "Permission Set Relation".AddNewPermissionSetRelation

Fixes https://github.com/microsoft/ALAppExtensions/issues/21197

@microsoft-github-policy-service agree [company="WSB Solutions B.V."]

@microsoft-github-policy-service agree company="WSB Solutions BV"

Let's get this processed.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 52176.

Availability update: We will publish a fix for this issue in the next update for release 21.
Build ID to track: 52214.


When you sell a partial, the stop loss needs to be adjusted for the amount sold

Edit: original problem was that the stop didn't adjust quantities. This works and was probably a user error.
New problem is that when you trim to close the position, the stop loss doesn't match the shares remaining. The program should check the stop loss quantity against the positions quantity

Ah what!? Guess that's another thing broken. It's coded in as well

Tried this again:

Stop amount adjusts to the amount remaining (good)
Stop then moved to breakeven (good)
There was 1 scenario where I had 1 share left, then I trim half to close the position. It sells the last share, but then I'm still left with a stop loss of 1 share. The stop triggers and im short 1 share

Another Scenario:

buy 14 shares
sell 4 shares on the TWS. 10 shares remain, stop adjusted to 10 shares
Trim 1/2 shares using ktrade, sells 7 shares, 3 shares remain. Stop is 7 shares

Both this and the issue in the title appear to come from the app not checking position size vs the stop size


Add initial documents.

This PR added initial documents including:

CONTRIBUTING.md
OWNERS
README.md
code-of-conduct.md
docs/proposal.md

/cc @feiskyer @resouer
@kubernetes-incubator/maintainers-cri-containerd

LGTM

Glad to see this project, please ping me if needs any help here!

Cool. Looking forward to this feature.

+1 for this project!

Thanks for reviewing everyone! Addressed comments and will squash the commit before merging.

@mikebrow Thanks for your patience! :)


AssertionError when test-lw-tree and test-lw-nn

I run the program following the hyper-params.md:
#lw-tree census13
just train-lw-tree census13 original base 64 200 100000 0 123
just test-lw-tree original_base-lwxgb_tr64_bin200_100k-123 census13 original base True 123
#lw-nn census13
just train-lw-nn census13 original base 64_64_64 200 100000 128 0 123
just test-lw-nn original_base-lwnn_hid64_64_64_bin200_ep500_bs128_100k-123 census13 original base True 123
The train is OK. But when it comes to the test, AssertionError happened and then the program exit.

Expect to your reply. Thanks a lot!

Can you show us which command did you use to generate the workload? Are these 2 methods using the same workload?

Workload Generation Command:
just wkld-gen-base census13 original name='base'
The 2 methods use the same workload.
PS:
I find that if I set the use_cache to False, it is OK. Can I ask why?
Thanks.

Workload Generation Command:
just wkld-gen-base census13 original name='base'
The 2 methods use the same workload.
PS:
I find that if I set the use_cache to False, it is OK. Can I ask why?
Thanks.

We cache the encoded features and labels for both lw-nn and lw-tree for convenience (e.g. do not need to query the database every time to get CE features). We also add this assertion line to make sure the cached workload is align with the testing one. The default value for use_cache is True, which means we will directly load the encoded query for test if there is one and check the consistency. If you set it to False then this query encoding will be generated on the fly so there won't be any check. The cache creation and loading logic can be found in lecarb/estimator/lw/common.py: load_lw_dataset.
In your case there is a mismatch between your cached workload and the workload you are testing. Can you try to delete the cache file (located in the lw directory for each dataset) and run train and test again?

Workload Generation Command:
just wkld-gen-base census13 original name='base'
The 2 methods use the same workload.
PS:
I find that if I set the use_cache to False, it is OK. Can I ask why?
Thanks.

We cache the encoded features and labels for both lw-nn and lw-tree for convenience (e.g. do not need to query the database every time to get CE features). During training, we will directly load this cache if it exists otherwise we will create one. For test, if use_cache is True, we do the same and also add this assertion line to make sure the cached workload is align with the testing one. If you set use_cache to False then we won't load the cache and encoding for each test query will be generated on the fly so there won't be any check. The cache creation and loading logic can be found in lecarb/estimator/lw/common.py: load_lw_dataset.
In your case there is a mismatch between your cached workload and the workload you are testing. Can you try to delete the cache file (located in the lw directory for each dataset) and run train and test again?

I followed your words and retried. It is OK now! Thank you for your patient and clear explain!


Unexpected general type error upon list concatenation

Environment data

Language Server version: XXX
OS and version: macOS Big Sur
Python version (& distribution if applicable, e.g. Anaconda): 3.9.1 Anaconda

Expected behaviour
The list concatenation step should not be highlighted as an error.
Actual behaviour
When concatenating 2 lists, both of types list[Tr], Pylance throws Operator "+" not supported for types "list[Tr]" and "list[Tr]"
Logs

Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
[Info  - 12:22:14 AM] Pylance language server 2021.5.1 (pyright df8b6b3c) starting
[Info  - 12:22:14 AM] Server root directory: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist
[Info  - 12:22:14 AM] No configuration file found.
[Info  - 12:22:14 AM] No pyproject.toml file found.
[Info  - 12:22:14 AM] Setting pythonPath for service "enema": "/Users/kjczarne/opt/anaconda3/envs/evday/bin/python"
[Warn  - 12:22:14 AM] stubPath /Users/kjczarne/Desktop/repos/enema/typings is not a valid directory.
[Info  - 12:22:14 AM] Assuming Python version 3.9
[Info  - 12:22:14 AM] Assuming Python platform Darwin
Search paths for /Users/kjczarne/Desktop/repos/enema
  /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib
  /Users/kjczarne/Desktop/repos/enema
  /Users/kjczarne/Desktop/repos/enema/typings
  /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stubs/...
  /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/bundled/stubs
  /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9
  /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/lib-dynload
  /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages
  /Users/kjczarne/Desktop/repos/cera
  /Users/kjczarne/Desktop/repos/archive/get_to_uw
  /Users/kjczarne/Desktop/repos/archive/diet_app
  /Users/kjczarne/Desktop/repos/enema
  /Users/kjczarne/Desktop/repos/px_speedread
[Info  - 12:22:14 AM] Searching for source files
[Info  - 12:22:14 AM] Found 5 source files
[FG] parsing: /Users/kjczarne/Desktop/repos/enema/enema/app.py (54ms)
[FG] parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/builtins.pyi [fs read 3ms] (66ms)
[FG] binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/builtins.pyi (28ms)
[FG] binding: /Users/kjczarne/Desktop/repos/enema/enema/app.py (6ms)
[Info  - 12:22:14 AM] Background analysis(1) root directory: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist
[Info  - 12:22:14 AM] Background analysis(1) started
Background analysis message: setConfigOptions
Background analysis message: ensurePartialStubPackages
Background analysis message: setTrackedFiles
Background analysis message: markAllFilesDirty
Background analysis message: setFileOpened
Background analysis message: setFileOpened
Background analysis message: setFileOpened
Background analysis message: setFileOpened
Background analysis message: getDiagnosticsForRange
Background analysis message: getSemanticTokens full
[BG(1)] getSemanticTokens full at /Users/kjczarne/Desktop/repos/enema/enema/app.py ...
[BG(1)]   parsing: /Users/kjczarne/Desktop/repos/enema/enema/app.py (58ms)
[BG(1)]   parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/builtins.pyi [fs read 3ms] (65ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/builtins.pyi (29ms)
[BG(1)]   binding: /Users/kjczarne/Desktop/repos/enema/enema/app.py (3ms)
[BG(1)]   parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/typing.pyi [fs read 1ms] (26ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/typing.pyi (8ms)
[BG(1)]   parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/_typeshed/__init__.pyi [fs read 1ms] (5ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/_typeshed/__init__.pyi (1ms)
[BG(1)]   parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/typing_extensions.pyi [fs read 0ms] (1ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/typing_extensions.pyi (1ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash/__init__.py [fs read 0ms] (1ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash/__init__.py (2ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_core_components/__init__.py [fs read 0ms] (4ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_core_components/__init__.py ...
[BG(1)]     parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_core_components/_imports_.py [fs read 0ms] (2ms)
[BG(1)]     binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_core_components/_imports_.py (0ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_core_components/__init__.py (4ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_html_components/__init__.py [fs read 1ms] (2ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_html_components/__init__.py ...
[BG(1)]     parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_html_components/_imports_.py [fs read 0ms] (8ms)
[BG(1)]     binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_html_components/_imports_.py (2ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_html_components/__init__.py (16ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_html_components/Button.py [fs read 1ms] (12ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_html_components/Button.py (1ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash/development/base_component.py [fs read 0ms] (10ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash/development/base_component.py (4ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/future/utils/__init__.py [fs read 0ms] (24ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/future/utils/__init__.py (5ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash/_utils.py [fs read 0ms] (6ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash/_utils.py (2ms)
[BG(1)]   parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/abc.pyi [fs read 0ms] (1ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/abc.pyi (0ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_html_components/Label.py [fs read 0ms] (2ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_html_components/Label.py (1ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/plotly/express/__init__.py [fs read 0ms] (5ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/plotly/express/__init__.py (1ms)
[BG(1)]   parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/bundled/stubs/pandas/__init__.pyi [fs read 0ms] (4ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/bundled/stubs/pandas/__init__.pyi (1ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/__init__.py [fs read 0ms] (19ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/__init__.py ...
[BG(1)]     parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/__init__.py [fs read 0ms] (11ms)
[BG(1)]     binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/__init__.py ...
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/numeric.py [fs read 0ms] (24ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/numeric.py ...
[BG(1)]         parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/umath.py [fs read 0ms] (2ms)
[BG(1)]         binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/umath.py ...
[BG(1)]           parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/bundled/native-stubs/numpy/core/_multiarray_umath.pyi [fs read 2ms] (62ms)
[BG(1)]           binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/bundled/native-stubs/numpy/core/_multiarray_umath.pyi (16ms)
[BG(1)]         binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/umath.py (78ms)
[BG(1)]         parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/numerictypes.py [fs read 0ms] (8ms)
[BG(1)]         binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/numerictypes.py (1ms)
[BG(1)]         parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/fromnumeric.py [fs read 1ms] (17ms)
[BG(1)]         binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/fromnumeric.py (4ms)
[BG(1)]         parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/arrayprint.py [fs read 0ms] (16ms)
[BG(1)]         binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/arrayprint.py (7ms)
[BG(1)]         parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/_asarray.py [fs read 0ms] (1ms)
[BG(1)]         binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/_asarray.py (1ms)
[BG(1)]         parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/_ufunc_config.py [fs read 0ms] (3ms)
[BG(1)]         binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/_ufunc_config.py (1ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/numeric.py (147ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/records.py [fs read 0ms] (7ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/records.py (6ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/memmap.py [fs read 0ms] (2ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/memmap.py (1ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/function_base.py [fs read 0ms] (9ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/function_base.py (2ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/machar.py [fs read 0ms] (4ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/machar.py (3ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/getlimits.py [fs read 0ms] (9ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/getlimits.py (3ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/shape_base.py [fs read 1ms] (9ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/shape_base.py (2ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/einsumfunc.py [fs read 0ms] (14ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/einsumfunc.py (4ms)
[BG(1)]     binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/core/__init__.py (249ms)
[BG(1)]     parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/__init__.py [fs read 1ms] (2ms)
[BG(1)]     binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/__init__.py ...
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/type_check.py [fs read 1ms] (5ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/type_check.py (2ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/index_tricks.py [fs read 0ms] (7ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/index_tricks.py (3ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/function_base.py [fs read 0ms] (32ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/function_base.py (14ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/nanfunctions.py [fs read 0ms] (8ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/nanfunctions.py (3ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/shape_base.py [fs read 1ms] (11ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/shape_base.py (1ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/stride_tricks.py [fs read 0ms] (1ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/stride_tricks.py (0ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/twodim_base.py [fs read 1ms] (4ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/twodim_base.py (1ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/ufunclike.py [fs read 0ms] (1ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/ufunclike.py (2ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/histograms.py [fs read 0ms] (5ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/histograms.py (2ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/polynomial.py [fs read 0ms] (10ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/polynomial.py (4ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/utils.py [fs read 0ms] (12ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/utils.py (4ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/arraysetops.py [fs read 0ms] (5ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/arraysetops.py (2ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/npyio.py [fs read 0ms] (20ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/npyio.py (13ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/financial.py [fs read 0ms] (5ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/financial.py (3ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/arraypad.py [fs read 0ms] (4ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/arraypad.py (4ms)
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/_version.py [fs read 0ms] (0ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/_version.py (1ms)
[BG(1)]     binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/lib/__init__.py (195ms)
[BG(1)]     parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/matrixlib/__init__.py [fs read 0ms] (0ms)
[BG(1)]     binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/matrixlib/__init__.py ...
[BG(1)]       parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/matrixlib/defmatrix.py [fs read 0ms] (4ms)
[BG(1)]       binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/matrixlib/defmatrix.py (2ms)
[BG(1)]     binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/matrixlib/__init__.py (6ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/__init__.py (466ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash/dependencies.py [fs read 0ms] (1ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash/dependencies.py (4ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_table/__init__.py [fs read 0ms] (1ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_table/__init__.py ...
[BG(1)]     parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_table/_imports_.py [fs read 0ms] (0ms)
[BG(1)]     binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_table/_imports_.py (0ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_table/__init__.py (1ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_daq/__init__.py [fs read 0ms] (1ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_daq/__init__.py (0ms)
[BG(1)]   parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/sqlite3/__init__.pyi [fs read 0ms] (1ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/sqlite3/__init__.pyi ...

[BG(1)]     parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/_collections_abc.pyi [fs read 0ms] (0ms)
[BG(1)]     binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/_collections_abc.pyi (1ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/collections/__init__.pyi (2ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash/resources.py [fs read 0ms] (1ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash/resources.py (1ms)
[BG(1)]   parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stubs/pkg_resources/pkg_resources/__init__.pyi [fs read 0ms] (3ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stubs/pkg_resources/pkg_resources/__init__.pyi (2ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash/_validate.py [fs read 0ms] (3ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash/_validate.py (1ms)
[BG(1)]   parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stubs/cryptography/cryptography/fernet.pyi [fs read 0ms] (1ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stubs/cryptography/cryptography/fernet.pyi (0ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_auth/basic_auth.py [fs read 0ms] (1ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_auth/basic_auth.py (0ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_auth/auth.py [fs read 0ms] (2ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/dash_auth/auth.py (0ms)
[BG(1)]   parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stubs/six/six/__init__.pyi [fs read 0ms] (2ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stubs/six/six/__init__.pyi (1ms)
[BG(1)]   parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/bundled/stubs/pandas/core/series.pyi [fs read 1ms] (16ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/bundled/stubs/pandas/core/series.pyi (3ms)
[BG(1)]   parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/bundled/stubs/pandas/_typing.pyi [fs read 1ms] (4ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/bundled/stubs/pandas/_typing.pyi (0ms)
[BG(1)]   parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/datetime.pyi [fs read 0ms] (2ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/typeshed-fallback/stdlib/datetime.pyi (3ms)
[BG(1)]   parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/bundled/stubs/pandas/core/indexes/api.pyi [fs read 0ms] (6ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/bundled/stubs/pandas/core/indexes/api.pyi (0ms)
[BG(1)]   parsing: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/bundled/stubs/pandas/core/indexes/base.pyi [fs read 0ms] (2ms)
[BG(1)]   binding: /Users/kjczarne/.vscode/extensions/ms-python.vscode-pylance-2021.5.1/dist/bundled/stubs/pandas/core/indexes/base.pyi (1ms)
[BG(1)]   parsing: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/testing/__init__.py [fs read 0ms] (0ms)
[BG(1)]   binding: /Users/kjczarne/opt/anaconda3/envs/evday/lib/python3.9/site-packages/numpy/testing/__init__.py ...

[BG(1)]   checking: /Users/kjczarne/Desktop/repos/enema/enema/api.py (3ms)
[BG(1)] analyzing: /Users/kjczarne/Desktop/repos/enema/enema/api.py (3ms)
[BG(1)] analyzing: /Users/kjczarne/Desktop/repos/enema/enema/app.py ...
[BG(1)]   checking: /Users/kjczarne/Desktop/repos/enema/enema/app.py (11ms)
[BG(1)] analyzing: /Users/kjczarne/Desktop/repos/enema/enema/app.py (11ms)
[BG(1)] analyzing: /Users/kjczarne/Desktop/repos/enema/enema/crypto.py ...
[BG(1)]   checking: /Users/kjczarne/Desktop/repos/enema/enema/crypto.py (1ms)
[BG(1)] analyzing: /Users/kjczarne/Desktop/repos/enema/enema/crypto.py (1ms)
[BG(1)] analyzing: /Users/kjczarne/Desktop/repos/enema/enema/data_model.py ...
[BG(1)]   checking: /Users/kjczarne/Desktop/repos/enema/enema/data_model.py (1ms)
[BG(1)] analyzing: /Users/kjczarne/Desktop/repos/enema/enema/data_model.py (1ms)
Background analysis message: getDiagnosticsForRange
Background analysis message: getSemanticTokens range
[BG(1)] getSemanticTokens range 53:0 - 191:0 at /Users/kjczarne/Desktop/repos/enema/enema/app.py (15ms)
Background analysis message: resumeAnalysis
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange

Code Snippet / Additional information

I'm using a Dash callback where I'm trying to build a table by concatenating rows with content to table header row. The expected output from this callback should be a list of Tr elements. The code works but Pylance thinks this is a mistake. Am I missing something or is Pylance not treating the Tr element containing Tds the same way as a Tr element containing Ths?
@app.callback(
    Output('subsystems-table', 'children'),
    Input('load-button', 'n_clicks'),
    State('state-memory', 'data')
)
def on_load_subsystems(n_clicks, n_clicks_prev):
    query = 'select * from subsystems left join nodes ON subsystems.node_id = nodes.node_id;'
    df = pd.read_sql_query(query, db_connection())
    rows = get_rows(df)
    subsystem_records = [
        # if I use Th instead of Td, the error goes away
        html.Tr(children=[html.Td()]) for row in rows
    ]
    if not n_clicks_prev:
        n_clicks_prev = 0
    if n_clicks:
        if n_clicks > n_clicks_prev:
            lst = [
                html.Tr(children=[
                    html.Th("Subsystem"),
                    html.Th("Node"),
                    html.Th("Busy?"),
                    html.Th("Claim/Release")
                ])
            ] + subsystem_records
            return lst
        n_clicks_prev = n_clicks
    return []

The concatenation step where this happens:
lst = [
        html.Tr(children=[
            html.Th("Subsystem"),
            html.Th("Node"),
            html.Th("Busy?"),
            html.Th("Claim/Release")
        ])
] + subsystem_records

https://github.com/kjczarne/pylance_dash_issue_demo -> here's the minimal example of what's happening. Let me know if you need any extra info 🙂

Thanks for the sample. I'm able to repro the problem.
Here's what's happening. You're using the library dash, which has no type information. You've enabled type checking (set "python.analysis.typeCheckingMode" to "basic") but have not disabled "python.analysis.useLibraryCodeForTypes". When the latter setting is enabled (as it is by default), pylance will attempt to infer type information from the Python code in untyped libraries. This type information is good enough for completion suggestions, but it is often not accurate enough for type checking. You're running into one of those issues here.
One of the techniques pylance uses when inferring types from non-typed library code is something we refer to as "pseudo-generic inference". It's used for classes that have constructors with unannotated input parameters. To improve inferred types, such classes are treated as generic classes that are parameterized by the input parameters to its constructor. That's what's happening here. The Tr class is being treated as a generic, and you're creating two different types of Tr instances — a Tr[Td] and a Tr[Th] instance. Then you're trying to add lists of these different types together, which is a type violation.
Here are some suggested workarounds:

Set "python.analysis.useLibraryCodeForTypes" to false. We recommend doing this when enabling type checking for the reasons above. When this setting is disabled, pylance won't attempt to infer any type information from untyped libraries, so all symbols from dash will be treated as "Unknown" types.
You can add an explicit type annotation to your code. In the sample you provided, you would need to annotate the local  variable subsystem_records as follows: subsystem_records: list[html.Tr] = ...
If dash is a library that's important to you, you could try to convince its maintainers to add type information to it. Or, if you're really ambitious, you could contribute PRs to add type annotations to it. This would be a significant undertaking, but it would benefit everyone in the community who relies on this library and wants to use it with type checkers.

@erictraut, thank you very much for a comprehensive explanation! I will for now use explicit type hints in my own code. If I get enough time over the upcoming weekends I might try annotating Dash with type hints. My personal belief is that Python with typing information is just better Python. Thank you once again for maintaining Pylance, it makes my daily work a breeze 🚀


Docker image is big

https://microbadger.com/images/gableroux/ansible
I'm sure we can do better than 381.9 MB

This should be based on python:3.7-alpine instead.

👍


The TPP can retrieve the account selected by the user when approve the consent when hasn't provided by the TPP in the consent

Description
When the TPP initiates a payment flow the debtorAccount is optional.
When the user approves a consent the debtorAccount data is fill with data relating to the account selected by the user.
Current Behaviour
The TPP initiates a payment and submits a consent without providing the debtorAccount data.
When the TPP gets the consent by consentId, there are different behaviours depending on the consent status:
a) Consent Status AwaitingAuthorisation or Rejected:
- the consent is still the same as when submitted by the TPP. The debtorAccount still the same.
b) Consent Status Approved, Revoked or Consumed:
- The consent has been updated with the debtorAccount for the account the PSU chose when they approved the consent.
The case b is wrong because the TPP doesn't provide the debtorAccount data with the consent when initiating a payment. They must not know the user account data (debtorAccount) as that is not necessary for this flow.
Proposed solution (To be reviewed)

Create a SupplementaryDebtorAccount object field in the payment managed object, outside of OBIntentObject definition that represents the account that the TPP chose when providing their consent.

When the psu user approves a consent and submits the consent decisions:

Implement a logic to check:

Check the debtorAccount field on OBIntentObject stored in IDM, if the debtorAccount has data it means that the TPP has been provided it, and It has been validated when the TPP submitted the consent.
If the debtorAccount hasn't been provided in the consent, the account selected by the psu, must be stored in SupplementaryDebtorAccount field object instead of patch the OBIntentObject.

Fixed in https://github.com/SecureApiGateway/SecureApiGateway/issues/947


Add OS Architecture to Configuration documentation

Add information on the supported OS architecture to the Configuration file.

Should we get the info from the docker images themselves or use the condition in the 'docker-compose.yml.j2' file to know which images support ARM architectures?

You can relay on the condition in the docker-compose.yml.j2 file. I went through the images on docker-hub to check which ones are supported on ARM and which are not.


Adding a forgot button

Adding a forgot button to the ABPadLockScreenAbstractViewController. There was a use case when a user actually forgets their passcode, but is still within the limit of the number of attempts. The ability to use a mechanism to teardown the app and reset it if the user forgot their passcode.
The forgot button appears after the first failed attempt of the passcode. Honors UI_APPEARANCE_SELECTOR for customization. Also includes a required ABLockScreenDelegate protocol method forgotPinForPadLockScreenViewController: to handle the button event.
Updated the demo project.

Hey @mteece
Apologies for the delay in replying. That's a great idea and thanks for contributing. It looks great, but the only issue I had was that it breaks the swift example) It looks like Travis only tests the Obj-C example code, so that's on me.
I'll merge this and apply the fix to the dev branch to save you the trouble. I mentioned this in a different ticket, but let me know if you're interested in becoming a core contributor.
Cheers,
Aron

@abury No problem, thanks for pulling and merging. Certainly interested in becoming a core contributor.


HTTP1: Refactor HTTP1 Active Request to be defer deletable.

Signed-off-by: Kevin Baichoo<EMAIL_ADDRESS>
Commit Message: Refactor HTTP1 Active Request to be defer deletable
Additional Description: This will be helpful to enable us to safely remove the ActiveStream from being subscribed to the response encoder of the codec level stream. See https://github.com/envoyproxy/envoy/pull/18895#discussion_r752652736 for context.
Risk Level: low (refactor)
Testing: Ran tests
Docs Changes: NA
Release Notes: NA
Platform Specific Features: NA
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

LGTM. Can you check tidy? Thanks.
/wait


CSV2DB.py | options.header_names

Hi all,
I've been trying to transition pipeline_scrnaseq.py (https://github.com/kevinrue/scseq/blob/cgatcore/pipelines/pipeline_scrnaseq.py#L891) from CGATPipelines to CGATCore, and have been facing an issue with a couple of P.concatenate_and_load that doesn't seem to apply header="track,gene_id,counts" to the table loaded in the csvdb.
I can see in pipeline.log that this argument is being properly translated to --header-names=header="track,gene_id,counts" for CSV2DB.py (https://github.com/cgat-developers/cgat-core/blob/master/CGATCore/CSV2DB.py#L192).
However, the table loaded in the database ends up using the first row of the (header-less) concatenated table as the database column headers (i.e., the table column titles are: first sample name, first gene id, first count RNA-seq count value). For illustration:
sqlite> SELECT ENSMUSG00000102693 FROM featurecounts LIMIT 3;
ENSMUSG00000102693
ENSMUSG00000064842
ENSMUSG00000051951
ENSMUSG00000102851

(while SELECT gene_id FROM featurecounts LIMIT 3; complains for lack of gene_id column)
From reading the code, it looks like --header-names is only applied if --replace-header is active (https://github.com/cgat-developers/cgat-core/blob/master/CGATCore/CSV2DB.py#L106).
            if options.replace_header:
                columns = options.header_names

While I can see the use case for that (i.e., renaming column titles), it looks like the other scenario is not implemented yet (i.e., adding column titles to a header-less table), despite what the documentation states:
    parser.add_option("--replace-header", dest="replace_header",
                      action="store_true",
                      help="replace header with --header-names instead of "
                      "adding it [default=%default].")

Still, I can see that this part of the code hasn't been updated in months, which suggests that no one has had any issue in a while.
I've searched cgat-flow for example usage of P.concatenate_and_load to use as template yet couldn't find only 3, which don't see to use the --header-names option:
$ grep -Rn concatenate_and_load CGATPipelines/*py
CGATPipelines/PipelineRnaseq.py:964:        P.concatenate_and_load(infile, outfile,
CGATPipelines/pipeline_rnaseqqc.py:1290:    P.concatenate_and_load(infiles, outfile, regex_filename=regex)
CGATPipelines/pipeline_windows.py:2259:    P.concatenate_and_load(infiles, outfile,

Might I have uncovered a scenario not yet implemented?
Thanks in advance for the feedback and guidance!

Pull request to fix: https://github.com/cgat-developers/cgat-core/pull/41

@kevinrue would you be able to test the last commits in #41 ?


Add supported columns, add alphabetization of authorities.

Added two more supported columns from incentive-a-thon spreadsheet, and added alphabetization of authorities' file. Typical json sorting wasn't successful as the key's name was the value to sort by, which wouldn't parse when defining the sort function.
That being said, a generalized alphabetization solution for both programs and authorities is also being experimented with, but this simpler solution is viable due to incentive calculation being sorted by state locations of incentives.
Programs still need to be sorted, and README details this.

@datu925 , any suggestions/preferences on how to test? This bit of the script currently has a file dependency that is hard to emulate. One option would be to modify the code to take in the auth file name as a parameter (or write file name as param for generalized alphabetical sorting).
I've already manually tested, but likely want to be able to long-term test the general code in the future anyways.

@datu925 , any suggestions/preferences on how to test? This bit of the script currently has a file dependency that is hard to emulate. One option would be to modify the code to take in the auth file name as a parameter (or write file name as param for generalized alphabetical sorting).
I've already manually tested, but likely want to be able to long-term test the general code in the future anyways.

I did the expedient thing but I generally think it's better to have heavy-lifting methods operate on data rather than filenames for this reason.
Maybe we have separate the method into an inner and outer method. The outer one reads/writes the file, and the inner one (which is the one that's tested) takes in JSON (or an TypeScript authorities object) and returns JSON (or a TypeScript authorities object). We could also do that as part of the constructor or at least pass in and store the filename in the constructor.
The repeated-appending complicates that separation, but I think we should just do a single write anyway – see my other comment in the file.

I've added some types in order to enable sorting the objects (yay), but it was more manual work than anticipated. The schema used in the authorities' types file doesn't match exactly what we use/want here, which is why there are new "types" being defined. LMK what you think.


Fixed some unnecessary borrows and clones.

As suggested by the clippy.

r? @Eh2406
(rust_highfive has picked a reviewer for you, use r? to override)

Something is wheered about CI, these changes are failing at runtime? @ehuss thoughts on how that is possible?

I think that was an erroneous git2 publish, if CI is restarted this should work again.

Thanks, I removed the last two commits, as I think the removal of the clones was fine.
@bors r+

:pushpin: Commit 686ccfa462aa40eef8421ea84bca52141a1e8814 has been approved by ehuss

:hourglass: Testing commit 686ccfa462aa40eef8421ea84bca52141a1e8814 with merge 6a2d62fb402d1cd47d8eadf890522fdba3044b48...

:sunny: Test successful - checks-azure
Approved by: ehuss
Pushing 6a2d62fb402d1cd47d8eadf890522fdba3044b48 to master...


s/Device.hasNotch()/DeviceInfo.hasNotch()/

Fixes #471
DeviceInfo.hasNotch() is collect?
source: hasNotch()

we don't recommend using this library for safe-areas anymore, will update docs soon. you should use this instead: https://github.com/th3rdwave/react-native-safe-area-context


is header level altitude?

I have increased my head in the project, but three labels cannot Stickies,do you have a solution? thanks!
-----arleady fix this pro'blem

can you commit your layout xml?

can you commit your layout xml?

1.1.1 fix this problem


Range of scroll bar should be improved

Description of the new feature/enhancement

The way scroll bar working now may bring some inconvenience.
Like this situation:

I often do this when I got too much windows opened.
The problem is....
I can't scroll the window (scroll bar have already reached it's bottom), and I can't see the content below ( I cricled it by red pen in the following picture )
I even can't see the command i'm typing now. really terrible....

but traditional cmd doesn't have this problem (in fact it have, but in most case I won't output so many text that use up the whole buffer, by default, 9001 line. )

Proposed technical implementation details (optional)
I think the way Visual Studio Text Editor work is better. (Extend the scroll bar range, so that the last line can be shown on the top of the window)
like this:
So even if I move half of the window out of the working area, I can still work without resize the window.

Just simply calculate the windows height divided by the height per line, and add it to the scroll range.

Is this hard to implement caused by VT sequence?

@kernelbin Thanks for the request, and sorry for the delay. This is one thing we will probably never do: we have worked so hard in conhost to make it possible not to do this.
If you want your window to be at the bottom of the screen, but still display all of its content ... can you not just make it shorter?

Oh ... It's a pity. But why worked hard to not to do this?
VT Sequence caused this or something else?

It makes it very hard to support every VT sequence. Many VT sequences operate on the “screen”, and if we let the “screen” scroll halfway off the window it will be very difficult.


SSH Reverse Proxy

How can I make it that who I ssh1.mydomain.tk calls are forwarded to a ssh server on the local network?
Thanks :)

Forward 22 port from your ext server to your local server
Отправлено из Mail.ru для Android пятница, 28 июня 2019г., 10:37 +03:00 от Marlon<EMAIL_ADDRESS>:

How can I make it that who I ssh1.mydomain.tk calls are forwarded to a ssh server on the local network?
Thanks :)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly,  view it on GitHub , or  mute the thread .

But I have to access more than one
SSH1 -> ssh1.mydomain.tk
SSH2 -> ssh2.mydomain.tk
....
I only have one ip.
What do I have to change the web interface


xcontract feature suggestion

Hi all,
As per our email (with Joris), I am opening up this issue to suggest adding in my xcontract site link feature which would allow dApps to be run with a click of the button (with ABI and contract address in url params). It would make it easy for ethereum devs to have their dApps runnable on a frontend without having to build one themselves.
https://xcontract.herokuapp.com/
Cheers and keep up the good work!

Do you have documentation for using this :)

Hi, there is some here: https://github.com/James-Sangalli/xcontract in documentation.md and the README includes an example URL call.
Still working on it thought as it is incomplete right now.
Cheers.

James has fixed up xcontract a bit and it can now take verified contract addresses in the URL without the need for the ABI.
Example: https://xcontract.herokuapp.com/api/0xfb6916095ca1df60bb79ce92ce3ea74c37c5d359


x-nullable support?

As per the conversation here: #63
The swagger spec doesn't support null as a type value. So when testing for an empty object the distinction between {} and { prop: null } doesn't make any sense.
I propose that swagger-model-validator supports x-nullable on properties. See add "null value support" in spec? https://github.com/OAI/OpenAPI-Specification/issues/229
This would allow {} and { prop: null } to validate differently, but I am currently unsure what other effects this would have on validation logic.

Are we trying to tell us which properties are unexpectedly null so we can easily track them down?
That is already really handled by the required parameter.  Adding support for the x-isnullable or x-nullable could make sense but I'm not sure of the use case. Any removal of a value should usually be done under a DELETE and could be handled without this.


bump joi-to-json-schema

Currently, joi-to-json-schema has several issues, so i've submitted a new PR request.
For further details, Please refer to: https://github.com/lightsofapollo/joi-to-json-schema/pull/28
but currently package is not available from npm (because it's not merged/published to npm yet),
nor from github directly (dist files ignored by .gitignore).
so i needed to fork it, and publish it as scoped package to use updated package from corgi.
If that PR has merged in near future, we should rollback this changes

👍


Demonstrate mock-based unit-testing using mocktail

Is your feature request related to a problem? Please describe.
We've found that our error handling of edge cases - particularly, but not only, related to connectivity - needs to be improved. Testing these scenarios exhaustively in functional or end-to-end tests is difficult and time-consuming both in engineer effort and in adding to how long it takes those tests to run. Unit testing using mock objects to simulate network interactions provides us a way to write many test permutations which can execute rapidly and cover ~all of the edge cases.
Describe the solution you'd like

A solid example of using mocktail to write unit tests for a non-trivial part of the @ platform.
Do a talk to walk through the example, and provide supporting materials

Closed by #481
Recording of tech talk and the accompanying slides are here


set beacons scan period

Hello!
I use your AprilBeacon-iOS-SDK. To start ranging all April beacons that are visible in range i use the method startRangingBeaconsInRegion: . Then I use delegate method  beaconManager:didRangeBeacons:inRegion:  to retrieve found beacons. This delegate method is calling every 1 second. How can I change this interval, for example if I want to retrieve found beacons more often?
Your SDK for Android has method “public void setForegroundScanPeriod(long scanPeriodMillis, long waitTimeMillis)”. Do you have analogy for iOS SDK?

SDK for iOS can not change the interval


Update mapInitialize to use non deprecated methods.

getItemInHand and getDurability are deprecated.

Actually thinking of it, is there even a need for this class loll

https://hastebin.com/epifurusoc.swift (just incase i need)


Catch absolute paths in our docs/code

Reference.
We should add a check to the CI scripts action - it should check absolute paths and fail if it does (can filter on the base URL).

so for example, if we're on jupysql's repo, we can add base_url = jupysql.ploomber.io to the pyproject.toml, then fail a check if we detect any links that start with the base URL, since those should be relative.


Promise reject doesn't fail the unit-test

This is a unit-test from on-http/spec/lib/services/sku-pack-service-spec.js
it('should fail to start with an invalid root', function() {
          fs.readdirAsync.rejects();
          skuService.start('./invalid').should.be.rejected;
      });

Mocha will generate following message. It passes the test though there is some error:
  SKU Pack Service
    ✓ should expose the appropriate methods
    ✓ should fail to start with an invalid root
Unhandled rejection AssertionError: expected promise to be rejected but it was fulfilled with undefined
    ✓ should start with a valid root
    ✓ should load a valid configuration file
    ✓ should not load an invalid configuration file

  5 passing (116ms)

If I change the unit-test to opposite assertion:
it('should fail to start with an invalid root', function() {
          fs.readdirAsync.rejects();
          skuService.start('./invalid').should.not.be.rejected;
      });

Mocha still marks it as passed:
  SKU Pack Service
    ✓ should expose the appropriate methods
    ✓ should fail to start with an invalid root
    ✓ should start with a valid root
    ✓ should load a valid configuration file
    ✓ should not load an invalid configuration file

  5 passing (141ms)

The expected behavior should the first one be failed.

@RackHD/corecommitters @zyoung51

This is another test case from on-tasks/spec/lib/jobs/analyze-os-repo-job-spec.js:
            describe('test the behavior if boot.cfg misses some required options', function() {
                var keys = ['mbootFile', 'tbootFile', 'moduleFiles'];
                keys.forEach(function(key) {
                    it('should throw error if not have required option [' + key + ']', function() {
                        var data = _.cloneDeep(esxParseResult);
                        delete data[key];
                        repoTool.downloadViaHttp.resolves();
                        repoTool.parseEsxBootCfgFile.returns(data);
                        return expect(job._esxHandle(repo)).eventually.be.rejected;
                    });
                });
            });

and the opposite one:
            describe('test the behavior if boot.cfg misses some required options', function() {
                var keys = ['mbootFile', 'tbootFile', 'moduleFiles'];
                keys.forEach(function(key) {
                    it('should throw error if not have required option [' + key + ']', function() {
                        var data = _.cloneDeep(esxParseResult);
                        delete data[key];
                        repoTool.downloadViaHttp.resolves();
                        repoTool.parseEsxBootCfgFile.returns(data);
                        return expect(job._esxHandle(repo)).eventually.not.be.rejected;
                    });
                });
            });

Both pass by mocha and generate following result:
  Analyze OS Repo Job

    analyze esxi repository
      test function _esxHandle
        ✓ should get correct result from boot.cfg
        ✓ should fetch BOOT.CFG if boot.cfg is not avaiable
        ✓ should throw error if both boot.cfg and BOOT.CFG is not avaiable
        ✓ should return correct result for normal input
        ✓ should not call the esxi handling function if osName is not 'esx'
        test the behavior if boot.cfg misses some required options
          ✓ should throw error if not have required option [mbootFile]
          ✓ should throw error if not have required option [tbootFile]
          ✓ should throw error if not have required option [moduleFiles]
    common options conversion
      ✓ should convert the repo to correct format
      test the behavior if not have required options
        ✓ should throw assertion error if miss required option [repo]
        ✓ should throw assertion error if miss required option [osName]
    test function _findHandle
      ✓ should find correct handle function
      ✓ should throw error if the handler is not a function
      ✓ should return empty function if not found handler

  14 passing (2s)

Seem the "not" doesn't work for Chai-as-Promise, this may not a real issue, but we need pay attention to this.

Ahh, got the cause:
The first one is due to the Promise is not returned.
The correct example:
return skuService.start('./invalid').should.be.rejected;

The second one is the chain syntax is not correct, the "eventually" should not connect with "rejected" or "fulfilled".
The wrong syntax is:
return expect(job._esxHandle(repo)).eventually.be.rejected;

The correct syntax is:
return expect(job._esxHandle(repo)).to.be.rejected;

The fix of latter one: https://github.com/RackHD/on-tasks/pull/96

@yyscamper is this line accurate or also incorrect:
return expect(job.cleanup()).to.eventually.be.fulfilled;

That's also in on-tasks (in spec/lib/base-job-spec.js)
There's also a line in on-core (in spec/lib/common/model-spec.js):
.should.be.fulfilled.and.eventually.be.undefined;

@heckj : yes, I need to scan all our repos.

The fix for SKU-pack has been included in this PR: https://github.com/RackHD/on-http/pull/68

@heckj : I tested this syntax is correct, so we don't need to change the on-core:
.should.be.fulfilled.and.eventually.be.undefined;

PR is merged, close it

I have created an issue for Chai-as-Promised: domenic/chai-as-promised#146


Generic Oauth2 backend won't start if the identity server is hosted on the same caddy instance

Describe the issue
I have a local identity server which I use for all my OpenID Connect purposes. Up until now I was using OpenResty to host all my services, including the OIDC server, and I would like to switch to Caddy. However, trying to use the Oauth2 backend fails if the OIDC server is also exposed on the same domain via the same Caddy instance, because it tries to validate the backend before it is actually exposed by Caddy.
Is it possible to delay this validation until after caddy is actually running and exposing the OIDC endpoints?

@alucryd , not at the moment. However, I can add the delayed start option over the weekend.

Thanks for the quick answer, that would be grand!

Oh yes, it would be great if something like a delyed start could be added.

@greenpau Any update on this? It would make selfhosted OIDC without the need to run two separate reverse proxies.

@michaelhthomas , will try making it a priority.

@alucryd , @IzeQube, @michaelhthomas , the delayed start is available in v1.4.19.
Please contribute to the development of Caddy auth plugins by:

Giving Github stars to github.com/greenpau/caddy-auth-portal and github.com/greenpau/caddy-auth-jwt
Promoting the plugins on Twitter by referencing the above repos and mentioning @caddyserver
Participating in https://caddy.community/
Blogging about your Caddy setup with auth plugins

Unfortunately @greenpau, though this new feature does work, the wait seems to be blocking the main thread and preventing Caddy from starting, which means the same errors occur. Because Caddy waits for the plugin to finish initializing, the proxy never starts and therefore the auth-portal plugin can never reach the oidc endpoint and fails to load. Please feel free to correct me if there's something I'm misunderstanding!
Relevant logs:
{"level":"debug","ts":1628709310.0207937,"logger":"http.handlers.authp","msg":"Configured connection retries for OAuth 2.0 backend","backend_name":"generic_oauth2_backend","retry_attempts":3,"retry_interval":10}
{"level":"debug","ts":1628709310.0268338,"logger":"http.handlers.authp","msg":"fetchMetadataURL failed","backend_name":"generic_oauth2_backend","attempt_id":0,"error":"failed to fetch metadata for OAuth 2.0 authorization server: Get \"$SERVER_NAME/.well-known/openid-configuration\": dial tcp $IP:443: connect: connection refused"}

the proxy never starts

@michaelhthomas , lol! this is new :-) what is that proxy? how is it related to "OIDC server"? Is it another plugin? Please show your Caddyfile and explain the "plumbing."

@michaelhthomas, I will add async options. This way the fetching of keys happens after the provisioning.

@greenpau Awesome, thanks so much!

@michaelhthomas , the change is now in v1.4.21. Please test and report back. You need to add delay_start <seconds> to your config.

@greenpau, Just tested with the new version with a 30 second delay and that worked perfectly, thanks so much for your work!

Just tested with the new version with a 30 second delay and that worked perfectly, thanks so much for your work!

@michaelhthomas , glad that it worked out! 👍
Please contribute to the development of Caddy auth plugins by:

Giving Github stars to github.com/greenpau/caddy-auth-portal and github.com/greenpau/caddy-auth-jwt
Promoting the plugins on Twitter by referencing the above repos and mentioning @caddyserver
Participating in https://caddy.community/
Blogging about your Caddy setup with auth plugins


"Out Of Stock" products still can be added to the cart

Hi,
Product with "Out Of Stock" condition are having buttons like "Buy Now" or "Add To Cart" in home page product listing and in Slider.
would it be better to show "product is out of stock" instead as in view_product.
Thanks

@studuino
This bug if fixed with this commit - https://github.com/kirilkirkov/Ecommerce-CodeIgniter-Bootstrap/commit/0e92c509dc4136779a573853c74848862774d748
I add "Hide buy button if product is shown but is out of stock" in administration Settings to hide it if you want. You must update the code and insert new record in value_store table in database.

Hi
After the code and value_store(db) update,
(1) when "Hide buy button if product is shown but is out of stock" switch is on, there's no buy button in not only Out of stock products but in stock products as well.

(2) With the BUY NOW in slider, out of stock products still can be bought.
(3) almost all the products in product listings disappeared (11 out of 13 in-stock products)
Out of stock products appear correctly.

@studuino
I fixed the slider.

@kirilkirkov
Thanks a lot for the update. We appreciate your quick actions.
One finding is that when products(in-stock or out-of-stock) are in slider, they do not appear in product listing(home page).

@studuino  Yes , this is a desired behavior to avoid duplication.
You can easy change it in  the Public_model query

@kirilkirkov
Thank you very much for the information.
We just found out there's a switch for that.

Works just perfect now.
Cheers.


Refactor packet data conversion in relayer and update IC and NEAR cha…

…in endpoints configuration

Closes: #XXX
Description

PR author checklist:

[ ] Added changelog entry, using unclog.
[ ] Added tests: integration (for Hermes) or unit/mock tests (for modules).
[ ] Linked to GitHub issue.
[ ] Updated code comments and documentation (e.g., docs/).
[ ] Tagged one reviewer who will be the one responsible for shepherding this PR.

Reviewer checklist:

[ ] Reviewed Files changed in the GitHub PR explorer.
[ ] Manually tested (in case integration/unit/mock tests are absent).

relate pr: https://github.com/octopus-network/verification-proxies/pull/10

this test demo can help explain this work: https://github.com/DaviRain-Su/icp-tutorials/tree/main


Support for venv

Hello,
First thanks for sharing trains with the world, it looks very promising.
I'm wondering why trains-agent uses virtualenv and not venv ? I usually use the latter, and when I tried trains-agent I got the following error

/usr/bin/python3.6: No module named virtualenv
trains_agent: ERROR: Command '['python3.6', '-m', 'virtualenv', '/home/florian/.trains/venvs-builds/3.6']' returned non-zero exit status 1.

I didn't know it could be a python module (I even thought it was kind of deprecated), so I then installed it (apt-get install python3-virtualenv) but now even with Python base version being 3.6(agent.default_python = 3.6), it creates a python 2.7 virtualenv:

New python executable in /home/florian/.trains/venvs-builds/3.6/bin/python2
Also creating executable in /home/florian/.trains/venvs-builds/3.6/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2

May be I'm doing something wrong ?
OS: Ubuntu18.04 LTS

Hey @fcalvet ,
First, thank you :)
Then regrading your questions:

venv was added to python on v3.3, since Trains also supports python 2.7 (yes people still use it), we chose virtualenv. Actually venv is dervied from virtualenv, and was merged into python, but since virtualenv is a python package, it is updated more frequently which is important when used as underlying infrastructure for an agent.
The fact that the trains-agent could not locate virtualenv is weird, since it in the trains-agent package requirements.
Could you reinstall and see if the problem consists?
$ pip install -U trains-agent
It seems trains-agent is running your experiment with python2 (which is supported, but  considering your virtualenv is for python 3.6, it seems wrong)
I would think this is some miss-configuration, how about deleting ~/trains.conf and configuring it again with the wizard?
 $ trains-agent init


DoH support

I'm coming back on the need of having DoH support directly in this library.
HTTP(/2) is a different beast and we should probably defer those implementation bits to net/http and friends.
What we do want to support is creating a Get or Post request and than handing it off.
Connection re-use and all kinds of complication could then be left out of this libary as it (mostly) deals with DNS binary conversions.
DNS over TCP (DoT) remains an interesting one; because that requires some of the bits (connection reuse, etc). and is easier to support because it is a simpler protocol.
@tmthrgd wdyt?

@miekg I’m not sure I agree. Everything connection-reuse related is entirely internal to net/http, as is everything HTTP(2). The transports that have complications regarding reuse are DNS-over-TLS and DNS-over-TCP.
In my eyes, the complication with DOH in this library comes entirely down to the existing API, which it doesn’t match up quite perfectly to.
If DOH support were to be removed from this library, I wouldn’t suggest adding any request creation functions in its place. It’s mostly trivial to construct a DOH request.
I think there’s value in having support for a secure transport that seems likely to gain adoption and use. (Granted there is no server support yet).

If DOH support were to be removed from this library, I wouldn’t suggest adding any request creation functions in its place. It’s mostly trivial to construct a DOH request.

I agree with this.
As to "it doesn't fit the API", that's true; but that requires a lot of work and I don't think anyone has time for that.
If starting implementing DoH client support in CoreDNS, and there is nothing that I wanted from this lib (unless it had a perfect implementation).
So, until there is a plan and time to properly implement a Client2, DoH should be dropped. The lack symmetry with the server is also jarring in my mind :)

With #800 merged, this issue seems fixed.

if anyone need doh-client support, you can use my lib doh client and this lib to query dns over https
package main

import (
	"fmt"

	"github.com/miekg/dns"
	"github.com/shynome/doh-client"
)

func main() {
	q := dns.Question{
		Name:   dns.Fqdn("remoon.net"),
		Qtype:  dns.TypeA,
		Qclass: dns.ClassINET,
	}
	m := &dns.Msg{
		MsgHdr:   dns.MsgHdr{Id: dns.Id(), Opcode: dns.OpcodeQuery, RecursionDesired: true},
		Question: []dns.Question{q},
	}
	co := &dns.Conn{Conn: doh.NewConn(nil, nil, "<IP_ADDRESS>")}
	if err := co.WriteMsg(m); err != nil {
		panic(err)
	}
	m, err := co.ReadMsg()
	if err != nil {
		panic(err)
	}
	if len(m.Answer) == 0 {
		panic("answer length must greater than 0")
	}
	fmt.Println(m.Answer)
}

Hello I am fastdns author who come from google search, if anyone is interested in an efficient DoH client, here is an example:
package main

import (
	"context"
	"fmt"
	"net/http"
	"net/url"
	"time"

	"github.com/phuslu/fastdns"
)

func main() {
	doh := "https://<IP_ADDRESS>/dns-query"

	client := &fastdns.Client{
		Addr: doh,
		Dialer: &fastdns.HTTPDialer{
			Endpoint:  func() (u *url.URL) { u, _ = url.Parse(doh); return }(),
			UserAgent: "fastdns/0.9",
			Transport: http.DefaultTransport,
		},
	}

	ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
	defer cancel()

	fmt.Println(client.LookupNetIP(ctx, "ip", "cloud.phus.lu"))
	fmt.Println(client.LookupHTTPS(ctx, "cloud.phus.lu"))
}


FastAPI, raise error 'url_for' is undefined

'url_for' is undefined

not enough information or context; closing


Why are we checking for 20 and 19 specifically?

https://github.com/NextBrain-ml/nbsynthetic/blob/d1cd760013cc563a25cfb5bb073ccc6aa30d29a5/src/nbsynthetic/data_preparation.py#L55

To detect if this data can be a datetime

I get that ... But why just 20 and 19? There should be a better way. Maybe parse the value using a date parser and if doesn't throw an exception, we can say it's a date. But that would be slow. We could check in an array of possible values: 1 - 31 something.