HDDS-11230. Make RocksDB cache configurable in Ozone Manager.

What changes were proposed in this pull request?
Make RocksDB cache configurable in Ozone Manager.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-11230
How was this patch tested?
unit tests

There are some test errors, I will update the code

Hi @weimingdiit thanks for looking in to this. Maybe it would be better to support RocksDB config files directly instead of needing to expose each config through the Ozone configuration? See the RocksDB docs on this for reference. We could first load the options set by the user in this file, and then override any in Ozone that must be set a certain way, for example creating column families that don't exist.

Hi @weimingdiit thanks for looking in to this. Maybe it would be better to support RocksDB config files directly instead of needing to expose each config through the Ozone configuration? See the RocksDB docs on this for reference. We could first load the options set by the user in this file, and then override any in Ozone that must be set a certain way, for example creating column families that don't exist.

Thank you very much for your comments，this is a good idea. However, for the current version, could we first make some of the more performance-critical parameters configurable, such as the ones I mentioned? I will continue to follow up on this issue and create a new issue to directly support RocksDB configuration files, so we don't have to rely on Ozone configuration. What do you think?

@jojochuang  Could you help review this PR? This PR is related to some caching in RocksDB. My current thought is to keep the status quo unchanged in this PR and then create a new issue to make all RocksDB parameters configurable. I'd like to hear your thoughts on this—what do you think?

Hi @weimingdiit thanks for looking in to this. Maybe it would be better to support RocksDB config files directly instead of needing to expose each config through the Ozone configuration? See the RocksDB docs on this for reference. We could first load the options set by the user in this file, and then override any in Ozone that must be set a certain way, for example creating column families that don't exist.

@errose28 Thank you very much for your comments，this is a good idea. However, for the current version, could we first make some of the more performance-critical parameters configurable, such as the ones I mentioned?

I thought about it, maybe it would be more convenient for us to manage it through ozone configuration, so that our configuration will have only one file, which is easier to manage.

I'm ok with adding this field in the Ozone configuration to start if there are no immediate plans to support the full RocksDB INI file. We can file that as a follow up Jira. In the future I do think we should support the INI file instead of propagating every RocksDB config we might need into the Ozone configuration. If we start getting into serious RocksDB tuning then there will be too many options in RocksDB for that approach to be sustainable long term.

I'm ok with adding this field in the Ozone configuration to start if there are no immediate plans to support the full RocksDB INI file. We can file that as a follow up Jira. In the future I do think we should support the INI file instead of propagating every RocksDB config we might need into the Ozone configuration. If we start getting into serious RocksDB tuning then there will be too many options in RocksDB for that approach to be sustainable long term.

I'm ok with adding this field in the Ozone configuration to start if there are no immediate plans to support the full RocksDB INI file. We can file that as a follow up Jira. In the future I do think we should support the INI file instead of propagating every RocksDB config we might need into the Ozone configuration. If we start getting into serious RocksDB tuning then there will be too many options in RocksDB for that approach to be sustainable long term.

@errose28  I will close this PR, and then make all RocksDB parameters configurable in https://github.com/apache/ozone/pull/7150


Feature: Make --graph-output less surprising and more usable

If isatty(sys.stdout), then do not spew a PNG or similar not-so-legible stuff onto the console (I did have a WTF moment), instead write it to a tempfile and either webbrowser.open_new_tab() it, or call xdg-open with it (if that's on the PATH).

I like what this wants to achieve but needs more research how this can work cross platform.


enable running of custom setup scripts at launch time

In some cases it makes sense to want to have some custom script that executes when the container is started. One use-case is to pip install -e install a package that is developed in the project itself (it's hard to do this otherwise because the repo itself is only mounted at runtime).
The suggestion is to have some convention for this script name like runtime-init.sh or something similar.
cc @nperraud

Another example of use cases is to prepare an environment that is user-specific and cannot be generalized to all users for the project - e.g. create site xml files required by certain packages that contain user specific attributes, such as usernames.
In that case it would make sense to let users (or groups) add custom attributes to their profiles, which can be injected in the scripts at start time.

This has been addressed by using a post-init.sh script, but probably isn't documented well enough yet. I will open a new issue about that.


Flatpak Packaging for Linux

New cross-platform standard format.
https://flatpak.org/
http://docs.flatpak.org/

91d7684db7d4a3f5bf55234a9e0ce33a7aadfea4
7c5540c7278d1dbc02975bb8e1b053e7205016d6
8ce4f33c1730725692acb1a617cd6c120cdb69cd
9965d54e908376d8c80ac74f0fa4e4e863e9ebf5
All necessary groundwork

Might need #5 to be done... not sure

Flatpak doesn't like the prebuilt binary for mkxp and mkxp doesn't seem to want to be built on a modern linux distro easily. Giving up on this. AppImages are great, Debian files are great.


hamcrest dependencies are not scoped as test

Hamcrest is used for testing purposes but is not scoped as such. This causes it to be included as a dependency to the cql-engine and cql-engine-fhir.

Resolved on develop


Broken Pipe and Protocol Security Negotiation Failure

System Config
Ubuntu 16.04
Remmina
FreeRDP 2.0.0~dev

[com.freerdp.core.transport] - BIO_should_retry returned a system error 32: broken pipe
[com.freerdp.core.nego] - Protocol Security Negotiation Failure
[com.freerdp.core] - freerdp_set_last_error ERRCONNECT_SECURITY_NEGO_CONNECT_FAILED
[com.freerdp.core.connection] - Error: protocol security negotiation or connection failure
[com.freerdp.client.x11] - Freerdp connect error exit status 1

Which command line / which protocol are you using ? I often get this error when I forget to set +sec-nla.

@jpmelnik what system you are connecting to?

I've forgotten to change my router local ip.
Router: <IP_ADDRESS>
VPN: <IP_ADDRESS>
Thanks you guys.


Add necessary header and update response status check for uploaded artifacts to Azure blob storage

TL;DR
While registering files that is created for fast registration to Azure blob storage, operation fails due to a missing required header. This change adds the missing header and also adds the accepted http response 201(CREATED) which is the default response for successful upload operations.
Type

[x] Bug Fix
[ ] Feature
[ ] Plugin

Are all requirements met?

[x] Code completed
[x] Smoke tested
[ ] Unit tests added
[ ] Code documentation added
[ ] Any pending items have an associated Issue

Complete description
During upload operation the following error message is thrown.
Error: failed to upload source code from [/tmp/flyte/pb_output/fast1ac88c953907bcb2a3ca7a6377277465.tar.gz]. Error: failed uploading to [<blob-location>]. bad status: 400 An HTTP header that's mandatory for this request is not specified.: <?xml version="1.0" encoding="utf-8"?>
<Error>MissingRequiredHeader<Message>An HTTP header that's mandatory for this request is not specified.
RequestId:d7bbcada-101e-0025-4281-ecea10000000
Time:2023-09-21T11:46:16.3019574Z</Message><HeaderName>x-ms-blob-type</HeaderName></Error>
{"json":{},"level":"error","msg":"failed to upload source code from [/tmp/flyte/pb_output/fast1ac88c953907bcb2a3ca7a6377277465.tar.gz]. Error: failed uploading to [<blob-location>]. bad status: 400 An HTTP header that's mandatory for this request is not specified.: \u003c?xml version=\"1.0\" encoding=\"utf-8\"?\u003e\n\u003cError\u003e\u003cCode\u003eMissingRequiredHeader\u003c/Code\u003e\u003cMessage\u003eAn HTTP header that\u0026apos;s mandatory for this request is not specified.\nRequestId:d7bbcada-101e-0025-4281-ecea10000000\nTime:2023-09-21T11:46:16.3019574Z\u003c/Message\u003e\u003cHeaderName\u003ex-ms-blob-type\u003c/HeaderName\u003e\u003c/Error\u003e","ts":"2023-09-21T11:46:16Z"}

In the error message it explains the issue which is a missing header. Added the header to the request and updated response check to accept 201 as a successful http response type.
Tracking Issue
NA
Follow-up issue
NA

It seems there is a dependency issue which causes checks to fail.

Not sure if there is already a pending fix for that. If not I can give it a try :)


Loaded cuDNN version 8400

Bug description
I try to run the demo with python3.9, cuda 11.3, torch 1.10.0+cu113, cudnn version 82003
How can I solve this problem? Thank you in advance.
Code snippet to reproduce the bug
from doctr.models import ocr_predictor
from doctr.io import DocumentFile

model = ocr_predictor(pretrained=True, export_as_straight_boxes=True)
# Image
single_img_doc = DocumentFile.from_images("test.png")
result = model(single_img_doc)
print("done")

Error traceback
C:\Users\rina\AppData\Local\Programs\Python\Python39\lib\site-packages\weasyprint\fonts.py:215: UserWarning: @font-face not supported: FontConfig cannot load default config file
warnings.warn(
C:\Users\rina\AppData\Local\Programs\Python\Python39\lib\site-packages\weasyprint\fonts.py:457: UserWarning: Expect ugly output with font-size: 0
warnings.warn('Expect ugly output with font-size: 0')
C:\Users\rina\AppData\Local\Programs\Python\Python39\lib\site-packages\weasyprint\document.py:35: UserWarning: There are known rendering problems and missing features with cairo < 1.15.4. WeasyPrint may work with older versions, but please read the note about the needed cairo version on the "Install" page of the documentation before reporting bugs. http://weasyprint.readthedocs.io/en/latest/install.html
warnings.warn(
2022-05-10 13:39:25.033842: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-05-10 13:39:25.461588: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 3969 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 2060, pci bus id: 0000:01:00.0, compute capability: 7.5
DEBUG:tensorflow:Layer lstm will use cuDNN kernels when running on GPU.
DEBUG:tensorflow:Layer lstm will use cuDNN kernels when running on GPU.
DEBUG:tensorflow:Layer lstm will use cuDNN kernels when running on GPU.
DEBUG:tensorflow:Layer lstm_1 will use cuDNN kernels when running on GPU.
DEBUG:tensorflow:Layer lstm_1 will use cuDNN kernels when running on GPU.
DEBUG:tensorflow:Layer lstm_1 will use cuDNN kernels when running on GPU.
2022-05-10 13:39:29.226392: I tensorflow/stream_executor/cuda/cuda_dnn.cc:368] Loaded cuDNN version 8400
Process finished with exit code -1073740791 (0xC0000409)
Environment
python3.9, cuda 11.3, torch 1.10.0+cu113, cudnn version 82003
Deep Learning backend
both True

Hi @skywalkerzhang 👋,
Could you please provide the output from: https://github.com/mindee/doctr/blob/main/scripts/collect_env.py ? If you have both frameworks in the same env installed and want to use PyTorch than you need to set a environment variable : For PT: export USE_TORCH=1 For TF: export USE_TF=1
Another think is, that we currently not test with a python 3.9 environment so would it be possible that you test it with 3.8 ? :)

DocTR version: 0.5.1
TensorFlow version: 2.8.0
PyTorch version: 1.10.0+cu113 (torchvision 0.11.1+cu113)
OpenCV version: 4.5.4
OS: Microsoft Windows 10 רҵ▒▒
Python version: 3.9.7
Is CUDA available (TensorFlow): Yes
Is CUDA available (PyTorch): Yes
CUDA runtime version: 11.6.124
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\bin\cudnn_ops_train64_8.dll
Fontconfig error: Cannot load default config file

Could you please check your Nvidia driver version ? https://nvidia.custhelp.com/app/answers/detail/a_id/2039/~/how-do-i-determine-which-nvidia-display-driver-version-is-currently-installed
And test if it's work with setting the environment variable USE_TORCH=1   ?
Otherwise i think it is a problem with the combination (Windows, python 3.9 and especially weasyprint) @frgfm
It is a bit hard for me to debug this without an windows environment so i need your support @skywalkerzhang  :sweat:

Thanks for your suggestion @felixdittrich92 . I tested with USE_TORCH=1 and it downloads pt files successfully. After that, another error comes up Fontconfig error: Cannot load default config file.
 WeasyPrint may work with older versions, but please read the note about the needed cairo version on the "Install" page of the documentation before reporting bugs. http://weasyprint.readthedocs.io/en/latest/install.html
  warnings.warn(
Downloading https://github.com/mindee/doctr/releases/download/v0.3.1/db_resnet50-ac60cadc.pt to C:\Users\rina\.cache\doctr\models\db_resnet50-ac60cadc.pt
101971968it [01:11, 1435110.95it/s]
Downloading https://github.com/mindee/doctr/releases/download/v0.3.1/crnn_vgg16_bn-9762b0b0.pt to C:\Users\rina\.cache\doctr\models\crnn_vgg16_bn-9762b0b0.pt
63287296it [01:44, 604988.32it/s]
done
Fontconfig error: Cannot load default config file

I don't know if is caused by weasyprint. I found this documentation WeasyPrint DOC and it says only win11 is supported but I use win10 this moment.

Hi all 👋
The last problem mentioned here is indeed linked to weasyprint, but the issue is reporting a problem about TensorFlow failing at loading cuDNN. So about this current issue, I'd suggest to simply reproduce this with TensorFlow and potentially reinstall this :)
Your error shows that the problem comes from much deeper than docTR, the framework is trying to load GPU-related hardware acceleration libraries and it fails. So this might indicate, as suggested by Felix, that you're gonna need to reinstall cuda!

@skywalkerzhang any update or can we close this ? :)


Led order is not correct

Currently there is no way (I think so) to change the led order. The led order of a led strip could be RGB, GRB, BRG etc.
I just faced this issue, the code was working perfectly but my new strip is not RGB, its BRG so the colors are not correct. If there is a way to change the order, let me know!
Thanks.

In this file, you can see there are all the different types of LED strip.
https://www.github.com/pimoroni/rpi_ws281x/tree/master/ws2811.h
You can import these and pass them as parameters to the PixelStrip class.
Example
https://github.com/cp2004/OctoPrint-WS281x_LED_Status/blob/877869973167acd6cd413f6ff45a4d57f4ce4df3/octoprint_ws281x_led_status/constants.py#L48-L63


Build failed when running react-native run-android

When i run react-native run-android i get this error:

FAILURE: Build failed with an exception.

Where:
Build file '/lc_app/node_modules/react-native-linear-gradient/android/build.gradle' line: 21

What went wrong:
A problem occurred evaluating project ':react-native-linear-gradient'.
Could not find method compileOnly() for arguments [com.facebook.react:react-native:+] on object of type org.gradle.api.internal.artifacts.dsl.dependencies.DefaultDependencyHandler. <

What should i do??
I did the install and the link.

@stacheski I had same issue and solved issue by opening this project in Android studio and upgraded the gradle version.

RN Version : 0.56.0
react-native-linear-gradient version : 2.4.2

I have the same problem.
react-native: 0.55.4
react-native-linear-gradient:2.4.2

I modify the node_modules/react-native-linear-gradient/android/build.gradle,solved it.
dependencies {
    compileOnly "com.facebook.react:react-native:+"
}

replace
dependencies {
    provided "com.facebook.react:react-native:+"
}

This commit changes over to the newer compileOnly which may break your build if your version of gradle is too old (pre 3.5 ??). If you are unable to upgrade your gradle version at this time, I think version 2.4.0 of react-native-linear-gradient is the highest you can go for now.

Same issue for me:
React-native: "0.56"
react-native-liniear-gradient: "^2.4.0"
Fixed by changing version of react-native-liniear-gradient in packages.json to "2.4.0"

Using the 2.4.0 version worked.
Thank you guys!

React-native: "0.55.4"
react-native-liniear-gradient: "^2.4.0"
That’s ♂Good！！

@ryanhomer thanks for the explanation

It`s work for me too
"react-native": "0.55.4",
"react-native-linear-gradient": "^2.4.0",
try to :
npm install<EMAIL_ADDRESS>--save

i have solved this problem with updating gradle at version 3.0.1


Support Linux

Hi, thx so much for this effort. I dont know how to program but would love to help in other ways (testing etc) since this could be really cool and useful
best
Z

That sounds awesome @zeltak ! As soon as we can get it working, we’ll need to test this out in Linux. Also, the README could always use some work. It’s really cool of you to offer to help! 👍

Hi! This seems like a really worthwhile project. I use Emacs daily and using it everywhere is something I would want. Do you have anyone working on a Linux version (I run Ubuntu 16.04) or any ideas on where to get started? I'm really interested in getting going on this. I think a first thing to consider is if the different desktop environments need to be handled differently from one another.

Hi @DavidLarsson ! It's awesome to hear you are optimistic about this project's utility!
I'm going to push a branch to get us started with this sometime today. Feel free to contribute and discuss as the work unfolds.
Right now, there is just a dmg and little script I wrote (hacked) just to make this thing work for me in OSX. We need to write bash script to check the OS it's running in and install accordingly.  The service for OSX already exists, it just needs to be separated from the Elisp, so that will be a breeze.
Most of the work will be implementing the app refocus and paste functionality in Linux, which is a problem plenty before us have solved before. And I think it would be nice to set a default keyboard shortcut for both OSX and Linux.
Something to keep in mind while we do this is that we will want to package this thing up (soon) for Homebrew, if not other package managers further down the road.

@zeltak, @DavidLarsson could you test #9 on Linux if you get a chance?

Hi @zachcurry
i tested on arch linux (dont have an ubuntu machine but can test mondat at work).
it installs well and the keybind pops up the emacs frame.
yet i cant seem to get the text to be pasted in the previous source window. i tried both M-x delete-frame and just closing with no success
happy to keep debugging!
Z


⚠ AGEG website (redirect) has degraded performance

In cbb837d, AGEG website (redirect) (https://ageg.ca/) experienced degraded performance:

HTTP code: 200
Response time: 2567 ms

Resolved: AGEG website (redirect) performance has improved in bb302ed.


geography barely mentioned

the Geography gets a scant mention before Geometry and all 4 examples A-D are of Geometry.
Should mention the useful Well-Known Text (WKT) and float functions, e.g.
declare	@pt1 geography = geography::STGeomFromText('POINT(50.4750671 -1.5165396)', 4326),
@pt2 geography = geography::Point(1.293561, 51.868143, 4326)
select	@pt1.STDistance(@pt2) as A, @pt2.STDistance(@pt1) as B

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 359226f6-b858-ffab-59ac-474e43381fd3
Version Independent ID: 0b8aaaa0-a09a-c72d-eb08-963d631aaec9
Content: Point - SQL Server
Content Source: docs/relational-databases/spatial/point.md
Product: sql
Technology: ****
GitHub Login: @MladjoA
Microsoft Alias: mlandzic

add another (sorry I missed first entry)
declare @pt3 geography = geography::STPointFromText('POINT(50.4750671 -1.5165396)', 4326)

@DickBaker -- Dick, thank you for your feedback.
@MladjoA -- Mladen, please look into this issue.

@MladjoA Mladen, could you look at this suggestion? Should we add more of these geography examples now or track this for a future improvement. @DickBaker thank you for the suggestion and apologies for the delay.

@MladjoA Mladen, could you look at this suggestion? Should we add more of these geography examples now or track this for a future improvement. @DickBaker thank you for the suggestion and apologies for the delay.

@DickBaker We're revisiting older open issues, and I wanted to ask if you'd be interested in opening a pull request that includes the geography examples you mentioned. Even if it isn't perfect, it'll get this enhancement ahead in the queue.

@rwestMSFT thanks Randolph I'll pass as think ano likely more proficient than me on GIS etc

Closing this issue and adding it to our backlog.


Convert all paths to relative or absolute

It'll be really handy to export all the paths using relative commands (or absolute).
Android doesn't know how to morph these two paths:
M10 10  L10 20  L20 10  Z
M10 10  l10  0  l 0 10  z

Android fails at runtime. Do something like this would be really aprecieated: http://stackoverflow.com/a/23716756/842697

I suggest you use https://shapeshifter.design to make the paths morphable with each other.
We have plans to at some point merge the features of AndroidIconAnimator and ShapeShifter into one. That should solve this problem.

That would be great! Because there are problems morphing between c and s too.

Shape Shifter solves that too. Android Icon Animator pretty much assumes the paths you enter are compatible right now.


Split up locked_updateCoverImage: into updating the cover image and calling the delegate

There is currently a deadlock in the interaction between ASImageNode+AnimatedImage startAnimating and PINCacheAnimatedImage l_cacheFrame as we are calling the coverImageReadyCallback with the lock held.
Example scenario:
Thread 0:

ASDisplayNode is calling startAnimating ([ASImageNode+AnimatedImage startAnimating] and grabs the ASDisplayNode lock
It calls  [ASDisplayNode _locked_startAnimating]
[ASDisplayNode _locked_startAnimating] calls [PINCacheAnimatedImage playbackReady] where it grabs the PINCacheAnimatedImage lock

Thread x:

Grabs the PINCacheAnimatedImage lock in l_cacheFrame

Start in [PINCacheAnimatedImage l_cacheFrame]
We are calling [PINCacheAnimatedImage l_updateCoverImage], which calls the _coverImageReadyCallback.
The _coverImageReadyCallback is set within the ASImageNode+AnimatedImage and calls setCoverImageCompleted which tries to grab the lock via self.interfaceState that is already hold within Thread 0

          1 Warning

:warning:
Any source code changes should have an entry in CHANGELOG.md or have #trivial in their title.

  Generated by :no_entry_sign: Danger

🚫 CI failed with log

          1 Warning

:warning:
Any source code changes should have an entry in CHANGELOG.md or have #trivial in their title.

  Generated by :no_entry_sign: Danger


Filter from the Audit Logs when banned from the Context Menu

Title should be self explanatory, if it's not. Here is a gist:
Basically, if you ban, you'll get 2 case logs instead of one, we should filter the logs if the banned user is that user and is from the moderator

Issue was fixed in latest commit in Edge branch


unable to find an inherited method for function ‘featureDefinitions’ for signature ‘"xcmsSet"’

Hi,
we tried to perform RAMClustR by importing the xcms preprocessed data as provided in your vignette. However, we couldn't proceed further due to the following error:
Error in (function (classes, fdef, mtable)  : unable to find an inherited method for function ‘featureDefinitions’ for signature ‘"xcmsSet"’
Note: we also got the same error on trying with the dataset from "faahKO" library that had been demonstrated in the RAMClustR vignette.
The session info of the same is given below for your reference:

sessionInfo()
R version 4.3.1 (2023-06-16 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default
locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252
time zone: America/New_York
tzcode source: internal
attached base packages:
[1] stats4    stats     graphics  grDevices utils     datasets  methods   base
other attached packages:
[1] devtools_2.4.5              usethis_2.2.2               RAMClustR_1.3.1             SummarizedExperiment_1.32.0
[5] GenomicRanges_1.54.1        GenomeInfoDb_1.38.2         IRanges_2.36.0              MatrixGenerics_1.14.0
[9] matrixStats_1.2.0           MsExperiment_1.4.0          pheatmap_1.0.12             pander_0.6.5
[13] RColorBrewer_1.1-3          faahKO_1.40.0               xcms_4.0.1                  MSnbase_2.28.1
[17] ProtGenerics_1.34.0         S4Vectors_0.40.2            mzR_2.36.0                  Rcpp_1.0.11
[21] Biobase_2.62.0              BiocGenerics_0.48.1         BiocParallel_1.36.0
loaded via a namespace (and not attached):
[1] rstudioapi_0.15.0           MultiAssayExperiment_1.28.0 magrittr_2.0.3              farver_2.1.1
[5] MALDIquant_1.22.1           fs_1.6.3                    zlibbioc_1.48.0             vctrs_0.6.4
[9] multtest_2.58.0             memoise_2.0.1               RCurl_1.98-1.13             MultiDataSet_1.28.0
[13] htmltools_0.5.7             S4Arrays_1.2.0              progress_1.2.3              curl_5.2.0
[17] dynamicTreeCut_1.63-1       SparseArray_1.2.2           mzID_1.40.0                 desc_1.4.3
[21] htmlwidgets_1.6.4           plyr_1.8.9                  impute_1.76.0               cachem_1.0.8
[25] qqman_0.1.9                 igraph_1.6.0                mime_0.12                   lifecycle_1.0.4
[29] iterators_1.0.14            ropls_1.32.0                pkgconfig_2.0.3             Matrix_1.6-1.1
[33] R6_2.5.1                    fastmap_1.1.1               GenomeInfoDbData_1.2.11     shiny_<IP_ADDRESS>
[37] clue_0.3-65                 digest_0.6.33               pcaMethods_1.94.0           colorspace_2.1-0
[41] ps_1.7.5                    pkgload_1.3.3               Spectra_1.12.0              fansi_1.0.6
[45] abind_1.4-5                 compiler_4.3.1              remotes_<IP_ADDRESS>             proxy_0.4-27
[49] doParallel_1.0.17           pkgbuild_1.4.3              MASS_7.3-60                 sessioninfo_1.2.2
[53] DelayedArray_0.28.0         tools_4.3.1                 httpuv_1.6.13               glue_1.6.2
[57] callr_3.7.3                 promises_1.2.1              QFeatures_1.12.0            grid_4.3.1
[61] cluster_2.1.4               generics_0.1.3              snow_0.4-4                  gtable_0.3.4
[65] class_7.3-22                preprocessCore_1.64.0       hms_1.1.3                   MetaboCoreUtils_1.10.0
[69] utf8_1.2.4                  XVector_0.42.0              stringr_1.5.1               RANN_2.6.1
[73] foreach_1.5.2               pillar_1.9.0                limma_3.58.1                later_1.3.2
[77] robustbase_0.99-1           splines_4.3.1               dplyr_1.1.4                 lattice_0.21-9
[81] survival_3.5-7              tidyselect_1.2.0            miniUI_<IP_ADDRESS>              statmod_1.5.0
[85] DEoptimR_1.1-3              stringi_1.7.12              lazyeval_0.2.2              codetools_0.2-19
[89] MsCoreUtils_1.14.1          tibble_3.2.1                BiocManager_1.30.22         cli_3.6.1
[93] affyio_1.72.0               xtable_1.8-4                processx_3.8.3              munsell_0.5.0
[97] MassSpecWavelet_1.68.0      XML_3.99-0.16               fastcluster_1.2.3           parallel_4.3.1
[101] ellipsis_0.3.2              ggplot2_3.4.4               prettyunits_1.2.0           profvis_0.3.8
[105] calibrate_1.7.7             urlchecker_1.0.1            AnnotationFilter_1.26.0     bitops_1.0-7
[109] MsFeatures_1.10.0           scales_1.3.0                affy_1.80.0                 e1071_1.7-14
[113] ncdf4_1.22                  purrr_1.0.2                 crayon_1.5.2                rlang_1.1.1
[117] vsn_3.70.0

Hi, thanks for reporting. To help us, it is always best to provide a copy&pasteable code snippet so we can reproduce things. Another easy thing to try after an error occurs is to run traceback() which gives the context where the error occurred.
In this case you seem to have hit an issue that featureDefinitions() is a method expecting an MsFeatureData, where you supplied an xcmsSet which is the old, deprecated XCMS data structure.
Yours, Steffen

Hi, we tried to replicate the code in RAMClustR vignette. However, I providing the code below for your immediate reference:
library(BiocManager)
library(xcms)
library(faahKO)
library(devtools)
library(RAMClustR)
cdfpath <- system.file("cdf", package = "faahKO")
cdffiles <- list.files(cdfpath, recursive = TRUE, full.names = TRUE)
xset <- xcmsSet(cdffiles)
xset <- group(xset)
xset <- retcor(xset, family = "symmetric", plottype = NULL)
xset <- group(xset, bw = 10)
xset <- fillPeaks(xset)
experiment <- defineExperiment(csv = TRUE) # experiment <- defineExperiment(force.skip = TRUE)
RC <- ramclustR(xcmsObj = xset, ExpDes=experiment)
We got the error in the last step stating "Error in (function (classes, fdef, mtable) : unable to find an inherited method for function ‘featureDefinitions’ for signature "xcmsSet"".
The traceback of error is provided below:
6: stop(gettextf("unable to find an inherited method for function %s for signature %s",
sQuote(fdef@generic), sQuote(cnames)), domain = NA)
5: (function (classes, fdef, mtable)
{
methods <- .findInheritedMethods(classes, fdef, mtable)
if (length(methods) == 1L)
return(methods[[1L]])
else if (length(methods) == 0L) {
cnames <- paste0(""", vapply(classes, as.character,
""), """, collapse = ", ")
stop(gettextf("unable to find an inherited method for function %s for signature %s",
sQuote(fdef@generic), sQuote(cnames)), domain = NA)
}
else stop("Internal error in finding inherited methods; didn't return a unique method",
domain = NA)
})(list(structure("xcmsSet", package = "xcms")), new("standardGeneric",
.Data = function (object, ...)
standardGeneric("featureDefinitions"), generic = structure("featureDefinitions", package = "xcms"),
package = "xcms", group = list(), valueClass = character(0),
signature = "object", default = NULL, skeleton = (function (object,
...)
stop(gettextf("invalid call in method dispatch to '%s' (no default method)",
...
4: xcms::featureDefinitions(xcmsObj)
3: rownames(xcms::featureDefinitions(xcmsObj))
2: rc.get.xcms.data(xcmsObj = xcmsObj, taglocation = taglocation,
MStag = MStag, MSMStag = idMSMStag, ExpDes = ExpDes, mzdec = mzdec,
ensure.no.na = replace.zeros)
1: ramclustR(xcmsObj = xset, ExpDes = experiment)
Hence, we are confused how the xcmsSet object is handled in the RAMClustR vignette tutorial? Is that mean that the vignette of RAMClustR has to be updated?
Thanks,
Priyanka


Fix Rate and Monitoring

This PR Attempts to fix #508 and #478. They're fairly related so I've done them in one PR.
Using -P for sending multiple packets to an IP currently does not respect the rate functionality. This is an issue because specifying a rate of 250k packets per second and a -P of 10 results in a true send rate of 2.5M packets per second. This PR addresses that by making the packet stream for-loop encapsulate the timing adjustments, sending of the packet, exiting, and monitoring adjustments. The for loop and the goto cleanup statements are the only changes I made to address that.
You can confirm this change by doing zmap -p 80 -r 1 -P 10  which will send 10 packets to that IP address one second apart.
The other changes are in regards to the monitor. state.sent and state.tried_sent were moved into the above mentioned for loop to sum packets sent, not hosts targeted. I've also changed these values to be uint64 type as the number of packets could easily be greater than 2^32. This mostly fixed the monitoring issue but time was still off, multiplying max_targets and max_index both by packet streams did the trick. I've also changed send_rate and recv_rate to use ceil since the current monitor outputs those rates as pseudo-integers (floats with 0 decimal digits).
There are three questions I have regarding this PR:

Is Packet Streams the way we want to describe sending multiple packets to a single host? To me, the term packet streams implies sending packets in parallel from multiple NIC's, which is not what is being done.
What should Recv in monitor tally? Currently, it tallies the amount of unique successful packets, which may not be the users chosen output filter. IMO, recv should reflect the output filter and I've made that change in this PR. Line 168
I'm not positive, but there could be other cases where switching the tallying to packets instead of hosts will overflow the uint32 we have, if you want to switch to tallying packets I'd be happy to find these other cases and fix them up in this PR.

Please let me know if there's any issues or adjustments I can make!

Sorry for the delay on this, I can try to map these out on Thursday or
Friday.
On Tue, Jul 17, 2018 at 8:46 PM Alex Holland<EMAIL_ADDRESS>wrote:

@ajholland commented on this pull request.
In src/monitor.c
https://github.com/zmap/zmap/pull/513#discussion_r203230580:

@@ -162,7 +165,7 @@ static void export_stats(int_status_t *intrnl, export_status_t *exp,
uint32_t total_tried_sent = iterator_get_tried_sent(it);
uint32_t total_fail = iterator_get_fail(it);
uint32_t total_recv = zrecv.pcap_recv;

uint32_t recv_success = zrecv.success_unique;

uint32_t recv_success = zrecv.filter_success;

Pinging to see what progress there is on this.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/zmap/zmap/pull/513#discussion_r203230580, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAMSUAKH6iCGBMHyznC1kSGLMqhrQj3Kks5uHpNhgaJpZM4TzdMj
.

Alrighty, I've tried to map this out here: https://docs.google.com/spreadsheets/d/1vG4InRqNRF5HmwLXljceFVeZoPL8lHVJYbrTfBvShBE/edit?usp=sharing. @ajholland let me know if this seems sane.

@zakird Looks good to me. I can make those changes in a separate PR?
The only blocker on this PR is deciding whether the monitor should output response success as filter success. (If so, there's a couple overflows I will fix).

I'm cool with a different PR. I don't think it matters too much what we decide here if we're going to immediately replace.

These stats are all tracked in odd places. I'm going to take a stab at moving some of this stuff around, but in the mean time I've updated this PR enough to accurately fix #508 and #478, without breaking time estimation. We now output "packets_sent", and "hosts_scanned". I don't currently output total_tried_send, since that information is no longer accessible in the global state object, and we'd need to change where that is tracked for multiple ports support, anyway.


Update set-up-an-archive-and-deletion-policy-for-mailboxes.md

Fixed typo

Issue#https://github.com/MicrosoftDocs/microsoft-365-docs/issues/4318

@JohanFreelancer9 please copy review

@markjjo Please merge this PR. Thanks


Updated translations from Transifex.

Trac ticket number

ticket-18488
Branch description
Added a translation for the error message "Item %(nth)s in the array did not validate" into the Turkmen language: "Massiwyň %(nth)s elementi tassyklanmady."
Checklist

[ ] This PR targets the main branch.
[ ] The commit message is written in past tense, mentions the ticket number, and ends with a period.
[ ] I have checked the "Has patch" ticket flag in the Trac system.
[ ] I have added or updated relevant tests.
[x] I have added or updated relevant docs, including release notes if applicable.
[ ] I have attached screenshots in both light and dark modes for any UI changes.

Hello @RobertGoodman08! Thank you for this work, however translations are manager through Transifex (https://docs.djangoproject.com/en/5.1/internals/contributing/localizing/).
So please fix these strings there. We generally fetch translations for the .2 or .3 point release of Django.


[WIP] Move NaN/Inf detection to a separate utilities file

What does this PR do?

Fixes #6815
Before submitting

[ ] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[ ] Did you read the contributor guideline, Pull Request section?
[ ] Did you make sure your PR does only one thing, instead of bundling different changes together?
[ ] Did you make sure to update the documentation with your changes? (if necessary)
[ ] Did you write any new necessary tests? (not for typos and docs)
[ ] Did you verify new and existing tests pass locally with your changes?
[ ] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review
Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

[ ] Is this pull request ready for review? (if not, please submit in draft mode)
[ ] Check that all items from Before submitting are resolved
[ ] Make sure the title is self-explanatory and the description concisely explains the PR
[ ] Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?
Make sure you had fun coding 🙃

Hello @ananthsub! Thanks for opening this PR.

In the file pytorch_lightning/trainer/training_tricks.py:

Line 37:121: E501 line too long (121 > 120 characters)

Do see the Hitchhiker's guide to code style


Production Environment

Overview
We should start thinking about the prod env where the server will run to prevent snags ahead of time if we can
Questions

How to handle installing foundry in the production server environment?
Which PaaS to use for hosting the express server?

https://railway.app/
https://www.heroku.com/

BG is running some other stuff on Railway so I will ask around to see if it is preferred.

@escottalexander fwiw I'm a big fan of Railway. Been using it to host express servers and psql dbs for years now. I find the UI/UX and docs to be much better than heroku. They also make it real ez to monitor usage and set spending limits incase some bad code goes sideways on you :sweat_smile:

Set up running here just for testing: https://eth-tech-tree-backend-production.up.railway.app/


Log flood on insecure permissions paralyzes system

Hi!
After mistakenly set permissions all over the system (please don't ask :) ) cups flooded root partition with it's logs (pics below) complaining about loose permissions of files under /usr/lib/cups.
System: Ubuntu 16.04.6.
Please add timeout maybe between attempts of opening those files or mechanism to avoid multiple messages and notify user in explicit way.

Ubuntu problem with default log size unlimited...

I have the same problem. I've set permissions all over the system too by mistake and now I have almost the same error. Please help me to solve this. How to stop cups from writing those messages? Cups service uses 100% of one cpu core just to write a tons of error messages.

@jebbyk Restore the permissions (chmod 755 /usr/lib/cups/notifier)

@michaelrsweet I've tried this but it doesn't helped me. So I've purged cups with manually deleting all the related folders (at least which I found) and installed it again. After this it looks like it works normally again


How get form parameters?

Your file upload example creates a form with an input, how can I get the value of file1 element?
For example:
 if (ctx.request.url == "/upload" && ctx.request.method == Method.POST) {
  // ctx.request.form["file1"].....

In that example, I specified the encoding type as multipart/form-data, which means that that's how the post data is formatted. Now, handy-httpd is quite minimalistic in that it doesn't provide parsers for various encoding types, but for multipart/form-data, this is what actually gets uploaded if I upload a file test.txt with the contents Hello world!\n:
-----------------------------21990287138527457151671821359
Content-Disposition: form-data; name="file1"; filename="test.txt"
Content-Type: text/plain

Hello world!

-----------------------------21990287138527457151671821359--

See here for more info about enctype: https://developer.mozilla.org/en-US/docs/Web/API/HTMLFormElement/enctype
See this StackOverflow post on the topic: https://stackoverflow.com/questions/4238809/example-of-multipart-form-data
I am actually working behind the scenes on refactoring the underlying IO layer for handy-httpd, such that it's easier to add various request body parsers (multipart, url-encoded, json, yaml, etc.), but that's still in the works.

I'm going to close this issue, but reopen if you have any other questions.

Thank you very much Andrew!


Fix benchmarks by block Pympler==1.1 release

References and relevant issues
Description
It looks like our benchmarks are failing because of the new pympler release (https://pypi.org/project/Pympler/1.1/). This is a fast fix, but I think that we should handle it somehow using constraints.

@Czaki maybe don't call the constraint pympler but benchmarks or asv?


feat(s2n-quic-rustls): update rustls from 0.21 to 0.23

This PR reverts the revert PR to upgrade rustls:

Original PR: (#2143)
Revert PR: (#2174)

Description of changes:
One noteworthy change: rustls now has a concept of crypto providers (ring or aws-lc-rs) and by default uses aws-lc-rs. For this reason I choose to use aws-lc-rs also, which is a change in behavior since previously we used ring. In the future we could expose the option for s2n-quic customers to choose their own crypto provider.
Call-outs:
I removed some deprecated exports because either:

they no longer exist (::rustls::{Certificate, PrivateKey})
they are not useful (pub static DEFAULT_CIPHERSUITES: &[rustls::SupportedCipherSuite])

Testing:
Existing unit tests and examples pass.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Since shipping this should be a breaking change, I think it can go a bit further that we should respect the CryptoProvider::get_default(), since that relying on the end user to do the initializing work, i.e. aws_lc_rs::default_provider().install_default() which will be another breaking change.


[SPARK-7575] Example code for OneVsRest

Java and Scala examples for OneVsRest. Fixes the base classifier to be Logistic Regression and accepts the configuration parameters of the base classifier.

Merged build triggered.

Merged build started.

Test build #32599 has started for   PR 6115 at commit 8cfc5d5.

Merged build triggered.

Merged build started.

Test build #32600 has started for   PR 6115 at commit 6adfa0c.

Merged build triggered.

Merged build started.

Test build #32602 has started for   PR 6115 at commit 196a59a.

Test build #32600 has finished for   PR 6115 at commit 6adfa0c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaOneVsRestExample
   static class Params
  case class Params(input: String = null,
      head("OneVsRest Example: multiclass to binary reduction using OneVsRest")

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32600/
Test FAILed.

Test build #32599 has finished for   PR 6115 at commit 8cfc5d5.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaOneVsRestExample
   static class Params
  case class Params(input: String = null,
      head("OneVsRest Example: multiclass to binary reduction using OneVsRest")

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32599/
Test PASSed.

Merged build triggered.

Merged build started.

Test build #32606 has started for   PR 6115 at commit 5323ff9.

Test build #32602 has finished for   PR 6115 at commit 196a59a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaOneVsRestExample
   static class Params
  case class Params(input: String = null,
      head("OneVsRest Example: multiclass to binary reduction using OneVsRest")

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32602/
Test FAILed.

Merged build finished. Test FAILed.

Merged build triggered.

Merged build started.

Test build #32609 has started for   PR 6115 at commit 935cf56.

@jkbradley , can you please review the example code? I have tested them out using bin/spark-submit locally
I'm starting work on documentation on a separate PR, but for now you can use the following website for testing the examples if you'd like.
http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/multiclass.html
(Not all datasets in this collection are labeled 0- (numClasses -1), but mnist as well as connect-4 should be.)
====== Sample Submission for Java =========================
bin/spark-submit --class org.apache.spark.examples.ml.JavaOneVsRestExample --driver-memory 4g examples/target/spark-examples_2.10-1.4.0-SNAPSHOT.jar ~/Downloads/mnist.bz2
=======Sample Submission for Scala ========================
bin/spark-submit --class org.apache.spark.examples.ml.OneVsRestExample --driver-memory 4g examples/target/spark-examples_2.10-1.4.0-SNAPSHOT.jar ~/Downloads/mnist.bz2 --fracTest 0.3 --regParam 0.01

Test build #32606 has finished for   PR 6115 at commit 5323ff9.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaOneVsRestExample
      head("OneVsRest Example: multiclass to binary reduction using OneVsRest")

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32606/
Test PASSed.

Merged build finished. Test PASSed.

Test build #32609 has finished for   PR 6115 at commit 935cf56.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaOneVsRestExample
      head("OneVsRest Example: multiclass to binary reduction using OneVsRest")

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32609/
Test PASSed.

I'll check it out ASAP.  Btw, could you please tag the PR title with "[ml] [doc]"?

Sure will do shortly and update PR, thx
Sent from my iPhone

On May 13, 2015, at 10:48 AM, jkbradley<EMAIL_ADDRESS>wrote:
I'll check it out ASAP. Btw, could you please tag the PR title with "[ml] [doc]"?
—
Reply to this email directly or view it on GitHub.

Merged build triggered.

Merged build started.

Test build #32632 has started for   PR 6115 at commit e0106d9.

Merged build triggered.

Merged build started.

Test build #32636 has started for   PR 6115 at commit 461eb38.

Test build #32632 has finished for   PR 6115 at commit e0106d9.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaOneVsRestExample
      head("OneVsRest Example: multiclass to binary reduction using OneVsRest")

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32632/
Test PASSed.

Test build #32636 has finished for   PR 6115 at commit 461eb38.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaOneVsRestExample
      head("OneVsRest Example: multiclass to binary reduction using OneVsRest")

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32636/
Test PASSed.

Merged build finished. Test PASSed.

@harsha2010 Thanks for the PR!  I like the examples and basically just had small/formatting comments.

Merged build triggered.

Merged build started.

Test build #32670 has started for   PR 6115 at commit d4c40d0.

Test build #32670 has finished for   PR 6115 at commit d4c40d0.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaOneVsRestExample
      head("OneVsRest Example: multiclass to binary reduction using OneVsRest")

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32670/
Test PASSed.

Merged build triggered.

Merged build started.

Test build #32679 has started for   PR 6115 at commit 7f4e127.

Merged build triggered.

Merged build started.

Test build #32682 has started for   PR 6115 at commit cb23995.

Merged build triggered.

Merged build started.

Test build #32687 has started for   PR 6115 at commit 8703e4f.

Test build #32679 has finished for   PR 6115 at commit 7f4e127.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaOneVsRestExample
      head("OneVsRest Example: multiclass to binary reduction using OneVsRest")

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32679/
Test PASSed.

Test build #32682 has finished for   PR 6115 at commit cb23995.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaOneVsRestExample
      head("OneVsRest Example: multiclass to binary reduction using OneVsRest")

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32682/
Test FAILed.

Merged build finished. Test FAILed.

Test build #32687 has finished for   PR 6115 at commit 8703e4f.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaOneVsRestExample
      head("OneVsRest Example: multiclass to binary reduction using OneVsRest")

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32687/
Test PASSed.

Merged build finished. Test PASSed.

Jenkins, retest this please.

@harsha2010  Those new comments are after trying out the examples on data.
One more small item on formatting results: It would be nice to have extra newlines separating the different result types: timing, confusion matrix, and FPR.
That's it though.  The examples are nice!

@jkbradley thanks for the feedback. i have refactored the examples based on feedback. also thanks for pointing out the issue with mnist test data, i copied your solution from the decision tree example.

Merged build triggered.

Merged build started.

Test build #32765 has started for   PR 6115 at commit dfd660c.

Merged build triggered.

Merged build started.

Test build #32766 has started for   PR 6115 at commit 7076084.

Test build #32766 has finished for   PR 6115 at commit 7076084.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaOneVsRestExample
      head("OneVsRest Example: multiclass to binary reduction using OneVsRest")

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32766/
Test FAILed.

Merged build triggered.

Merged build started.

Test build #32770 has started for   PR 6115 at commit f5d9891.

Merged build triggered.

Merged build started.

Test build #32774 has started for   PR 6115 at commit 87ad3c7.

Test build #32765 has finished for   PR 6115 at commit dfd660c.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaOneVsRestExample
      head("OneVsRest Example: multiclass to binary reduction using OneVsRest")

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32765/
Test PASSed.

Test build #32770 has finished for   PR 6115 at commit f5d9891.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaOneVsRestExample
      head("OneVsRest Example: multiclass to binary reduction using OneVsRest")

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32770/
Test PASSed.

Test build #32774 has finished for   PR 6115 at commit 87ad3c7.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaOneVsRestExample
      head("OneVsRest Example: multiclass to binary reduction using OneVsRest")

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32774/
Test PASSed.

Merged build finished. Test PASSed.

LGTM  It worked fine when I ran it again just now.  Thanks for the PR!  Merging into master and branch-1.4


pact-jvm-provider-gradle fails to verify a message provider for versions 3.2.5, 3.2.6 and 3.2.7

While trying to use the pact-jvm-provider-gradle plugin (version 3.2.5, 3.2.6 or 3.2.7) to verify a message provider I encounter the following error:
Failures:

0) Verifying a pact between customer-details-consumer and customer-details-provider - a created event for the new customer Given a new customer is created
      No signature of method: static au.com.dius.pact.provider.ResponseComparison.compareMessage() is applicable for argument types: (au.com.dius.pact.model.v3.messaging.Message, java.lang.String) values: [au.com.dius.pact.model.v3.messaging.Message(a created event for the new customer, a new customer is created, au.com.dius.pact.model.OptionalBody(PRESENT, {"address":{"addressLine1":"address-line-1","addressLine2":"address-line-2","personId":1},"person":{"firstName":"Mr","id":1,"lastName":"Test"}}), [:], [Content-Type:application/json; charset=UTF-8, contentType:application/json]), ...]
      Possible solutions: compareMessage(au.com.dius.pact.model.v3.messaging.Message, au.com.dius.pact.model.OptionalBody)

When I downgrade to version 3.2.4 the same test passes:
  Given a new customer is created
         WARNING: State Change ignored as there is no stateChange URL
  a created event for the new customer
    generates a message which
      has a matching body (OK)

BUILD SUCCESSFUL

While I don't think my pact file or provider test function contain anything special I have provided them below in case it helps with reproducing the issue.
Pact file
{
    "consumer": {
        "name": "customer-details-consumer"
    },
    "provider": {
        "name": "customer-details-provider"
    },
    "messages": [
        {
            "metaData": {
                "contentType": "application/json",
                "Content-Type": "application/json; charset=UTF-8"
            },
            "providerState": "a new customer is created",
            "contentType": "application/json",
            "contents": {
                "address": {
                    "addressLine1": "address-line-1",
                    "addressLine2": "address-line-2",
                    "personId": 1
                },
                "person": {
                    "firstName": "Mr",
                    "id": 1,
                    "lastName": "Test"
                }
            },
            "matchingRules": {

            },
            "description": "a created event for the new customer"
        }
    ],
    "metadata": {
        "pact-specification": {
            "version": "3.0.0"
        },
        "pact-jvm": {
            "version": "3.2.7"
        }
    }
}

Provider function
    @PactVerifyProvider("a created event for the new customer")
    public String shouldCreateANewCustomer() {
        CustomerDetails customerDetails = new CustomerDetails(
                new Address(1, "address-line-1", "address-line-2"),
                new Person(1, "Mr", "Test")
        );

        return JsonOutput.toJson(customerDetails);
    }

It must be a regression. I'll have a look at what has changed since 3.2.4.

Fixed now


WEBSITE_RUN_FROM_PACKAGE recommendation

A user reported an issue in https://github.com/Azure/Azure-Functions/issues/1306 where the root cause was that the published function app was not using WEBSITE_RUN_FROM_PACKAGE = 1.
Although VS Code defaults to this setting, if the user creates the function app from the Portal and then deploys through VS Code, the setting is not added. Is it possible to log some sort of warning or just informational text that recommends that they set WEBSITE_RUN_FROM_PACKAGE to improve performance and predictability if it's not detected in the deployed site, especially with Node.js function apps?
cc: @ColbyTresness

Potentially. @ehamai can you comment on the feasibility of the above? And potentially help us with the data ask?

I still think it's super annoying that WEBSITE_RUN_FROM_PACKAGE isn't the default everywhere since it's so clearly better, but as long as we agree it's very low risk to modify WEBSITE_RUN_FROM_PACKAGE to 1 I think I'm coming around to this
Do we prompt the user to confirm this change? I hope not - since I think it'll just confuse people. But if we don't prompt, that means all users are forced to have WEBSITE_RUN_FROM_PACKAGE if they want to deploy from VS Code. Are there valid use cases where people don't want that setting and do we need to provide a workaround so they can still deploy from VS Code?

Sorry - I forgot this issue was just about a warning/recommendation. I thought it was about changing WEBSITE_RUN_FROM_PACKAGE automatically. What exactly do we want?

@EricJizbaMSFT - I think there's a separate thread going on with Paul, but for Windows apps, my preference is adding WEBSITE_RUN_FROM_PACKAGE to 1 automatically if it's not already there (note this has to happen before doing zip deployment). The perf gains are pretty massive (ex: I have a function deployed that only references the durable-functions package and the cold start is almost a minute which is just nuts). I've also seen apps have issues caused by our file change watcher that we enable if editing is enabled.
WEBSITE_RUN_FROM_PACKAGE does disallow editing but if people really want to edit it deployed code anyways, they can delete WEBSITE_RUN_FROM_PACKAGE and edit without issues.


strconv: ParseFloat should accept 'p' notation for binary exponents

See https://groups.google.com/d/topic/golang-dev/oIB-wBj3ufw/discussion.
The language specification never mentioned binary exponent float representation, but it was previously included in the gc implementation and it is included as a formatting option via strconv.AppendFloat with the 'b' fmt argument. However, it now lives on as a parsing option only in the compiler and test code in strconv.
The capacity to represent exact float values in a clear human-readable way is valuable in numeric code, for example here, where otherwise comments are required to explain the magic hex.
It is not clear how this should be included, since parsing a string is failable at runtime and these values are likely to nearly always be compile time constants.
/cc @griesemer

Some comments:

The compiler is using the mechanism to write (export) and read (import) exported float constants, written using the p exponent, because it permits an easy and lossless representation of a float in decimal form (decimal mantissa, exponent to power of 2, but written in decimal form). Note that the need for this format is likely going away since a binary representation of the exported data is more compact, just as precise, and faster to read and write (I'm working on a respective change).

The strconv.AppendFloat representation of the 'b' format requires the bitsize of the argument (32 or 64). It simply interprets the mantissa as a large decimal, and then prints the exponent. For instance, a float64 0.0, using 'b' format, formats as: "0p-1074" which is somewhat odd as it requires understanding of the 64bit float format to explain how the result was obtained. Similarly, 1.0 is printed as<PHONE_NUMBER>370496p-52, that is it is the float64 mantissa (53 bits) interpreted as a decimal, which is then printed<PHONE_NUMBER>370496, same as 1<<52), followed by the exponent (http://play.golang.org/p/Rt0SIFzzHi). Again, it requires the mantissa size to explain the output. (0p0 and 1p0 would be just a valid, but be more expensive to derive - basically it's the same mantissa with trailing 0's removed and the exponent adjusted - a canonical form).

And some questions:

What are you proposing? (Is this a proposal?)
Are you arguing that this format should be acceptable syntax in the language?

This is not a proposal, I wanted to sound things out first.
I don't think that it needs to be part of the language, the utility outside numerics is limited. At the most making strconv.ParseFloat handle it is what I am thinking.

Having strconv.ParseFloat handle the format sounds reasonable to me. I think the next step would be to define the exact format (syntax). It's probably something like:
number = [sign] mantissa 'p' [sign] exponent.
sign = '+' | '-' .
mantissa = decimalDigit {decimalDigit}.
exponent = decimalDigit {decimalDigit}.

Questions:

Should both 'p' and 'P' be permitted? Why/why not?
Can the mantissa be hexadecimal? Why/why not?

It seems to me that a single case for 'p' makes sense because it is a (marginally) simpler thing to look for instances (visually and mechanically) when there is only one thing to look for and that thing is visually distinct from a digit (in the decimal case). Mantissa optionally as hex makes moderate sense since a bit pattern may be what is being specified.

Permitting only 'p' sounds good. Perhaps for a start, also leave away hexadecimal notation. Thus, a float in p notation is essentially a signed integer followed by a 'p' exponent.
Venture to send a change list? (strconv/atof.go)

Yeah, I'll look into that. Just an initial observation though; it seems that fmt.Scan* handle binary exponent float representations judging by the test cases that exist (though also with dot rather than int-only mantissa). big.Float also handles these cases, but also includes hex input (including non-int).
Before I add to the variety, I'd like to get input on that.

I haven't looked at fmt.Scan. Permitting a decimal point for a decimal mantissa is tricky. The point of the 'p' notation is 100% lossless conversion with a fast and simple algorithm. In general that's not true anymore once a decimal point is permitted.
big.Float uses a different format: the mantissa is represented by a hex number which corresponds to the bits after (to the right) of the "decimal" point - that is, that mantissa value m is 0.5 <= m < 1.0. It's essentially used for testing (and could possibly be changed).
Given a sign s (-1, +1), a mantissa m that is simply a decimal unsigned integer, and a binary exponent b, the floating point value x is x = s * m * 2**b . No further explanation needed.
There are design decisions to be made when printing using a binary exponent: The mantissa may be scaled arbitrarily. Currently, printing simply prints the float32/float64 mantissa bits like if they were int32 or int64 bit numbers (with appropriate exponent). This requires knowing the bit size of the type to reproduce. Another option would be to always print a canonical form; for instance such that the mantissa is the smallest possible value before requiring a decimal point. That is equivalent to having no trailing 0's in the mantissa (or the mantissa being odd, except for x == 0).
But for parsing it doesn't matter.
More generally: it seems that strconv conversion routines should parse numbers that it can print.

Agreed. Just getting clarification.

@kortschak, regarding your initial comment:

The capacity to represent exact float values in a clear human-readable way is valuable in numeric code, for example here, where otherwise comments are required to explain the magic hex.

And the code says:
var (
	// dlamchE is the machine epsilon. For IEEE this is 2^-53.
	dlamchE = math.Float64frombits(0x3ca0000000000000)

	// dlamchP is 2 * eps
	dlamchP = math.Float64frombits(0x3cb0000000000000)

	// dlamchS is the "safe min", that is, the lowest number such that 1/sfmin does
	// not overflow. The Netlib code for calculating this number is not correct --
	// it overflows. Found by trial and error, it is equal to (1/math.MaxFloat64) * (1+ 6*eps)
	dlamchS = math.Float64frombits(0x4000000000001)

	...
)

I want to make the point, unrelated to what we do in strconv, that this is unnecessary in Go. This kind of thing - specifying floating point constants in hexadecimal - is rampant in C because C compilers have historically been quite bad at reading floating point inputs. Using hex was the only way to guarantee the compiler arrived at the number you intended. But modern practice has improved, and Go gets this right. There are any number of ways you could write the above code using plain floating point constants, but the most direct is:
var (
	// dlamchE is the machine epsilon. For IEEE this is 2^-53.
	dlamchE = 1.1102230246251565e-16

	// dlamchP is 2 * eps
	dlamchP = 2.220446049250313e-16

	// dlamchS is the "safe min", that is, the lowest number such that 1/sfmin does
	// not overflow. The Netlib code for calculating this number is not correct --
	// it overflows. Found by trial and error, it is equal to (1/math.MaxFloat64) * (1+ 6*eps)
	dlamchS = 5.56268464626801e-309

	...
)

This is guaranteed to have the same effect as the math.Float64frombits calls.

Postponing the strconv work.

This misses the point. If we were able to say
var (
    // dlamchE is the machine epsilon. For IEEE this is 2^-53.
    dlamchE = 1p-53 // or package-provided equivalent.

    // dlamchP is 2 * eps
    dlamchP = 2*dlamchE

    // dlamchS is the "safe min", that is, the lowest number such that 1/sfmin does
    // not overflow. The Netlib code for calculating this number is not correct --
    // it overflows. Found by trial and error.
    dlamchS = (1/math.MaxFloat64) * (1+ 6*dlamchE)

    ...
)

then I would agree, but we can't. The capacity to express exact float values is less than half the problem.

@kortschak FWIW, in Go we can express 1p-53 quite elegantly as the constant expression 1.0/(1<<53). All the various forms agree with the value computed from the bit pattern: http://play.golang.org/p/VjkVDA8PrL .
Or more generally, any float constant of the form xxxp+exp or xxxp-exp can be expressed as xxx<<exp or xxx.0/(1<<exp) . Thus, the need for the p notation in code is diminished.

@griesemer Thanks for that tip. This covers our use case better than a strconv parser, though I feel probably the last sentence in https://github.com/golang/go/issues/12518#issuecomment-139406627 justifies this addition for other uses.

FWIW I agree that since strconv can generate the p form it should also accept the p form. That said, doing so correctly at the boundaries is tricky. It's not a 1-liner.

Decision is in my previous comment above: yes, it's fine to do this just get the corner cases right please.

In looking into this I have found that fmt.Scan does binary exponent format float parsing (as should be expected from the documentation). However, it is less restrictive than the documentation in fmt suggests it should be (for example, this works although the documentation specifies a "decimalless scientific notation"). It is even more relaxed than would be acceptable for strconv.ParseFloat since the routines backing the scan functions do not error when the exponent is out of range (instead setting the value to ±Inf). So I guess a question here is whether the scan float binary exponent functionality should be backed by a new strconv binary exponent float parser (keeping the behaviour that overflows are silently converted to Infs by discarding the out of range error), and whether the the Scan behaviour with decimal mantissas should be brought into the strconv.ParseFloat function (and documented in fmt).

The fmt scan overflow to inf behavior for binary exponents is a bug. Note that decimal exponents are handled right: https://play.golang.org/p/SHc0zAdyhx. It's just one more corner case that makes this non-trivial. It would be fine to support 1.2p4 as fmt.Scan does. In fact that's probably important. But it adds more corner cases.
I think we should probably postpone this to Go 1.9 since it's not urgent and there's little time left in Go 1.8. I'm interested to see this happen though.

@griesemer
The comment here shows a technique for representing these constants using shifts, however this only really handles a minority of cases easily; anything that is not a multiple of a power of 2 is difficult and (more troublingly) values where the exponent is greater than the spec minimum integer constant representation width cannot easily either. An example of this is the value LAPACK DLAMCH("S") which is 1p-1022 and which cannot be expressed using the expression 1.0 / (1 << 1022), requiring instead 1.0 / (1 << 256) / (1 << 256) / (1 << 256) / (1 << 254) to shoehorn float exponents into the integer model.

@kortschak Ack. Are you proposing the p notation in the language?

At this stage, that's just a data point that I had not noticed before.
The issue here is that I would suppose that a very small group of Go programmer would need that (and go compiler authors who have that in internal code as far as I remember). For the use that we have (1 case in two locations), I am suggesting for use to use the longer expression above. It would be nice to be able to express these values simply, but whether that is worth your (pl.) time is not clear to me.

Not happening for 1.9.

I think the agreement here is that we accept the 'p' notation for binary exponents with strconv.ParseFloat and eventually update fmt.Scan to match its behavior with respect to error handling or corner cases.
@martisch Is this something you might be interested in looking into (starting with ParseFloat)? If so, feel free to assign this issue to yourself.

I am interested and added myself.

/cc @ericlagergren

Moving this off 1.12. This takes a dedicated careful effort. We get to it when we get to it.

This looks like it was fixed in 07717247d8decf5a5793f04c368eab3f43fad44f.


Bump minimal python version to 3.8

All of the FV3 infrastructure has been changed to use Python 3.8, so no issues on that front.

All of the FV3 infrastructure has been changed to use Python 3.8, so no issues on that front.


Crash after changing metro.config.js - > node_modules\expo-router\entry.js: traverse is not a function

Describe the bug
after changing metro.config.js to
module.exports = {
...config,
transformer: {
...config.transformer,
babelTransformerPath: require.resolve(
"react-native-react-bridge/lib/plugin"
),
},
};
i can't start react app anymore:
node_modules\expo-router\entry.js: traverse is not a function
To Reproduce
Steps to reproduce the behavior.
Expected behavior
A clear and concise description of what you expected to happen.
Platform:

OS: iOS, Android, wen
Running on: Device, Browser
Version of this package: ^0.11.0

Additional context
Web Bundling failed 826ms
error: node_modules\expo-router_error.js: traverse is not a function
Web node_modules\expo-router/entry.js ░░░░░░░░░░░░░░░░  0.0% (0/1)
Metro error: node_modules\expo-router\entry.js: traverse is not a function
Call Stack
MetroBundlerDevServer.getStaticResourcesAsync<EMAIL_ADDRESS>process.processTicksAndRejections (node:internal/process/task_queues)
async MetroBundlerDevServer.getStaticPageAsync<EMAIL_ADDRESS>async<EMAIL_ADDRESS>Web Bundling failed 10ms
error: node_modules\expo-router\entry.js: traverse is not a function
Web Bundling failed 9ms
error: node_modules\expo-router\node\render.js: traverse is not a function
Web Bundling failed 12ms
error: node_modules\expo-router_error.js: traverse is not a function
› Detected a change in babel.config.js. Restart the server to see the new results. You may need to clear the bundler cache with the --clear flag for your changes to take effect.

Facing the same issue!

Thanks!
I've released a patch 0.11.1. Could you try it?

Hey! The patch 0.11.1 worked, the error is gone but now I'm facing the following:
ReferenceError: Property 'n' doesn't exist, js engine: hermes.
To reproduce it, just create an Expo project and follow the instructions on this package's Homepage to create a demo project.

Hey! The patch 0.11.1 worked, the error is gone but now I'm facing the following:
ReferenceError: Property 'n' doesn't exist, js engine: hermes.
To reproduce it, just create an Expo project and follow the instructions on this package's Homepage to create a demo project.

Are you suggesting we use the new_script function as is but instead of using a path we copy the whole code block in myAlert.js inside the double quotes? Or should we not use the new_script function at all and refer to our code in a different way? We're struggling to understand how we should reference the code block. This is our standalone setup that runs by itself in html/js that we are trying to run inside the react component using react-native-react-bridge: https://gist.github.com/zserge/6617cdbf328aac76913ff9ae85350c10

I am also facing same issue with version 0.11.1.

Having the same issue. Any ideas?

0.11.2 is released. Thanks!


docs for buildkite-agent meta-data doesn't work

i see
buildkite-agent@build1:~$ buildkite-agent meta-data set "foo" "bar"
2018-05-14 12:14:45 FATAL  Missing job. See: `buildkite-agent meta-data set --help`
buildkite-agent@build1:~$ buildkite-agent meta-data set --help
Usage:

   buildkite-agent meta-data set <key> [<value>] [arguments...]

Description:

   Set arbitrary data on a build using a basic key/value store.

   You can supply the value as an argument to the command, or pipe in a file or
   script output.

Example:

   $ buildkite-agent meta-data set "foo" "bar"
   $ buildkite-agent meta-data set "foo" < ./tmp/meta-data-value
   $ ./script/meta-data-generator | buildkite-agent meta-data set "foo"

Options:

   --job value                 Which job should the meta-data be set on [$BUILDKITE_JOB_ID]
   --agent-access-token value  The access token used to identify the agent [$BUILDKITE_AGENT_ACCESS_TOKEN]
   --endpoint value            The Agent API endpoint (default: "https://agent.buildkite.com/v3") [$BUILDKITE_AGENT_ENDPOINT]
   --no-color                  Don't show colors in logging [$BUILDKITE_AGENT_NO_COLOR]
   --debug                     Enable debug mode [$BUILDKITE_AGENT_DEBUG]
   --debug-http                Enable HTTP debug mode, which dumps all request and response bodies to the log [$BUILDKITE_AGENT_DEBUG_HTTP]

which is supposed to work?

Hey @floswald!
If buildkite-agent was run as part of a build pipeline there would be a $BUILDKITE_JOB_ID environment variable, but because you're running it on the command line (not in the context of a job) it's likely that you don't have $BUILDKITE_JOB_ID set.
All metadata is tied to a job (a key/value store for each job).
You should be able to test it on the command line by adding the --job flag, eg:
buildkite-agent meta-data set "foo" "bar" --job <jobIDhere>

The error on this could absolutely be better!

Moved over to the Agent repo to get the error updated.


”Ezeriș” is listed in the drop-down menu

Description:
"Ezeriș" is listed in the drop-down menu alongside its corresponding county and commune, according to the law.
Precondition:
The website is up an running.

Step 1
Type "Ezeriș" in the search bar.

Expected results
The location is listed in the drop-down menu as "Ezeriș, Caraș-Severin".


Bump PyJWT

Currently, Saleor uses PyJWT==2.50. Bumping to 2.7.0 makes the tests fail with the following error:
E           AssertionError: [{'message': 'The token is not yet valid (iat)', 'locations': [{'line': 3, 'column': 9}], 'path': ['orders'], 'extensions': {'exception': {'code': 'ImmatureSignatureError'}}}]

Steps to reproduce:

Install PyJWT==2.7.0

$ pip install pyjwt==2.7.0

Run tests:

$ pytest

As a result, multiple tests are failing with the above error. We should investigate what is causing the error and fix it so we can upgrade to newer versions of PyJWT.

This is duplicate of https://github.com/saleor/saleor/issues/11047.


Report voltage level

Enhance heartbeat and complex get messages by also reporting the voltage levels

there is an interrupt HLVDIF that triggers and interrupt if voltage is low or high, then perhaps the voltage comparator module can help with this reading in general

closing, would not help with the scenario anyway since power save mode should really not be used to send too much excessive traffic


Understand max tile sources fix

In #657 I introduced a change that fixes the blend figure for the magnitude as a function of blendedness. The breaking changed was introduced in #650.
But this is not satisfying, I believe there is a bug that we need to track but I need to dig more so I'm leaving it for later

As I said in my original comment back on #642, it would be best just to bump up the max sources everytime this condition is met. While this amounts to re-allocating the tensors each time, I think it's more important that the mapping between TileCatalog and FullCatalog representations is a true bijection.
As far as I am aware, the conversion from FullCatalog to TileCatalog doesn't occur in any "hot" parts of the code anyways (i.e. during the training or prediction steps of BLISS), so I don't see much of a downside of this approach.

From #642
| In theory the number of galaxies per tile is can go up to infinity, of course even 2 or 3 is an unlikely outcome.
I agree that this scenario may be more unstable than converting FullCatalogs from astronomical surveys (which was the original intention of the FullCatalog -> TileCatalog conversion). But, even in this simulation study, for a given FullCatalog, the number of sources per tile is bounded above by the total number of objects in the catalog, and this scenario would be quite pathological (every source in a single tile). So I'm not convinced that we should break the bijectivity of the representations just for this use case.

They were two problems:
(1) Actually, the function to_tile_params works fine in this context, the problem was in UniformBrightestCenterGalsimGalaxy that imposed a specific order in the catalog. It should remain arbitrar.  We could improve to_tile_params to always randomize which source it picks to be in a tile when there are multiple ones. This seems unncessary because the order of a catalog should be arbitrary in general
(2) In this context actually, I did FullCatalog -> TileCatalog -> FullCatalog without a good reason, this was the source of the issue. No reason to do a lossy transformation for this.
I feel like (2) was the real source of the issue, so I will fix that in a PR, I can't imagine a context where (1) is a problem except when I impose an order myself.  I will also avoid that in UniformBrightestCenterGalsimGalaxy


Updated dependency: beacontools

Description:
Updated the beacontools package to the lastest version in order to fix dependency conflict. See home-assistant/hassio#356
Related issue (if applicable): fixes home-assistant/hassio#356
Checklist:

[x] The code change is tested and works locally.

Thank you for your quick update.

@balloob we need make a 0.63.2 soon as possible. This fix a big problem with our docker images.

@pvizeli @rytilahti 11 hours ago a new construct version was released which breaks the miio components again. We tried to avoid pinning in the past and failed multiple times now. A new release of HA wouldn't help at the moment.
https://github.com/rytilahti/python-miio/pull/220

As soon as this PR (https://github.com/home-assistant/home-assistant/pull/12392) is merged the xiaomi_miio components are up and running again.


[BUG] BaseModel with simple root type not allowed in Path and Query params

Describe the bug
I want to extract enums from model schema to separate definition, and the solution is:
from enum import Enum

from pydantic import BaseModel
from devtools import debug

class HelloType(str, Enum):
    World = 'World',
    Friend = 'Friend',

class HelloEnum(BaseModel):
    __root__: HelloType

class Foo(BaseModel):
    bar: HelloEnum

debug(HelloEnum.schema())
debug(Foo.schema())

Output:
./main.py <module>
    HelloEnum.schema(): {
        'title': 'HelloEnum',
        'enum': [
            'World',
            'Friend',
        ],
        'type': 'string',
    } (dict) len=3
./main.py <module>
    Foo.schema(): {
        'title': 'Foo',
        'type': 'object',
        'properties': {
            'bar': {
                '$ref': '#/definitions/HelloEnum',
            },
        },
        'required': ['bar'],
        'definitions': {
            'HelloEnum': {
                'title': 'HelloEnum',
                'enum': [
                    'World',
                    'Friend',
                ],
                'type': 'string',
            },
        },
    } (dict) len=5

But I can't use HelloEnum in Query parameters:
@app.get('/')
async def root(hello_type: HelloEnum = Query(...)):
    return {'message': f'Hello {hello_type}'}

And I get the error:
File "./main.py", line 28, in <module>
    async def root(hello_type: HelloEnum = Query(...)):
  File "<..>/python3.8/site-packages/fastapi/routing.py", line 514, in decorator
    self.add_api_route(
  File "<..>/python3.8/site-packages/fastapi/routing.py", line 453, in add_api_route
    route = route_class(
  File "<..>/python3.8/site-packages/fastapi/routing.py", line 370, in __init__
    self.dependant = get_dependant(path=self.path_format, call=self.endpoint)
  File "<..>/python3.8/site-packages/fastapi/dependencies/utils.py", line 308, in get_dependant
    assert isinstance(
AssertionError: Param: hello_type can only be a request body, using Body(...)

To Reproduce

Create a file with:

from enum import Enum

from fastapi import FastAPI, Query
from pydantic import BaseModel

app = FastAPI()

class HelloType(str, Enum):
    World = 'World',
    Friend = 'Friend',

class HelloEnum(BaseModel):
    __root__: HelloType

@app.get('/')
async def root(hello_type: HelloEnum = Query(...)):
    return {'message': f'Hello {hello_type}'}

Run app.
It trows an exception AssertionError: Param: hello_type can only be a request body, using Body(...).
But I expected it to run successfully.

Expected behavior
Run app successfully, and in openapi schema enum extracted in the separate definition.
Environment

macOS 10.15.4
FastAPI 0.54.2
Python 3.8.2

Do you think it's a duplicate of #911?

Partially. In this case, I am trying to extract the enum in the schema definitions. This useful for openapi-generator, otherwise it generates separate enums (e.g. for typescript).

I have a similar issue with request body.
For example:
class QueryDict(BaseModel):
    __root__: dict[str, str]

    # some validators here
    ...

class Query(BaseModel):
    query: QueryDict

@app.post('/foo')
def endpoint(query: Query):
    ...

Request:
curl -X POST "http://<IP_ADDRESS>:8888/foo" -H  "accept: application/json" -H  "Content-Type: application/json" -d '{"query":{"hello":"world"}}'

In this case I get a validation error:
{
  "detail": [
    {
      "loc": [
        "body",
        "query",
        "__root__"
      ],
      "msg": "field required",
      "type": "value_error.missing"
    }
  ]
}

Pydantic models are not supported for other things apart from bodies. I want to add a way to support them for query parameters and others, but that's not supported yet.

Sorry for the long delay! 🙈  I wanted to personally address each issue/PR and they piled up through time, but now I'm checking each one in order.


Support Multiple World Languages in editor

Starting languages would be Spanish, French, German and Arabic.
This is a huge amount of work, especially as it may require a new parsing mechanism for handling fonts as many fonts only support one language or so and some languages require wide-character strings which is currently not supported in the Editor's Engine at all.

I will move this over to Summer 2024 as this will involved changing encodings and/or from the std::string class if needed. Also some of the modules such as sdl2_fonts will definitely need to be redone for this.


Add xiaozhu36 as approver

xiaozhu36 is the maintainer of repo bosh-alicloud-cpi-release and stemcells-alicloud-index. As the cloudfoundry
org requirements pr, the two repos will move to cloudfoundry.

Hi @xiaozhu36,
could you please proceed as suggested in the issue and open a PR to move the Ali Cloud CPI to the FI WG. If you need support with that I can help.

@beyhan Yes, I want to move it, but I have no privilege of cloudfoundry org to do it. It is very appreciate if you can help me.

@xiaozhu36 the move will be executed by us but you can create the pr to create the required repository in the cloudfoundry org. When a pr like https://github.com/cloudfoundry/community/pull/784 is merged it triggers automations which creates the repository. If you need help to create the pr, please let me know.

@beyhan OK, thanks a lot. Please help me to do it.

@xiaozhu36 I created https://github.com/cloudfoundry/community/pull/798 and will close this one.


Should the KafkaMessage type allow for null value?

It is useful to produce null records "tombstone" to a key. It may also be useful to know when consuming a null record that has not yet been compacted out.
Currently with the types defined for KafkaJS we can describe null values when producing:
https://github.com/tulios/kafkajs/blob/1904481eefe53ce169b38420db59cc68ec819cf7/types/index.d.ts#L56-L62
But not on the consumer side: https://github.com/tulios/kafkajs/blob/1904481eefe53ce169b38420db59cc68ec819cf7/types/index.d.ts#L402-L410
Should the KafkaMessage type allow for null value?

It is useful to produce null records "tombstone" to a key

Totally! Tombstones are key (pardon the pun) to making changelogs work.

It may also be useful to know when consuming a null record that has not yet been compacted out.

Another definitely! Again, essential to making change logs work.

Should the KafkaMessage type allow for null value?

It probably should be. While the Kafka protocol requires a value to be bytes (hence the Buffer typein Node), it also allows for null. AFAIKnull` equating to a tombstone is more of a convention than part of the spec, but supporting it as a payload should probably supported regardless.

Fixed by #764


Add new buildconfig that can be used by the docker-based slsa generator

I'll merge this to allow some testing, and will iterate on it later.


cmd/server: rename manual refresh endpoint to /data/refresh

Issue: https://github.com/moov-io/ofac/issues/98

Codecov Report

Merging #101 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master    #101   +/-   ##
======================================
  Coverage    58.1%   58.1%
======================================
  Files          21      21
  Lines        2270    2270
======================================
  Hits         1319    1319
  Misses        807     807
  Partials      144     144


I am paying for Github

Type: Bug
How come I have to wait 6 hours? I have subscription
Extension version: 0.22.1
VS Code version: Code 1.95.0 (Universal) (912bb683695358a54ae0c670461738984cbb5b95, 2024-10-28T20:16:24.561Z)
OS version: Darwin arm64 23.6.0
Modes:

System Info

Item
Value

CPUs
Apple M2 Max (12 x 2400)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: enabled_ondirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_graphite: disabled_offvideo_decode: enabledvideo_encode: enabledwebgl: enabledwebgl2: enabledwebgpu: enabledwebnn: disabled_off

Load (avg)
7, 8, 8

Memory (System)
32.00GB (0.04GB free)

Process Argv
--crash-reporter-id 8979f77b-4812-441e-8ec7-4cdbef52961a --crash-reporter-id 8979f77b-4812-441e-8ec7-4cdbef52961a

Screen Reader
no

VM
0%

A/B Experiments
vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscod805:30301674
binariesv615:30325510
vsaa593:30376534
py29gd2263:31024239
vscaac:30438847
c4g48928:30535728
azure-dev_surveyone:30548225
962ge761:30959799
pythongtdpath:30769146
pythonnoceb:30805159
asynctok:30898717
pythonmypyd1:30879173
h48ei257:31000450
pythontbext0:30879054
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
724cj586:31013169
dvdeprecation:31068756
dwnewjupytercf:31046870
impr_priority:31102340
nativerepl2:31139839
refactort:31108082
pythonrstrctxt:31112756
cf971741:31144450
iacca1:31171482
notype1cf:31157160
5fd0e150:31155592
dwcopilot:31170013

Hi @aysuncookie23 can you please elaborate on what you said? What do you mean by waiting for 6 hours?

Sure, I subscribed to Github Copilot, and I realize that I had to wait 6 hours to be able to chat with github copilot  after couple of prompts. Also why is it so slow? why when you ask it; "where is this function x located", it doesn't know?

Concerning speed we are looking into it. Next time you hit the issue with Copilot not being able to locate code, could you please share the response with us by clicking on the thumbs down icon, which would open the context menu, in which there would be an action to share the response?


library doesn't allow \n in text

For example here is the very first demo:

based on text:
`Let's make some
<ml>multiline</ml>
and <ms>multistyle</ms> text for
<pixi>Pixi.js!</pixi>`,

This one is preventing me from doing any testing or integrating into my app, so if you have time please have a look.

Something is up with the demo. This was all working fine AFAIK when I originally uploaded everything and sent you the link so I'll have to figure out what happened.

@rizen I'm still not sure how I missed this, but after testing, this commit seems to be the last good commit: 29dbc806ef424f640275eebbc145d85f88bd35cf
If you need to use something for testing, I'd check that out and start there while I figure it out.


Question: Any recommendation on Social Sign In support

Hi @ottokruse,
Is there any support from the current client SDK to build social sign in (e.g. Google sign in)?
Comparing to Amplify Auth for client SDK, they have built-in support on using federated sign in to support Social logins or other OAuth endpoints (and you can customize based on Amplify Authentication documentation.
Thank you very much!
Frankie

Hi @fkwfung
In this case I'd recommend to use both this lib and Amplify, and init federated sign in with the Amplify method.
(You could then see this Passwordless lib as a helper to Amplify, complementing Amplify)
You've probably seen this already: Usage with AWS Amplify
You're not the first user to ask by the way, see discussion in #76 Initiating Oauth2 is pretty straightforward (just a browser redirect) and handling the redirect back from Cognito isn't hard either, so we were tempted to add it to this lib too (so as to be a one stop shop for Auth if it can), but it has little priority.

Yeah, there were 3 main issues (I'm using React as web application):

Auth state checking - On each page refresh or Google sign in redirect, I will need to build a controller to determine which way the user is authenticated in order to retrieve id_token / access_token for API call from either Passwordless React hook or Amplify Auth functions. (I used a custom Lambda authorizer of which the frontend to pass the id_token as bearer to validate from the Cognito)
Concurrent sign in if federated identity not properly signing out - In some scenarios, if a user requests a Magic Link, and at the same time the user signs in via Google, after signing in, he then clicks back on the Magic Link from email. It will have both ways signed in. Understanding this is Application logic to handle various odd cases, but separating into two ways of authentication will increase the complexity of the controller to handle in scenarios like this.
Using "out-of-the-box" Amplify Analytics on Kinesis - The setup of the React application was originally overriding Auth config at runtime in React without defining at Amplify Backend. However, this is not possible if Analytics is applied. As Amplify forces to apply Auth (a Cognito must be defined in backend either new or re-use existing) for both authenticated and guest (unauthenticated) event tracking via Kinesis, we need to define a dummy Cognito, and manually assigning PutRecord permission on the original Passwordless Identity Pool.

Now I will see more value adding OAuth support on the Passwordless stack 👍
Thanks!

Oof sounds like some rabbit holes 😓 Glad though you were able to get there in the end.
If you can share code snippets of your work arounds that'd be great for better understanding.


Notification emails are unstructured

Notification emails don't reference each other, leading to my inbox being interspersed with a stream of unrelated messages sharing the same subject, but being completely unthreaded. This is in departure to what mailing lists do, where messages which are replies to each other reference each other, so that the client can group them into related threads.
The usual way to do it is to fill in the In-Reply-To: and References fields, like this:
In-Reply-To<EMAIL_ADDRESS>References<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
Ideally, the fields reference the Message-Id of the previous email, but I think sufficiently smart email clients could recognize when multiple emails share the same In-Reply-To even when the original message is not present. Then the "original" could just point to a stable address of the Zulip stream/thread. (This would need more testing to determine actual client support)

Closing as a duplicate of  #14337.


Find first section for chevron navigation

I wasn't sure how to sort by weight and also filter for only enable=true. I tried something similar to this using where but wasn't able to get it to work.
https://discourse.gohugo.io/t/solved-sort-and-where-in-a-range-site-data/8225/2

@blueyetisoftware I have pushed a change. Previously, the disabled sections was not handled properly. For example, if About section is disabled using section.enable: false it will be still present in the sorted map. Hence, the chevron was pointing to the about section.


Remove implicit interval type coercion from ScalarValue comparison

Which issue does this PR close?

Part of #7353
Rationale for this change

We have explicit type coercion machinery during plan construction that handles this, doing this implicitly within ScalarValue introduces inconsistency and is unnecessary. I am also not sure the type coercion logic is correct, it is certainly not consistent with the arrow-cast kernel.
Potentially more critically this definition of PartialEq is inconsistent with the definition of Hash, which could cause some very strange behaviour
What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Thank you -- this makes a lot of sense to me. I think some of this was left over from when the arrow cast kernels didn't support all the different types of coercion
cc @berkaysynnada

Yes, you are right. We did this before arrow kernels, when I didn't fully know the purpose of Interval type. I can now say that they were erroneous and they are unnecessary at the moment.


Add switch between Hosted Checkout and Mollie Components

This introduces a switch in the checkout that toggles between a card with available payment methods that are dynamically retrieved from the methods API and a card that displays Mollies components for credit cards.
See: https://docs.mollie.com/docs/mollie-components
Since the card with available payment methods is a server component, we will pass it as a prop to a client component, wrapping it in suspension so the UI can update async

to do:

Skeleton for payment selection ✅
components card duh ✅
make components look nice ✅
pass cardToken to mollieCreatePayment (optionally)