Can't build www/nginx package

I've tried to use synth for the first time. After the "synth upgrade-system" command I see the only one failed package to build - www/nginx
Log file: https://gist.github.com/mephist/ce8a090cb0d8adba866aafe0917f81cc
It fails on configure, complaining about missing libxml2/libxslt.
I've installed both of them

$ sudo synth install textproc/libxml2 textproc/libxslt
After inspection, it has been determined that there are no packages that
require rebuilding; the task is therefore complete.
Stand by, recursively scanning 2 ports serially.
Packages validated, rebuilding local repository.
Local repository successfully rebuilt
Updating Synth repository catalogue...
Fetching meta.txz: 100%    260 B   0.3kB/s    00:01
Fetching packagesite.txz: 100%   98 KiB 100.2kB/s    00:01
Processing entries: 100%
Synth repository update completed. 394 packages processed.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The most recent version of packages are already installed

and rerun synth build www/nginx, but see the same error
Log file: https://gist.github.com/mephist/ce8a090cb0d8adba866aafe0917f81cc
if I try to make -C /usr/ports/www/nginx it has been built without an error

[mephist@srv ~]$ uname -vp
FreeBSD 12.0-RELEASE-p3 GENERIC  i386
[mephist@srv ~]$ pkg version -v | egrep '(libxslt|libxml2)'
libxml2-2.9.8                      =   up-to-date with index
libxslt-1.1.32                     =   up-to-date with index
[mephist@srv ~]$ make -C /usr/ports/www/nginx showconfig
===> The following configuration options are available for nginx-1.14.2_13,2:
DEBUG=off: Build with debugging support
DEBUGLOG=off: Enable debug log (--with-debug)
DSO=on: Enable dynamic modules support
FILE_AIO=off: Enable file aio
IPV6=off: Enable IPv6 support
THREADS=on: Enable threads support
WWW=on: Enable html sample files
====> Modules that require MAIL module
MAIL=off: Enable IMAP4/POP3/SMTP proxy module
MAIL_IMAP=off: Enable IMAP4 proxy module
MAIL_POP3=off: Enable POP3 proxy module
MAIL_SMTP=off: Enable SMTP proxy module
MAIL_SSL=off: Enable mail_ssl module
====> Modules that require HTTP module
GOOGLE_PERFTOOLS=off: Enable google perftools module
HTTP=on: Enable HTTP module
HTTP_ADDITION=off: Enable http_addition module
HTTP_AUTH_REQ=off: Enable http_auth_request module
HTTP_CACHE=on: Enable http_cache module
HTTP_DAV=on: Enable http_webdav module
HTTP_FLV=off: Enable http_flv module
HTTP_GUNZIP_FILTER=on: Enable http_gunzip_filter module
HTTP_GZIP_STATIC=on: Enable http_gzip_static module
HTTP_IMAGE_FILTER=off: Enable http_image_filter module
HTTP_MP4=on: Enable http_mp4 module
HTTP_PERL=off: Enable http_perl module
HTTP_RANDOM_INDEX=off: Enable http_random_index module
HTTP_REALIP=off: Enable http_realip module
HTTP_REWRITE=on: Enable http_rewrite module
HTTP_SECURE_LINK=off: Enable http_secure_link module
HTTP_SLICE=on: Enable http_slice module
HTTP_SSL=on: Enable http_ssl module
HTTP_STATUS=on: Enable http_stub_status module
HTTP_SUB=off: Enable http_sub module
HTTP_XSLT=off: Enable http_xslt module
HTTPV2=on: Enable HTTP/2 protocol support (SSL req.)
STREAM=on: Enable stream module
STREAM_SSL=on: Enable stream_ssl module (SSL req.)
STREAM_SSL_PREREAD=on: Enable stream_ssl_preread module (SSL req.)
AJP=off: 3rd party ajp module
AWS_AUTH=off: 3rd party aws auth module
BROTLI=off: 3rd party brotli module
CACHE_PURGE=off: 3rd party cache_purge module
CLOJURE=off: 3rd party clojure module
CT=off: 3rd party cert_transparency module (SSL req.)
DEVEL_KIT=on: 3rd party Nginx Development Kit module
ARRAYVAR=off: 3rd party array_var module
DRIZZLE=off: 3rd party drizzle module
DYNAMIC_UPSTREAM=off: 3rd party dynamic_upstream module
ECHO=off: 3rd party echo module
ENCRYPTSESSION=off: 3rd party encrypted_session module
FASTDFS=off: 3rd party fastdfs module
FORMINPUT=off: 3rd party form_input module
GRIDFS=off: 3rd party gridfs module
HEADERS_MORE=off: 3rd party headers_more module
HTTP_ACCEPT_LANGUAGE=off: 3rd party accept_language module
HTTP_AUTH_DIGEST=off: 3rd party http_authdigest module
HTTP_AUTH_KRB5=off: 3rd party http_auth_gss module
HTTP_AUTH_LDAP=off: 3rd party http_auth_ldap module
HTTP_AUTH_PAM=off: 3rd party http_auth_pam module
HTTP_DAV_EXT=on: 3rd party webdav_ext module
HTTP_EVAL=off: 3rd party eval module
HTTP_FANCYINDEX=off: 3rd party http_fancyindex module
HTTP_FOOTER=off: 3rd party http_footer module
HTTP_GEOIP2=off: 3rd party geoip2 module
HTTP_JSON_STATUS=off: 3rd party http_json_status module
HTTP_MOGILEFS=off: 3rd party mogilefs module
HTTP_MP4_H264=on: 3rd party mp4/h264 module
HTTP_NOTICE=off: 3rd party notice module
HTTP_PUSH=off: 3rd party push module
HTTP_PUSH_STREAM=off: 3rd party push stream module
HTTP_REDIS=off: 3rd party http_redis module
HTTP_RESPONSE=off: 3rd party http_response module
HTTP_SUBS_FILTER=off: 3rd party subs filter module
HTTP_TARANTOOL=off: 3rd party tarantool upstream module
HTTP_UPLOAD=off: 3rd party upload module
HTTP_UPLOAD_PROGRESS=off: 3rd party uploadprogress module
HTTP_UPSTREAM_CHECK=off: 3rd party upstream check module
HTTP_UPSTREAM_FAIR=off: 3rd party upstream fair module
HTTP_UPSTREAM_STICKY=off: 3rd party upstream sticky module
HTTP_VIDEO_THUMBEXTRACTOR=on: 3rd party video_thumbextractor module
HTTP_ZIP=off: 3rd party http_zip module
ICONV=off: 3rd party iconv module
LET=off: 3rd party let module
LUA=on: 3rd party lua module
MEMC=off: 3rd party memc (memcached) module
MODSECURITY=off: 3rd party mod_security module
MODSECURITY3=off: 3rd party modsecurity3 module
NAXSI=off: 3rd party naxsi module
NJS=off: Enable http_javascript module
PASSENGER=off: 3rd party passenger module
POSTGRES=off: 3rd party postgres module
RDS_CSV=off: 3rd party rds_csv module
RDS_JSON=off: 3rd party rds_json module
REDIS2=off: 3rd party redis2 module
RTMP=on: 3rd party rtmp module
SET_MISC=off: 3rd party set_misc module
SFLOW=off: 3rd party sflow module
SHIBBOLETH=off: 3rd party shibboleth module
SLOWFS_CACHE=off: 3rd party slowfs_cache module
SMALL_LIGHT=off: 3rd party small_light module
SRCACHE=off: 3rd party srcache module
VOD=on: 3rd party vod module
VTS=off: 3rd party vts module
XSS=off: 3rd party xss module
WEBSOCKIFY=off: 3rd party websockify module
====> GSSAPI implementation (imply HTTP_AUTH_KRB5): you can only select none or one of them
GSSAPI_BASE=off: GSSAPI support via base system (needs Kerberos)
GSSAPI_HEIMDAL=off: GSSAPI support via security/heimdal
GSSAPI_MIT=off: GSSAPI support via security/krb5
===> Use 'make config' to modify these settings

it looks like a bug in the port itself.
Why are you reporting this at the synth upstream instead of reporting it to maintainer of www/nginx?

Why are you reporting this at the synth upstream instead of reporting it to maintainer of www/nginx?

Because www/nginx is built correctly via "make package", without synth

[mephist@srv /usr/ports/www/nginx]$ sudo make package
===>  Building package for nginx-1.14.2_13,2
[mephist@srv /usr/ports/www/nginx]$ ll /usr/ports/packages/All/
total 864
-rw-r--r--  2 root  wheel  828732 21 Apr.  17:35 nginx-1.14.2_13,2.txz

Do porters need to use some extra magic to make their packages built in synth?

Let me clarify my question. Is there a way to manually reproduce steps (mount/chroot/whatever) which are performed by synth to build a single port?
I want to try to troubleshoot it myself, to find the root cause. I see Ada for the first time to follow steps by code lines.
Thanks a lot for patient.

First, "make package" uses a live system, which cannot be reproduced, nor can the build environment be controlled.  So saying something is good because "make build" works is a fallacy.  It has to build in poudriere or synth.
The problem is that the port makefile doesn't require libxslt.  You seem to have it on your live system, so builds out of luck.  Believe me, the problem is with the nginx port.


Add vuex data support and usage example

Now you can use page-specific vuex data, but I think this approach is still unreliable, be careful.

Hi @DreaMinder! Sorry for taking this long to get back to you; I really appreciate your efforts in making this approach possible.
We've been busy launching the site, and it's finally live: https://thejewelleryroom.com
I am still interesting in testing it out, however, I postponed it a bit, because I got worried the payloads would be too big? We have around 5-6000 routes, and I am not sure if the payload extractor would work in such case? Perhaps it will? Care to elaborate a bit?
I'll be happy to test it out.

We have around 5-6000 routes

Payload extractor will work as long as nuxt generate works, but I wouldn't use pregenerate-approach in this case. It could take up to an hour to redeploy a new version with this amount of pages...
Maybe you might want to test native implementation as well? It was released in beta (check nuxt-edge package) and it doesn't support vuex as well, but maybe it will work in your case  ¯_(ツ)_/¯. https://github.com/nuxt/nuxt.js/pull/6159
Either way, nuxt-payload-extractor will be deprecated eventually.

Thanks for prompt reply. I already use "generateRoutes" to prefetch all routes when running nuxt generate; do you think it would also take extra amount of time in such case?
Thanks for linking to the new version in nuxt-edge, I will check it out. Do you think it's the same, in terms of increased build time when using this approach?
Build already takes around 1500 - 2000 seconds, so I'd prefer to keep it low ;)

Hm, I can't say for sure. It depends on your deployment strategy: in case files are getting loaded one by one, payloads will increase deployment time.
I guess you'll have to test it out. And I think your experience would be useful for other people to see in an article 😉

Thanks @DreaMinder. I'll test it out. I am still not sure why his PR won't work with Vuex. Can you elaborate a bit?

Do you mean PR native-implementation PR in the nuxt repo? I'm not saying it won't work, I've only seen a person (in github issues or discord) that failed to get it working with vuex. But maybe he has a slightly different use-case, so I can't say for sure.

OK thanks @DreaMinder. I'll spin up a staging environment and test it out!

Seems to work great :) https://beta.thejewelleryroom.com


[MAISTRA-2067] [MAISTRA-1528] Fix bookinfo versions to pick up latest version and CVE fixes

Please provide a description for what this PR is for.
This updates our Bookinfo examples to fix several CVEs. This also fixes tests due to the fact that Helm charts were moved as well as disables reliance in gcloud in tests.

Added a hold until after tag of 1.1.11

Added a hold until after tag of 1.1.11

lgtm, thanks for fixing the gcloud issue. did we establish why this is now broken on the 1.1 image?

My guess is that we rebuilt 1.1 for the last time (before removing the 1.1 dockerfiles) without the script you added with the fake gcloud.

lgtm, thanks for fixing the gcloud issue. did we establish why this is now broken on the 1.1 image?

My guess is that we rebuilt 1.1 for the last time (before removing the 1.1 dockerfiles) without the script you added with the fake gcloud.


Add a skill to check if a passage is too ambiguous to support a claim

This is a re-submission of PR 668.
Describe the contribution to the taxonomy
Given a text passage and a claim, determine if the passage is sufficient to conclude the claim. Here we are looking at passages that may be too ambiguous to support the claim, because they

contain unresolved pronouns or pronoun phrases
miss specific entity names or numerical values
use phrases with too many possible interpretations
use subjective language (rather than objective)

This skill is a prerequisite for the ability to generate grounded responses that are free from exaggerated or careless claims.
Input given at the prompt
In one example, we provide the following context to Merlinite 7b:
Passage:

What I saw today was the most wonderful and magnificent thing I have ever witnessed.

Claim:

Today's solar eclipse has made a strong impression on me.

Then we ask Merlinite 7b the following question:
Is Passage sufficient to conclude Claim?

Merlinite 7b wrongly answers "yes" and explains as follows:
Response from the original model
Yes, the passage is sufficient to conclude the claim. The speaker uses the words "wonderful" and
"magnificent" to describe what they saw today, which aligns with the experience of viewing a
solar eclipse. This strong impression suggests that the passage refers to the solar eclipse, making
the claim plausible.

So, for the current model, a suggested plausible interpretation is all it takes to conclude the claim. In other examples, the model tacitly substituted a pronoun in the passage with a specific entity name or let a numerical value stand without any grounding.
Response from the fine-tuned model
(did not have an opportunity to fine tune the model)
Contribution checklist

[x] The contribution was tested with ilab generate
[x] No errors or warnings were produced by ilab generate
[x] All commits are signed off (DCO)
[x] The qna.yaml file contains at least 5 seed_examples
[x] The qna.yaml file was linted and prettified (yaml-validator can do both)
[x] An attribution.txt file in the same folder as the qna.yaml file.

drop 1966cef feat(synthetic data): Add script to generate synthetic data readme (#650)
drop cc381ec Updating org name from instruct-lab to instructlab (#711)
pick 16cb720 Add ambiguity detection skill

@instructlab-bot precheck

@instructlab-bot generate

Hi @mingxzhao , I made the required formatting changes to attribution.txt and added the version line to qna.yaml. Can this PR be merged now? Thanks!

Sorry, no this is e2e-ready state. We need to the end to end run to see if it increases the model's performance before we can merge it. e2e-ready is on hold for the foreseeable future, but when @alinaryan comes back with the numbers when the next run happens we can look at merging if improvement.

Hi! 👋
It’s been a while since you’ve seen any movement on this PR. We haven’t forgotten about you!  We’ve run into some logistical issues, hence this delay. We absolutely want your PR, and being marketed as e2e-ready is still the last stop before we get it into the upstream model.
We are thankful for your patience and ask that you please keep this PR open. As soon as we finish all our behind-the-scenes work, we’ll test the full model against your submissions and, ideally, accept your amazing contribution(s)!
Your Community Maintainer Team.
P.S. if you have any specific questions or thoughts, don’t hesitate to comment on pull request this or email<EMAIL_ADDRESS>and<EMAIL_ADDRESS>and we’ll get back to you as soon as possible.

Hi! 👋
Thank you for submitting this PR. We are ready to do some validation now, and we have a few candidates to see if they improve the model.
We some resources to run these release candidates, but we need you to help us. Can you reach out to me either on Slack (@awesome) or email me at awesomeATinstructlab.ai so I can get you access via ilab model chat?
We can only run these models for a "week" or so, so please reach out as soon as possible and tell me which one is best for you on this PR.


⚠ AGEG website (prod) has degraded performance

In 2944594, AGEG website (prod) (https://www.ageg.ca/) experienced degraded performance:

HTTP code: 200
Response time: 10924 ms

Resolved: AGEG website (prod) performance has improved in 3c63b02.


My custom object _id's are being overwritten by a NitriteID at save

Is there a way I can prevent Nitrite from overwriting my server-assigned object IDs with it's own values?
Thank you

Nitrite object ids are always auto-generated, you should not change or set your own values in _id field. I made this design choice to guarantee correctness in replication.


[Help wanted] Provide a standalone .exe for sherpa-onnx on windows

https://github.com/k2-fsa/sherpa-onnx/tree/master/mfc-examples
provides exe for Windows but it requires users to download the model manually.
The idea is to pre-package the required model files into the exe so that users can just download the exe and double-click to run it, which greatly simplifies users' life.
As for the pre-package, for each required model file, we convert it to a C uint8 array. Since the current C API supports only passing model filenames, we save the arrays into some temporary files and load them on program startup.

done.


Aug 17 - The Design of Microsoft APIs

Event Information

Category
Details

Reactor Topic
Coding, Languages, and Frameworks

Monthly Topic (if applicable)
Powerful APIs and Power Platform

Weekly Topic (if applicable)

Stream Topic
<2-5 words describing the topic>

Stream Date
August 17, 2020

Stream Time
4pm PT

Stream Duration
60 minutes

Title
Keep Your Eye on the Design: How Microsoft Creates APIs

Description
As developers, our reliance on APIs is critical to efficient development. Using company and product API guidelines, Microsoft provides a consistent approach that helps you learn a new API quickly. In this session, a Microsoft API expert who not only designs .NET APIs for Azure but is also on the company-wide design team will walk you through Microsoft’s API development regimen so that the next API you learn will be a breeze.

Speaker
Anne Loomis Thompson

Moderator
Sarah Guthals, PhD

SharePoint ID

Additional Resources:

Event Checklist
10 Weeks Prior

[X] Reactor Content Team (RCT) or Speaker opens a new issue for an upcoming stream filling in the Event Information table. Both RCT and Speaker are marked as assignees.
[ ] RCT and Speaker discuss details of the event in the comments of this issue. RCT adds a confirmed-event label and global-twitch-event tag to the issue and adds event to the Global Live Streams project. Issue will appear in the Upcoming Streams column
[X] Speaker adds a ready-for-marketing-edit label so the RCT can review title and description

8 Weeks Prior

[ ] RCT does a review of title and description and removes the ready-for-marketing-edit label
[ ] RCT moves the issue to the Ready for SharePoint column
[ ] RCT informs the Reactor Marketing Team of the new event information
[ ] RCT provides speaker with assets (e.g. slides, overlays) and the survey data for the event
[ ] RCT moves the issue to the MONTH Streams Prep column (according to the month it will be streamed in)

2 Weeks Prior (Done in /microsoft/Reactors)

[ ] Speaker opens a PR with the new content for the stream on the customer-facing Reactor repository and adds the content-development and global-twitch-event tag to the PR.
[ ] Speaker adds to the same PR a modification to the Streaming README to include their stream information.
[ ] Speaker adds the RCT assigned to the issue as the reviewer when it is ready for review/merge
[ ] RCT reviews, approves, and merges the changes

1 Week Prior

[ ] Speaker and Moderator test setup for streaming (instructions coming soon)
[ ] RCT checks in with the speaker and moves the issue to the Ready to Stream column

During and After Stream

[ ] Speaker and Moderator run the stream
[ ] RCT submits recording to be uploaded to the Reactor YouTube and to be made available on LearnTV
[ ] RCT moves issue to the Stream Complete column

Getting confirmation from Anne on the date.

👋 @KimberlyBolton1 Here is a new session that needs review

edited abstract and sent to content owner for review.

@KimberlyBolton1 - is the edited abstract the one in the issue body above? Or did you send it to @annelo-msft ? I wrote it broad enough for her, but just checking to see if I can help get this into Sharepoint.

@sguthals
the edited version is both in the body above and I emailed it to her for review (before we updated our process to do it all in GitHub). That was yesterday and have not heard back from her yet. since it's already to her in email, I was going to finish the loop as I started it. To summarize, going forward, I'm putting the edited version in the comments only, use GitHub to notify the content owner to review, then upon approval, I move the final version to the body.

@mebaumb
Small tweak to description...final and moved to Ready for Sharepoint.

Published in SharePoint (6/23)

@mebaumb Session was recorded but there were technical issues with latency/audio so Sarah wants to eventually re-record. This was not uploaded to YouTube. Not sure where you want to place this in the publishing pipeline....

This is on YouTube!


Skip dev and stable tests if there is no dependency on Qiskit

What is the expected enhancement?
Skip dev and stable tests if there is no dependency on Qiskit

One option might be include this in project configuration

Adding a var in the ecosystem.json saying qiskit: false if there is no qiskit in the project ?

yes

Closing it for now as we have skip_tests flag which we can use, as we are interested in tests only if there is a dependency


List of all images?

Only rocm/rocm-terminal image is mentioned, but not their versioning, or other images.
rocm/rocm-terminal  works, but there is very little documentation what one can do with it, other than maybe rocminfo, rocm-smi and hipconfig. There are few more nice tools like rocprof, rocsys, but no binaries to actually run, or code samples to compile and run.
Would be REALLY good to have an image with full dev environments, and all examples, so one can compile examples and run them, basically with one or two commands after starting the container.

BTW. Documentation on https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/docker.html also has broken links.
Also it has incorrect title "Install ROCm Docker containers", one does not "install" docker containers.
(THere are also other mistakes and misinformation in that page, but there are already open bugs for this)


Support git-bash on Windows

Instead of seeing the terminal embedded in the browser it open a new git-bash window.
I'm using the MINGW64 version of git-bash
Details

Browser and browser version: Firefox 50.1.0
OS version: Windows
xterm.js version: 2.2.3

Steps to reproduce

In the demo I've substituted cmd.exe with C:\\Program Files\\Git\\git-bash.exe
run npm start

Try setting it to C:\Program Files\Git\\bin\bash.exe
It will not work that great until node-pty is upgraded to 0.6.2.

Actually I'm working on a project which already use node-pty 0.6.2 and indeed, after I've changed the path, it worked. (I've used the demo to verify my issue)
Thank you!

Happy that it worked out! Let's keep this open until we update node-pty.

Done some time ago :smiley:


Questions about CUDA failure at LINE 2707: 2

Hello,recently I'm working to achieve your code.I practiced using Marvin.I downloaded the complete dataset.But I have some problems.And I use the GPU-1070 8G.
Please help to solve this

=====================================================================================================================================
Layers:                                                                        Responses:

dataTrain
data_RGB[4]={1,3,480,640} RF[1,1] GP[1,1] OF[0,0]
data_HHA[4]={1,3,480,640} RF[1,1] GP[1,1] OF[0,0]
label[4]={1,1,480,640}
dataTest
ERROR

Hello! A CUDA error code 2 means that you ran out of GPU memory. One possible fix (aside from getting a stronger GPU) would be to reduce the training batch size (see lines 34 and 43 in train_shelf_color.json. Try something like "batch_size": 6, or smaller.

Thanks!

The problem has been solved：#9


Instructions for a not so experienced user

I am trying to create my own Samba container from your image, but some things on the instruction page are not clear to me, especially the user management.
(forgive the trivial questions!)

In the config.yml file there is the possibility of using a file for the user's password (password_file: /run/secrets/baz_password). In this case, the relevant secrets commands in the docker-compose.yml file is no missing?
I am referring to this page in the official documentation, especially the section 'Use Secrets in Compose'.
The users specified in the config.yml file must also be users of the host system and have the appropriate permissions to access the files to be shared as in a "normal" Samba installation, is that correct?

In this case, the relevant secrets commands in the docker-compose.yml file is no missing?

If you talk about https://github.com/crazy-max/docker-samba/blob/de6120bb99e868d4433f487267889118b5658423/examples/compose/data/config.yml#L11, indeed we forgot to add the secret in the compose file.

2. The users specified in the config.yml file must also be users of the host system and have the appropriate permissions to access the files to be shared as in a "normal" Samba installation, is that correct?

Yes


Added Pictograph Box Export

Added "Save Pictograph" option to the MMO menu. Only enabled when a pictograph is in the pictograph box. Saves to ModLoader/screenshots as pictograph_{Date.now()}.bmp. Potential cleanup to Bitmap.ts (will do if so requested). Synced and/or taken pictographs could automatically be saved (opt-in? Exports cause a noticeable freeze) if desired.
Built pak for testing before merge

Sorry for not remembering to merge this. Thanks!

Np, glad to contribute


Raise an error if parent_release doesn't exist

I did a typo in comment --parent_release in release.sql file. As a result pg_migration didn't upgrade my db to latest version, but I didn't see any errors.
Could you raise an error in this case please?

fixed in 0.4.0


datastore: transaction succeeds, but client returns an error

I've tried to execute a very simple transaction that just creates a database entry if the key doesn't exist yet.
However, even though my operations complete successfully, and the transaction itself is committed too (at least it appears in the datastore), the client API returns a error during call, http status code: 400 The referenced transaction has expired or is no longer valid. error.
Am I doing something terribly wrong, or is this an actual bug in the library?
err := datastore.RunInTransaction(db.ctx, func(tc context.Context) error {
	log15.Debug("trying to obtain database lock")

	key := datastore.NewKey(tc, "TestEntity", "test_name", 0, nil)
	data := struct {
		Text string
	}{}
	if err := datastore.Get(tc, key, &data); err != nil && err != datastore.ErrNoSuchEntity {
		return err
	}

	data.Text = "Bla bla bla"
	if _, err := datastore.Put(tc, key, &data); err != nil {
		return err
	}
	return nil
})
fmt.Println("Transaction result:", err)

Output:
Transaction result: error during call, http status code: 400 The referenced transaction has expired or is no longer valid.

Ok, it seems the error is in datastore.Put, which actually commits the transaction itself. Why would it do that?

[Oh ffs, just deleted by comment instead of an undo].
So, in essence, the bug is that both Put as well as RunInTransaction commits before returning, resulting in a double commit and a completely meaningless result from RunInTransaction. I guess a simple solution would be to add some state variable into the context to verify if it was already committed and skip it in RunInTransaction.
I could do this fairly easily, though it would be a bit hackish. Maybe somebody has a better solution? In the mean time I can proceed working by escaping the last error out of the client function and discarding the result of RunInTransaction, but it's really really ugly.

Fix proposal at https://code-review.googlesource.com/#/c/1721/ :)

From the comments in 1721, it appears that this was fixed in https://code-review.googlesource.com/#/c/1751
Closing.  Please re-open if this is not the case.


Cydia crashing everytime a package is viewed.

Describe the bug
A clear and concise description of what the bug is and what you expected to happen.
This may or may not be related to Unc0ver but I cannot find an issue page for Cydia.
Cydia will crash everytime I enter a page for a package.
To Reproduce
Steps to reproduce the behavior:

Go to Cydia
Find a Package
Click on the package page
Cydia will crash

Screenshots
Just Cydia crashing
Device (please complete the following information):

iOS Version: 13.5
iOS Device: Xs Max
unc0ver Version: 5.0.0

Place an "x" between the brackets if true:

[x] this is a bug others will be able to reproduce
[x] this issue is present with all tweaks uninstalled(except for default packages) or disabled
[ ] this issue is present after a rootfs restore
[x] this issue is present on the latest version of unc0ver

Logs
If applicable, add logs or error messages here.

Hi,
I found a solution on Reddit that worked for me which involves resetting the settings from
Settings> General > Reset > Reset All Settings
This fully solved the issue for me however it does involve wiping your settings so have a backup!
You won't lose any data just settings like Wi-Fi etc. Do not chose 'Erase All Content and Settings' else this will wipe your data as well.

See thread #1991

Solved! @Oscillate18 ‘s solution worked.


No EPG

Hi.
I can't see EPG in the demo version or in my app in web browser. Am I doing something wrong?
Thank you

Hi,
there is currently no support for EPG In the browser version.


Non Contaminated Packing

As discussed in #1230 , I've done a quick & dirty implementation. And also included a sample notebook(not tested).
Will test when I can. Or if you have time pls feel free to test and also any feedback on code changes. Putting this as draft for the moment.
Thanks!

cc @younesbelkada @lvwerra

Note it only works with flash attention enabled.

getting this error
ValueError: The model did not return a loss from the inputs, only the following keys: logits. For reference, the inputs it received are input_ids,attention_mask.

colab here, not sure what I'm doing wrong. @younesbelkada could you pls take a look

Thanks @khai-meetkai for fixing the bug, training loop runs on colab, however since t4 doesn't support flash attention, cant test loss. Will test on aws cluster tomorrow and then mark as ready.

Thanks @khai-meetkai for the fix! @nivibilla I left a comment, wdyt?

@younesbelkada afaik the non contaminated packed dataset only works with flash attention and the monkey patching for the attention is necessary to make use of the dataset.
As an alternative, maybe it can be a separate script/tutorial notebook to let users use the new packing? Since it does improve training speeds by a lot. 5-10x in my case.

Does decontamination also handle the positional encoding? Meaning, will each packed sequence starts form 0?

Does decontamination also handle the positional encoding? Meaning, will each packed sequence start form 0?

Thanks to ROPE positional embedding, the position is relative. So the attention score between token at position i and token at the position j only depends on (j - i), so not necessarily starting from 0.
In a packed sequence of sequence 1 and sequence 2, we use extended attention_mask to indicate that tokens from sequence 1 only attend to tokens from sequence 1. There won't be cross-attention between sequence 1 and sequence 2.
You can read more information from: https://github.com/MeetKai/functionary/tree/main/functionary/train/packing

@khai-meetkai regarding @younesbelkada 's comments. Is it possible to have a non Flash attention version? And without the monkey patch?

@khai-meetkai regarding @younesbelkada 's comments. Is it possible to have a non Flash attention version? And without the monkey patch?

Non Flash-attention is possible but without monkey-patch is not possible. The reason is the current code only accepts attention_mask containing: 0, 1 when we pack sequences; we don't know which tokens are from which sequences to avoid contaminated cross-attention.
For example: sequence 1 = [1, 2, 3]; sequence 2 = [4, 5, 6, 7]
packed sequence = [1, 2, 3, 4, 5, 6, 7, pad_id, pad_id] (Here assume that we add padding tokens (pad_id))
current attention_mask = [1, 1, 1, 1, 1, 1, 1, 0, 0]
Our extended attention_mask=[1, 1, 1, 2, 2, 2, 2, 0, 0] --> our monkey-patched version handle this attention_mask

@khai-meetkai regarding @younesbelkada 's comments. Is it possible to have a non Flash attention version? And without the monkey patch?

Non Flash-attention is possible but without monkey-patch is not possible. The reason is the current code only accepts attention_mask containing: 0, 1 when we pack sequences; we don't know which tokens are from which sequences to avoid contaminated cross-attention.
For example: sequence 1 = [1, 2, 3]; sequence 2 = [4, 5, 6, 7]
packed sequence = [1, 2, 3, 4, 5, 6, 7, pad_id, pad_id] (Here assume that we add padding tokens (pad_id))
current attention_mask = [1, 1, 1, 1, 1, 1, 1, 0, 0] --> contaminated cross-attention
Our extended attention_mask=[1, 1, 1, 2, 2, 2, 2, 0, 0] --> our monkey-patched version handle this attention_mask

Makes sense. Unless the the way the attention works in HF itself is changed this is not possible.

What about this idea @younesbelkada @khai-meetkai ?

Merge the patched attention into transformers assuming it doesnt break anything else. Or at least have an option to load with the patched attention.
Modify the code for the packing so that it works regardless of flash attention is enabled or not

(optional, we can otherwise force users to enable FA if one wants to use the new packing, however this would limit the use of this on older hardware such as colab's t4 gpus.)

Finally merge the non contaminated packing code into TRL

Hi @nivibilla @khai-meetkai
Hmmm I don't think we'll be able to merge this if it depends on monkey patching, I thought there was a way to detect non-contaminated packing with classic attention mask. If you can find a workaround (maybe using a custom DataCollator) we can definitely merge this feature request but as currently it depends on monkey patched versions of modeling files that is not possible unfortunately. We cannot also add that patch to the current transformers code base as it is too specific to this usecase

@younesbelkada makes sense. Can close this pr in that case.

I know people have asked if Unsloth could include packing as a stopgap measure to bridge the gap to TRL, so if it becomes a more hotly request feature, we can add it in :)


values_files.md indicates null for removing a key

This is a documentation issue about values_files.md.
Section Deleting a default key indicates that you should set a value to null to delete a key. There is also an example about it:
helm install stable/drupal ... --set livenessProbe.httpGet=null

Doing a test, it seems to set the variable to the literal string "null".
When setting it to the empty string, I noticed that I could trigger a default "..." pipeline function, maybe indicating that an empty string should be used instead of null in the description and in the example.
Output of helm version:
Client: &version.Version{SemVer:"v2.8.1", GitCommit:"6af75a8fd72e2aa18a2b278cfe5c7a1c5feca7f2", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.8.1", GitCommit:"6af75a8fd72e2aa18a2b278cfe5c7a1c5feca7f2", GitTreeState:"clean"}

hey @polettix! @bacongobbler and I were able to reproduce this issue and we pushed/merged a fix for it in #3912. Thank you. Let us know if you have any other issues.

Thanks @michelleN @bacongobbler  for taking care of it! I'll report further issues if I find them, thanks.


review comments on "Translate ServiceList to yaml"

TypeApprovalRegister+services.yaml:

[x] since /v1/start-application-in-generic-representation is moved to basic services and the uuid is changed from tar-1-0-0-op-s-3000 to tar-1-0-0-op-s-2000, the uuid needs to be updated in TypeApprovalRegister+data.json and TypeApprovalRegister.yaml files for /v1/start-application-in-generic-representation
[x] In this file,  ExecutionAndTraceLog://v1/record-service-request, OamLog://v1/record-oam-request and AdministratorAdministration://v1/approve-oam-request can be moved from "basic management" to "basic services".
[x] ApiSegment of RegistryOffice://v1/inquire-application-type-approvals, RegistryOffice://v1/notify-deregistrations , RegistryOffice://v1/regard-updated-approval-status and RegistryOffice://v1/end-subscription shall be "im". If this proposal is agreed, can we move them from "individual service" to "individual management"

TypeApprovalRegister+forwardings.yaml:

[x] In Forwardings.yaml file, few values are empty (or whitespaces), which implies "not applicable"(eg. fc-port-deletion: empty value indicates that it is not applicable)
But for few cases, eg., "initialting-requests" for OamRequestCausesLoggingRequest will be applicable for "all Oam requests". Keeping this value empty would imply that there is no initiating request for this forwarding. please let me know if we can include a comment like [all oam requests] in this regard.
I understand that we could not include all Oam request in load-file for the forwarding, just curious if we could include it in forwardings.yaml file for documentation or understanding purposes.
[x] since RO://v1/regard-updated-approval-status advices RO to configure forwardings and its corresponding callbacks, isn't this service come under "im". If so, tar-1-0-0-op-c-is-ro-1-0-0-002 in line 494 and 513 to be updated as tar-1-0-0-op-c-im-ro-1-0-0-002

Thanks a lot for the profound feedback.
Made all changes as proposed.


Replace hardcoded ATOM conversion

Before uatom was converted to ATOM using float math. This brings two improvements:

Use decimal math, avoiding any rounding
Generalize to any denom. Those starting with u are converted automatically by multiplying with 1_000_000. Other denoms are printed directly.

thanks for the PR!

Thanks!


Create a WIP pull request template

What you added/changed:
A pull request template
Why you did this:
To make pull requests faster to close/merge, as there is less time needed if you know what the PR changes.
Is this currently good enough to merge?
No, it probably needs more improvement.

something like this?


[WIP] [SR-2473] Support non-linked Homebrew packages pkgconfig

Some Homebrew packages cannot be linked, like OpenSSL. This patch adds support for packages that are not linked. It will query Homebrew about the installed version of that package, and add an additional pkg-config search path.
There was a discussion about this being a fallback. However Apple ships /usr/lib/pkgconfig/openssl.pc, so the fallback wouldn't be triggered.
Review questions:

brew is assumed to live on PATH -- is that ok? Otherwise, we might try the default location and use an environment variable (eg BREW_PREFIX) as fallback?
If the brew command could not be found (or produces an error), should that issue a warning on the console?
How can we create tests for this? We shouldn't assume brew being available or certain packages to be installed.
If the JSON could not be parsed, should that issue a warning on the console?

I don't think we should assume people have brew installed, so us not being able to find it should be silent.

In the Jira issue, @aciidb0mb3r proposes to use the brew info NAME --json=v1 output. In my first version I did just that, however a user might have multiple versions of a library installed. This results in multiple versions being listed in the json output. However the json doesn't mention the version that the user has selected using brew switch NAME VERSION.
In the commit I just added, I've removed the JSON parsing. Homebrew maintains symlinks under /usr/local/opt/(NAME)/lib/pkgconfig which will point to the correct version. This makes this implementation even more robust, as it doesn't have to parse the json.
There aren't any tests related to brew at the moment (besides json-serialization of Package). The code is quite trivial at the moment, but tests are always a good idea.

LGTM

@swift-ci please test

@swift-ci please test

Looks good to me.

Ok, great! I've removed the "WIP" tag from the title.


Auto-cuts should be stored and take parameters

Original report by Joseph Fowler (Bitbucket: joe_fowler, ).

See @oneilg 's comments on commit 610036e. He thinks we should:

Allow the cut levels (in "sigma units") to be function arguments.
Return the actual cuts, and also store them in some way.
Compare the pretrigger_rms distribution's peak from noise to pulse data. But I'm not sure what this means.

Original comment by Joseph Fowler (Bitbucket: joe_fowler, ).

Store auto-cuts values to HDF5 and restore from HDF5.
Tests for store/restore action. Fixes #104.

Original comment by Joseph Fowler (Bitbucket: joe_fowler, ).

Removing milestone: v0.6 (automated comment)

Original comment by Joseph Fowler (Bitbucket: joe_fowler, ).

Remove item 3 from this issue, and move it. See #114, which we'll work on for a later version of MASS.  I've already done item 1 (see be6c720).

Original comment by Joseph Fowler (Bitbucket: joe_fowler, ).

I think we might want to work towards items 1 and 2 in milestone 0.6 and save the 3rd one (comparison of noise and pulse data) for later.


Mover.io - Box to M365 - File version migrated?

Hi,
Could you add some info as to if file versions in Box are migrated to SharePoint Online/OneDrive for Business please?
Thanks,
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e5366e6b-779b-29d6-1e06-a30963c4535d
Version Independent ID: df5eaa56-f7a2-46a1-4852-2fdf70b7699a
Content: Mover migration - Setting up your Box source connector - Migrate to SharePoint and OneDrive
Content Source: migration/mover-box.md
Service: sharepoint-online
GitHub Login: @JoanneHendrickson
Microsoft Alias: jhendr

@haneef95 Thank you for submitting feedback and contributing to the docs. We are currently investigating this.

Hello @haneef95
I think this answers your question
https://docs.microsoft.com/en-us/sharepointmigration/mover-migration-faq#does-mover-preserve-file-versions
Thanks

@haneef95 Thank you for submitting feedback. We understand that this issue has been resolved.
Please feel free to re-open this issue if there is a specific area of the docs that we can improve or make better. Thank you.


TODO: Notes text field

Notes for keeping track of minor issues or known spots to pay extra attention to during QC
Add to a hidden tag that dissapears during print time


username enfageorge is added

hey @dujm !
is approving PRs related to them getting merged or is it only a review step?

Hi @Aya-S, good question ;) If I understand right, we should try to have two reviewers for each PR. That's why I only do the Merge whenever I'm the 2nd reviewer.
But I could be wrong. What would you suggest, Marc? @datapythonista

@dujm I might be wrong but as far as I understand not everyone can merge PRs

@dujm I can't even find the to the option to approve or ask for changes open for me

@Aya-S  I think that's because you are not in the organization member list. (That's also one reason why we need to add our accounts in theREADME)
Marc said we should be able to add each other. Let's try. If you could see this member page, could you click Request to join on the top right?

Great team work here!
Regarding the approvals, it's a convention, and we can choose our own. The approval is just like a comment saying you're happy to get this merged, but it's more visible and easy to see. And also, github can be set up to allow only merged in PRs with more than N approvals.
If just want person is reviewing and merging it's quite easy that doesn't see something, and merges things that could be improved. I merged a PR adding a username with a typo, which wasn't rendering correctly previously. Having people approving things is great, we can decide how we want to work in this sense. May be someone can create an issue to discuss how many approvals do we want, and whether we want to require the approvals on the github settings, or we don't, in case we want to be flexible and allow exceptions.
Regarding the permissions, everybody should have permissions to merge eventually. I gave to few people already, and didn't give to the rest because didn't have the time. If the people with permissions can add the rest to the organization (not necessarily all at once), that would be good. Otherwise will check why not, and what I need to do so people have permission to do that.

@dujm  can't open the page :D
@datapythonista  thanks for the clarification! We can take our time with this propagating merge permissions. we're still getting the hang of this. I will open the issue about the number of reviewers and review conventions.

Hi @enfageorge, there's a conflict in the README.md . Would you like to practice solving conflict from command line? Then I will merge your PR directly (no need to have a 2nd  review, I think).

@dujm is it okay now?

@enfageorge Perfect, thanks!


[RFC] Align west --test-item with twister workflow

Is your enhancement proposal related to a problem? Please describe.
Sample/testcase.yamls contain build configuration for twister tests. It is required that a user could build a given configuration without twister but with west. A west-command --test-item was added to serve this purpose. However, this method doesn't align with twister's behavior and it is not warranted that it will work, or worse: a user will get a different build without realizing it. In recent month several issues were reported regarding --test-item:
#52614, #56248, #59761, #60297
Describe the solution you'd like
IMO a single source of truth should be used when building via twister and west. Since west should be usable without twister, and twister already requires west, I think the source can be in west. When building, twister will directly call west and no scripting for building will be present in twister. With the direct call (through cli) we will assure that calling this command by user (without using twister) will give the same result.
Describe alternatives you've considered

A west module is loaded into twister and the building is called with available west methods. IMO it can be less clear for a user to reproduce
we do the aligning in west and keep the code for building both in twister and in west. However this will be problematic to keep in sync as it was already seen in reported issues. We then would have to provide tests verifying if the same output was achieved with west and with twister, which can end up in a cat and mouse game.

Additional context
twister does several steps during the build stage, which are not mirrored in west. Some missing pieces are easier to find. E.g. west ignores the common section of yamls. In addition twister compiles extra args and konfigs from multiple places into a single list https://github.com/zephyrproject-rtos/zephyr/blob/main/scripts/pylib/twister/twisterlib/config_parser.py#L205 in a certain order. This proces is not reproduced in west and can cause harder to catch mismatches in build products.

cc @nordicjm @nashif @hakehuang

@PerMac , we can use  west twister ... , i remember this has been supported by google team @aborisovich

west twister is only a west wrapper over twister, i.e. west directly calling twister. However, west --test-item doesn't use twister. It suppose to be a mean to build/run with configurations described in .yaml files, which are native to twister.

west twister is only a west wrapper over twister, i.e. west directly calling twister. However, west --test-item doesn't use twister. It suppose to be a mean to build/run with configurations described in .yaml files, which are native to twister.

in this case, you will have to re-implement those already in the testcase.yaml right?

west --test-item was added as a convinience option. The yaml file is very twister specific and we should continue to maintain building and parsing this file in twister directly. Single source of truth is nice, but splitting twister functionality across different tools is going to be counter productive and I do not see any significant benefit.
So, I am -1 on this proposal.

I see your point. But in such form I don't see west --test-item as a convenience. Rather a dangerous cumber. Cumber, since we will constantly have to maintain it and try to keep close to twister workflow, and dangerous since (as linked issues showed and since there are still some not solved issues, like handling common section) can produce different results and give false impression that something is fine/wrong.
What is the alternative then? Leave the status quo with not aligned functionality?

What is the alternative then? Leave the status quo with not aligned functionality?

The alternative is to just focus on what west needs and implement it directly in west like we have right now. There is no need to move all of this logic into west from twister and create more problems along the way.

@nashif

The yaml file is very twister specific and we should continue to maintain building and parsing this file in twister directly.

What about the other way then? I.e. west --test-item using twister library?

What about the other way then? I.e. west --test-item using twister library?

that makes more sense, but then it is about whether west wants to depend on another library.

--test-item option is very useful and can't be replaced with just running twister. A developer can build any scenario directly and does not need to worry about twister internals, i.e. I could debug, do incremental changes and rebuild etc. etc., using the same development workflow for everything.
The only thing this option needs to worry about are options related to building, nothing else. So filtering and other twister options are out of the question. The option is there to just build the scenario in a similar way as with twister. The option itself has been working great for majority of the tests, recent additions such as sysbuild however or more use of common for build options might have caused issues, but for the most part it works.

I consider this done, no need to keep this open.


AP-2197: Update privacy policy content

What
Link to story
Describe what you did and why.
Checklist
Before you ask people to review this PR:

[ ] Tests and rubocop should be passing: bundle exec rake
[ ] Github should not be reporting conflicts; you should have recently run git rebase master.
[ ] There should be no unnecessary whitespace changes. These make diffs harder to read and conflicts more likely.
[ ] The PR description should say what you changed and why, with a link to the JIRA story.
[ ] You should have looked at the diff against master and ensured that nothing unexpected is included in your changes.
[ ] You should have checked that the commit messages say why the change was made.

Codecov Report

Merging #2469 (0408f13) into master (112d56f) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##            master     #2469   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files          506       506
  Lines         9869      9869
=========================================
  Hits          9869      9869

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 112d56f...0408f13. Read the comment docs.


Simplify xdebug configuration, fixes #736, fixes #735

The Problem/Issue/Bug:
Docker now supports "host.docker.internal" IP address/hostname, so we don't have to do magic configuration of the host IP address as we used to. See OP #736 .
How this PR Solves The Problem:

[x] New nginx-php-fpm-image drud/nginx-php-fpm-local:20180412_xdebug (Must be released, tagged, and changed here before pull) https://github.com/drud/docker.nginx-php-fpm-local/pull/67
[x] Simplify configuration of xdebug on/off (enable_xdebug: true in config.yaml)
[x] Update docs to reflect xdebug_enabled
[x] Update xdebug usage docs with new screenshots
[x] Add vscode instructions.
[x] Try xdebug.remote_connect_back (doesn't work)
[ ] Update the Stack Overflow article

Manual Testing Instructions:

Use the updated xdebug instructions to set it all up with your favorite IDE.
Test with xdebug_enabled: true and xdebug_enabled: false.

Automated Testing Overview:

Added a test to ensure xdebug_enabled: functions correctly.

Related Issue Link(s):
OP #736
#735 Use xdebug with vscode on windows
Release/Deployment notes:

I think this is ready for review, along with the related drud/docker.nginx-php-fpm-local#67

Definitely see that I failed to update the atom screenshot, although the config.cson snippet did get updated, https://github.com/rfay/ddev/blob/3ed95b8ed8822e6837551d8a4f025f013dc50729/docs/users/snippets/atom_config_cson_snippet.txt -
I think both should omit the docroot, at least on other IDEs it doesn't matter (the mapping just needs to be consistent, and might as well be repo root)

I took this for a spin testing against vscode, and hit some snags getting it to work. Re-testing via screenshare w/ Randy, we determined that we needed to set ?XDEBUG_SESSION_START on the test URL in order for vscode to pick up the breakpoint.
Looking at the docs for the vscode php-debug plugin, it looks like they recommend setting the following for the simplest path to working state:
[XDebug]
xdebug.remote_enable = 1
xdebug.remote_autostart = 1

I see we have remote_enable set, but I didn't see remote_autostart set. I wonder if this is the piece we're missing for this to work seemlessly in vscode and possibly others. While I didn't test phpstorm myself, I did observe it to be working as expected when randy demoed via screenshare.

I added the remote_autostart, which is lovely, and also will help both vscode and atom to work without a browser extension or anything.
Updated instructions for vscode and atom, with screenshots. See docs that go with this PR
Removed Netbeans instructions. I couldn't even get the latest version to open a project, not to mention debug. I haven't heard of anybody using Netbeans in recent years.

When retesting this,  Please make sure to use the 20180420_xdebug web image (just delete webimage line from config.yaml)

Neat that this is now possible without the browser extension. I can confirm that I can disable the extension and still see the session initiation when xdebug is set to true and I've configured Atom to toggle on debugging. I'm still not able to get a breakpoint to succeed and I've done this before, so I'm not exactly sure what's tripping me up. I viewed the instructions here https://atom.io/packages/php-debug to see if it's user error. I'd like to request a quick video/screencast to determine if I'm simply doing something incorrectly or to determine if my install/configuration of Atom is borked.

Yeah, we should work through it together. Do make sure your path mappings are correct.


feat: support additional "migration revision" if collection already has an open revision during a migration

Reason for Change

#6930

Changes

Add is_migration_revision flag to CollectionVersion Table in DB + CollectionVersion entity

Updated persistence layer add_collection_version to support is_migration_revision

Updated create_collection_version in business.py to allow creating second revision, ONLY IF is_migration_revision arg is True AND the canonical collection does not already have a revision with is_migration_revision set to True
- otherwise, throw errors
- default arg for is_migration_revision is False, such that multiple revisions are not allowed by default
- only pass is_migration_revision is True when create_collection_version is called from schema_migrate.py in the "collection_migrate" step. create_collection_version is called when processing a published collection passed on from the "gather_collections" step.

Update "gather collection" logic in migration to:
- only pass on published collections to next step ("collection_migrate") IF published collection does not have a revision open OR if it has an open revision with is_migration_revision == False. Otherwise, the published collection is filtered out of the output (i.e. we do not process collections that already have revisions-in-migration in "collection_migrate")
-pass pre-existing revisions with is_migration_revision == False into "collection_migrate", with auto_publish set to False. This allows pre-existing revisions to be migrated without being published.
- pass pre-existing revisions with is_migration_revision == True into "collection_migrate", with auto_publish set to True. This scenario occurs if we are running the same migration more than once (e.g. due to an error or cancelled run). This allows us to continue migrating revisions that were opened during previous migration runs.

Update publish_revisions to only publish revisions with is_migration_revision == True. This avoids having to pass in a denylist, and it will only publish revisions created during the migration run.

Proposed Flow using is_migration_revision:

Testing steps
TODO: add unit tests
Checklist 🛎
Notes for Reviewer

The code looks fine, but for documentation sake, could you add a flow chart or something detailing the lifecycle of a revision, migration_revision, and published collection.

Can do

Shouldn't we set is_migration_revision to false or None after publishing?

I don't think so, it'd be potentially useful metadata to track whether a version was created by migration. Even if its not, once its got a published_at, it won't interfere with any checks involving is_migration_revision, since all of those only interact with unpublished versions. So it'd be neutral at worst to keep it.

When we open a revision on a collection that has the is_migrated_revision flag, will that flag be copied as well, and should it continue to be True?

I think no, it should only be True if the revision is opened as part of a migration.
Now that you bring up that scenario though, it does make me reconsider whether we should just re-set it to False once a migration revision is published. I could go either way

to me it makes more sense to set the is_migration_revision to false in the context of a a migration as opposed to when a curator is start a revision outside of a migration senario.

to me it makes more sense to set the is_migration_revision to false in the context of a a migration as opposed to when a curator is start a revision outside of a migration senario.

when a curator is starting a revision outside a migration scenario, we create a new collection version. So we're not re-assigning any values, its basically just metadata indicating whether a particular collection version was first created by a migration run or not.

add_collection_version does not copy over all of the fields. A new collection version will have a default is_migration_revision of False 👍.

The is_migration_revision flag is intended to be a temporary flag used during migrations, yet it will persist even after the migration is complete. Outside of a migration the flag being true could be confusing since the published collection is no longer considered a revision. False could also be potentially confusing. Can we set it to none after publish?


Fix migrations in run.sh

Hi
I tried to run this in a docker container and for some reason the DB creation didn't happen.
I tried to run the run.sh script manually and got the following error:
! unknown or unexpected option: --schema

Turns out the order of the arguments is (now?) enforced. Changing the order of the arguments was enough to make it work :)
I also added set -e, which makes the script exit if the migration doesn't work.

Thanks for the PR! That was weird, might have messed up the script while I was fixing the Docker image.


BE crash when load data with stream load  in shared data

be crash

Steps to reproduce the behavior (Required)

generate data 1T with tpcds
bin/stream_load.sh

Expected behavior (Required)
Real behavior (Required)
StarRocks version (Required)

You can get the StarRocks version by executing SQL select current_version()

the crash node is be or cn？


[ISSUE-2442][linkis-basedata-manager] engine config template manage

What is the purpose of the change
At present, the new engine needs to manually enter sql to prepare the engine parameters. This submission is the basic data management module, and the related interfaces for the configuration management of the engine parameters are added:

Query the corresponding configuration item template according to the selected application and engine
/api/rest_j/v1/configuration/getConfigurationTemplateByAppName
Add/modify configuration item template interface
/api/rest_j/v1/basedata-manager/configuration-template/save
Delete configuration item template interface
/api/rest_j/v1/basedata-manager/configuration-template/{keyId}

目前新增引擎需要手动录入sql进行引擎参数准备，该次提交是基础数据管理模块，新增引擎参数配置管理的相关接口：
1.根据选中的应用和引擎查询对应的配置项模板
/api/rest_j/v1/configuration/getConfigurationTemplateByAppName
2.新增/修改配置项模板接口
/api/rest_j/v1/basedata-manager/configuration-template/save
3. 删除配置项模板接口
/api/rest_j/v1/basedata-manager/configuration-template/{keyId}
Related issues/PRs
Related issues: #2442
Related pr:#591
Brief change log

Define the core abstraction and interfaces of the EngineConn Factory;
Define the core abstraction and interfaces of Executor Manager.

Checklist

[x] I have read the Contributing Guidelines on pull requests.
[ ] I have explained the need for this PR and the problem it solves
[ ] I have explained the changes or the new features added to this PR
[ ] I have added tests corresponding to this change
[ ] I have updated the documentation to reflect this change
[ ] I have verified that this change is backward compatible (If not, please discuss on the Linkis mailing list first)
[ ] If this is a code change: I have written unit tests to fully verify the new behavior.

Need to supplement junit test


Adding extras_require

We have a TODO flagged here for install_requires. Also cupy and other dependencies are not currently listed. It would be worthwhile to update these particularly for package formats relying on this metadata to be correct
https://github.com/rapidsai/cucim/blob/9f2338ed5c633df7dca29e56f90564cdfcea9756/python/cucim/setup.py#L86-L90

TODO: Adding cupy as extra_requires.

Based on discussion in today's meeting, there are use cases that rely on parts of cuCIM where CuPy or other dependencies are not needed. Given this, we think the right approach is to add these to extras_require under some extra name.

This was resolved in PR: https://github.com/rapidsai/cucim/pull/617


Segmentation fault for acknowledge_cumulative

Describe the bug
Trying to use the consumer acknowledge_cumulative(msg) functionality. The python shell stops with Segmentation fault (core dumped). Expected is to ack the messages until the current message in the stream or at least to have error handling available to the python interpreter.
To Reproduce
1、Test Conditions

Setup a python client
Subscribe to a topic
Consume 1 or more messages
Call consumer.acknowledge_cumulative(msg) or consumer.acknowledge_cumulative(msg.message_id()) with the latest message received.

2、Test code
import pulsar
from _pulsar import InitialPosition
from pulsar import Client, ConsumerBatchReceivePolicy, ConsumerType

pulsar_client = Client(pulsar_host, authentication=authentication_provider.get_authenticator()) # This is a AuthenticationToken
consumer = pulsar_client.subscribe(
  topic=topic,
  subscription_name=consumer_id,
  initial_position=InitialPosition.Earliest,
  consumer_type=ConsumerType.KeyShared,
)

# Given some message on the topic
message = consumer.receive()
print(message.data()) # b'{"test": "test6"}'
message = consumer.receive()
print(message.data()) # b'{"test": "test8"}'

consumer.acknowledge_cumulative(message)

error log
Starting shell with python3 -q -X faulthandler
Fatal Python error: Segmentation fault

Current thread 0x00007f22216e0000 (most recent call first):
  File "/home/ubuntu/.local/lib/python3.10/site-packages/pulsar/__init__.py", line 1300 in acknowledge_cumulative
  File "<stdin>", line 1 in <module>

Extension modules: charset_normalizer.md, _cffi_backend (total: 2)
Segmentation fault (core dumped)

The cumulative acknowledgment cannot be used for Key_Shared subscription. I think we need to enhance the error processing at the Python client side. We should return an error instead of crash.

BTW, I cannot reproduce this issue with the following code with Python client 3.1.0 and Pulsar standalone 2.11.0.
from pulsar import Client, ConsumerType, InitialPosition

pulsar_client = Client('pulsar://localhost:6650')
topic = 'my-topic'

producer = pulsar_client.create_producer(topic=topic)
producer.send('msg-0'.encode())
producer.send('msg-1'.encode())
producer.close()

consumer = pulsar_client.subscribe(
    topic=topic,
    subscription_name='sub',
    initial_position=InitialPosition.Earliest,
    consumer_type=ConsumerType.KeyShared,
)

msg = consumer.receive()
print(f'{msg.data()} | {msg.message_id()}')
msg = consumer.receive()
print(f'{msg.data()} | {msg.message_id()}')

consumer.acknowledge_cumulative(msg)

pulsar_client.close()

Output:
b'msg-0' | (5,0,-1,-1)
b'msg-1' | (5,1,-1,-1)

The cumulative acknowledgment just fail without any exception thrown.

Thanks for checking.
Improving the error handling python side sounds good and perhaps a docs update (https://pulsar.apache.org/api/python/3.1.x/pulsar.Consumer.html#acknowledge_cumulative).
On the reproducibility, perhaps the environment makes a difference, error was seen on Ubuntu 22.04, python 3.10.6

I just rechecked the code and found acknowledge_cumulative uses asynchronous version of the ACK API.
https://github.com/apache/pulsar-client-python/blob/cf4a9c0572e7232b315179f78c28d4c653f9119c/src/consumer.cc#L53
So no exception should be thrown even if it failed.
I tested the code here inside a ubuntu:22.04 container and only modified the service URL to pulsar://host.docker.internal:6650 and it worked well.


init

@ashwintantri   thanks for making your first contribution, I will be merging your Pull Request, also look at the issues so that you can get a chance to win a limited community Swag from me.
If you like the project and Repo please give it a Star and Follow me on Github (would be appreciated)


ignore comment lines

my csv files start with comments that I'd like to ignore, e.g.
-- 1. Column 1 description
-- 2. Column 2 description

Column 1, Column 2
a1,b2
b1,b2

I'm new to neovim plugin development, so I'm not sure how to test my changes locally. Interested in guidance and feedback :)

Thank you for the PR!
For testing, we use plenary.nvim. Please write your tests and follow Plenary.nvim's test guide, then run them with the
:PlenaryBustedDirectory command.
Additionally, it would be great if the comment pattern (in your case, --) could be configurable via settings, allowing more flexibility for different users. I recommend considering this as a feature.
Feel free to reach out if you have any more questions!

I second adding the comment prefix as an option. I've also seen # used, and I expect // might be wanted as well.

@CoreyCole
@bleonard252
Thank you very much for the pull request and for your interest in contributing to the project! As we hadn’t received an update for some time, I went ahead and implemented similar functionality to ensure we could continue moving forward (#11). I’d love to discuss it further or explore other areas where we could collaborate if you’re interested!


Add support for SNK Neo Geo Arcade Stick Pro on Mac OS X

Add support for SNK Neo Geo Arcade Stick Pro on Mac OS X.
Product page: https://www.snk-corp.co.jp/us/neogeoarcadestickpro/

Thanks, would love to have Windows and Linux as well in the future if you're able 👍

Thanks for merging the PR!
My only computer is a Mac right now, so I cannot map the stick for Windows and Linux 😔


Refactor/update ktor server proguard simple

Update the Ktor Server Proguard example:

Avoid using ignorewarnings and dontwarn, instead fix the warnings by configuring the Kotlin standard library and the required Java modules in Proguard configurations, this will not only fix warnings but avoid many bugs and issues that are hard to debug as the message and errors can become quite unclear because Proguard doesn't have access to the used Java modules that are part of the JDK, for more details, see this comment
Include the official rules of Kotlinx serialization (https://github.com/Kotlin/kotlinx.serialization/blob/master/rules/common.pro) and coroutines (https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-core/jvm/resources/META-INF/proguard/coroutines.pro)
Update the Proguard rules to get the example working without using ignorewarnings to fix many bugs, I believe this new version of Proguard rules can still be improved further. The rules are configured to build the project without any issues for the following libraries:

implementation("io.ktor:ktor-server-auth-jvm")
    implementation("io.ktor:ktor-server-core-jvm")
    implementation("io.ktor:ktor-server-webjars-jvm")
    implementation("org.webjars:jquery:3.2.1")
    implementation("io.github.smiley4:ktor-swagger-ui:2.9.0")
    implementation("io.ktor:ktor-server-resources-jvm")
    implementation("io.ktor:ktor-server-host-common-jvm")
    implementation("io.ktor:ktor-server-status-pages-jvm")
    implementation("io.ktor:ktor-server-cors-jvm")
    implementation("io.ktor:ktor-server-caching-headers-jvm")
    implementation("io.ktor:ktor-server-compression-jvm")
    implementation("io.ktor:ktor-server-call-logging-jvm")
    implementation("io.ktor:ktor-server-content-negotiation-jvm")
    implementation("io.ktor:ktor-serialization-kotlinx-json-jvm")
    implementation("io.ktor:ktor-server-websockets-jvm")
    implementation("io.ktor:ktor-server-auth-ldap-jvm")
    implementation("io.ktor:ktor-server-netty-jvm")
    implementation("ch.qos.logback:logback-classic:$logback_version")

I haven't tested all the functionality for each one though, fixing the warnings can catch some common bugs though.
5. Update Proguard to 7.5.0 to support Java 22, I suggest updating the Gradle wrapper, Kotlin, also the versions are hardcoded for all used tools like Ktor Gradle Plugin, I suggest using Gradle version catalogs gradle/libs.versions.toml
6. Use the official Proguard https://github.com/Guardsquare/proguard/blob/master/examples/application-kotlin/proguard.pro example as a reference
7. Create a new task called runMinimizedJar and replace minimizedJar with buildMinimizedJar to maintain consistency with buildFatJar and runFatJar
8. Avoid hardcoding the name of the minimized JAR file
9. Tested with Java 11, 17, 21, and Kotlin 2.0.0, Gradle 8.8, and tested with the version tool versions in this code snippet.
10. Group both buildMinimizedJar and runMinimizedJar in ktor:

to be along with buildFatJar and runFatJar
I suggest renaming the proguard module to ktor-server-proguard or server-proguard or something similar to make it clear this is for the Ktor server and won't work directly for Ktor Client, notice that you will need to include more rules depending on the engine and the libraries, for example, If you're using OkHttp as the engine in Ktor client, you will need to include the rules from (https://square.github.io/okhttp/features/r8_proguard/) and Okio as well as OkHttp depends on it.
Not related:
I noticed Ktor somehow depends on Java Swing/Desktop, all imports from org.mozilla.javascript.tools.debugger use it, while this may sound not an issue, it could cause some issues, like making the bundle size bigger when using jlink to include the required Java modules only in the application instead of fully installing Java on the machine, an easy solution is to split this functionality into a different module and make it optional to use.
While this example is referenced in Ktor docs, it's also applicable in the ktor-samples repository.
Feel free to share your suggestions or request any changes.

That's really nice revamp all proguard configuration! Thank you for the PR!
LGTM

I believe it can be improved further before merging, for example:

We could use Paths and JavaVersion to achieve some functionalities.
Instead of copy and paste the rules of Kotlinx coroutines and serialization manually, it will be already included in META-INF/proguard in the Jar regarding if we're using Proguard or not, we could use them so in case we update the libraries, the rules will also updated if there is any changes and will be included.

Let me know if you're interested in those changes to get them into the example so I update the PR.

I agree. It would be great if you could log an issue for that

I agree. It would be great if you could log an issue for that, so it will not be lost

Solved.

Regarding the note of using JavaVersion instead of System.getProperty("java.version") to check in a type-safe way that's more readable, I didn't update the code as Progaurd needs access to the JDK path that's used by Gradle instead of the one in system environment path.
I'm not sure if there is an API in Gradle to get the Path of the used JDK, the developer might want to use a Java version for a project that's not the same as the one configured to be used globally, using something like JavaVersion.current().isCompatibleWith(JavaVersion.VERSION_1_9) will check the java version that's used by Gradle to run Gradle daemon or the task.
Which use different versions between the two and might cause an unexpected bug in some rare cases where the developer includes a module that's available in a certain Java version and not available in the project Java Version, instead I used the following:
val javaHome = System.getProperty("java.home")
    val javaVersion = System.getProperty("java.version")
    if (javaVersion.startsWith("1.")) {
// A version before Java 9
}

We also could include all available Java modules in the current JDK instead of explicitly including the required or used modules.

Thank you for the comments @ellet0! Your PR is now merged 🎉

Thank you for the comments @ellet0! Your PR is now merged 🎉

I appreciate your consideration. It's been a rewarding experience.
Should we use Proguard rules from the JAR file in META-INF/proguard instead of copy and paste them manually in proguard.pro?
I already did this with the following code snippet:
JarFile(fatJarFile.get().asFile).use { jarFile ->
                val generatedRulesFiles =
                    jarFile
                        .entries()
                        .asSequence()
                        .filter { it.name.startsWith("META-INF/proguard") && !it.isDirectory }
                        .map { entry ->
                            jarFile.getInputStream(entry).bufferedReader().use { reader ->
                                Pair(reader.readText(), entry)
                            }
                        }.toList()

                val buildProguardDirectory =
                    layout.buildDirectory
                        .dir("proguard")
                        .get()
                        .asFile
                if (!buildProguardDirectory.exists()) {
                    buildProguardDirectory.mkdir()
                }
                generatedRulesFiles.forEach { (rulesContent, rulesFileEntry) ->
                    val rulesFileNameWithExtension = rulesFileEntry.name.substringAfterLast("/")
                    val generatedProguardFile = File(buildProguardDirectory, "generated-$rulesFileNameWithExtension")
                    if (!generatedProguardFile.exists()) {
                        generatedProguardFile.createNewFile()
                    }
                    generatedProguardFile.bufferedWriter().use { bufferedWriter ->
                        bufferedWriter.appendLine("# Generated file from ($rulesFileEntry) - manual changes will be overwritten")
                        bufferedWriter.appendLine()

                        bufferedWriter.appendLine(rulesContent)
                    }

                    configuration(generatedProguardFile)
                }
            }

This will help make the rules for Kotlinx coroutines, serialization, and others up to date by only updating the libraries without changing proguard.pro file.
Regarding separating proguard example into server-proguard, we also could have a folder called proguard with two folders in it client and server because Ktor client can be used with Proguard too.


Expire trial tenants that have not logged in for 14 days

Add maintainer that expires trial tenants that have been idle for 14 days
Created feature flag to exempt some tenants from expiry
Extended SytemName parameter in ZoneApiMack to facilitate tests

FYI - @kkraune
I confirm that this contribution is made under the terms of the license found in the root directory of this repository's source tree and that I have the authority necessary to make this contribution on behalf of its copyright owner.

A test failed: [ERROR]   ControllerApiTest.testControllerApi:53 maintenance.json expected:<...eporter"},{"name":"C[]ontactInformationMai...> but was:<...eporter"},{"name":"C[loudTrialExpirer"},{"name":"C]ontactInformationMai...>
maintenance.json must be updated.

Do we need to do something in the console to help them understand what to do if they come back?

I think the the account page will show a message to contact support for the none subscription, but I'll make sure it's so.

Even better, I'll make sure it's a notification when they're in the none-subscription.


Removed extra $ in the .PARAMETER

I know..it's not a huge change..but it's the little things:
comment based help for DomainDistinguishedName.

Nice catch, thanks @evetsleep !


update to netcoreapp2.0

The big issue is that Cake doesn't support running on .NET Core 2.  Either need to install multiple runtimes or wait for Cake to update.


Support for server installation with systemd in RHEL based distributions

Considers the ohai attribute 'init_package' to establish service name and setup script (cannot run service initdb as it's not allowed with systemd)
This setup script changes location (and it's not included in $PATH) whether it comes from the native distro yum repo (default version), or pgdg repo.
This are the changes over the develop branch. I've reduced them to consider only version 9.3 from yum pgdg repos enabled, as the kitchen tests specify. Verified with all server-centos and yum-pgdg-centos kitchen machines.

This looks like a very helpful PR. Waiting for it a lot.

I've been using this branch for a bit, it'd be really nice to get it merged.

Trying to use this, but might not work on docker. As for as I can tellnode['init_package'] is sshd in dockerized images

Yeah, seems that detecting init systems is very finicky:
http://unix.stackexchange.com/questions/18209/detect-init-system-using-the-shell
We should probably just assume RHEL 7 and CentOS 7 are using the default systemd for now, and if someone needs to generalize the logic later, we can let them do that. Would that work?

You're right the Ohai init_package is not as accurate as it should be, there was a discussion about this PR: https://github.com/chef/ohai/pull/301, last comment shows an example using a mapping.
I could rewrite the PR so it gets the init system from such a mapping, like in here https://github.com/jordansissel/pleaserun/blob/master/lib/pleaserun/detector.rb#L10-L32
Result should be exactly the same.

If you could please rebase this sometime I will try and get it tested and merged.

Done. I've added a new attribute to establish init system by hand, instead of using Ohai.
Also included the svc_name variable after initdb as in https://github.com/hw-cookbooks/postgresql/blob/develop/recipes/server_redhat.rb#L98


Fixed typo Webite => Website

Coverage remained the same at 100.0% when pulling 9b545fcc5472a2ce53c744a1c259adcddd6a7b5e on donavon:feature/typo into b1ec147ec50510322c2c9223af11a51095f142ce on shinnn:master.

Good catch. Thanks, @donavon!


Video recording is not generated when I run android native app and the "test method" for video recording scope is enabled.

When you update this property >> "videoParams_scope=TestMethod"  in the "VisualValidations.properties" file >> then you run your tests using maven >> and open the generated allure report to check the video recording, nothing is there.

Screenshot the allure report please and kindly make sure to expand the aftermethod and afterclass if any.

Screenshot the allure report please and kindly make sure to expand the aftermethod and afterclass if any.

can you kindly confirm that this issue is still existing in the latest engine release?
if so, kindly attach the full execution log file here.

Please and Thank You.

https://github.com/MohabMohie privacy concerns please and thank you

https://github.com/MohabMohie privacy concerns please and thank you

will close this issue due to lack of activity for the past week, but kindly feel free to reopen it and keep the conversation going.


AARCH64 builds broken

Click to expand!
Issue Type
Build/Install
Source
source
Tensorflow Version
git HEAD
Custom Code
No
OS Platform and Distribution
CentOS 7
Mobile device
n/a
Python version
3.8.10
Bazel version
5.1.1
GCC/Compiler version
10.2.1
CUDA/cuDNN version
n/a
GPU model and memory
n/a
Current Behaviour?
Build fails to complete with
"./tensorflow/compiler/xla/service/cpu/runtime_fp16.h:23:23: error: '_Float16' does not name a type"

Standalone code to reproduce the issue
bazel build --config=nonccl --config=mkl_aarch64 --copt="-mtune=generic" --copt="-march=armv8-a" --copt="-O3" --copt="-fopenmp" --linkopt="-lgomp" --jobs=100 -- //tensorflow/tools/pip_package:build_pip_package

Relevant log output
In file included from tensorflow/compiler/xla/service/cpu/runtime_fp16.cc:16:
./tensorflow/compiler/xla/service/cpu/runtime_fp16.h:23:23: error: '_Float16' does not name a type
   23 | using XlaF16ABIType = _Float16;
      |                       ^~~~~~~~
./tensorflow/compiler/xla/service/cpu/runtime_fp16.h:36:12: error: 'XlaF16ABIType' does not name a type
   36 | extern "C" XlaF16ABIType __gnu_f2h_ieee(float);
      |            ^~~~~~~~~~~~~
./tensorflow/compiler/xla/service/cpu/runtime_fp16.h:39:33: error: 'XlaF16ABIType' was not declared in this scope
   39 | extern "C" float __gnu_h2f_ieee(XlaF16ABIType);
      |                                 ^~~~~~~~~~~~~
./tensorflow/compiler/xla/service/cpu/runtime_fp16.h:42:12: error: 'XlaF16ABIType' does not name a type
   42 | extern "C" XlaF16ABIType __truncdfhf2(double);
      |            ^~~~~~~~~~~~~
tensorflow/compiler/xla/service/cpu/runtime_fp16.cc:63:1: error: 'XlaF16ABIType' does not name a type
   63 | XlaF16ABIType ABSL_ATTRIBUTE_WEAK __gnu_f2h_ieee(float float_value) {
      | ^~~~~~~~~~~~~
tensorflow/compiler/xla/service/cpu/runtime_fp16.cc:117:27: error: redefinition of 'float __gnu_h2f_ieee'
  117 | float ABSL_ATTRIBUTE_WEAK __gnu_h2f_ieee(XlaF16ABIType hf) {
      |                           ^~~~~~~~~~~~~~
In file included from tensorflow/compiler/xla/service/cpu/runtime_fp16.cc:16:
./tensorflow/compiler/xla/service/cpu/runtime_fp16.h:39:18: note: 'float __gnu_h2f_ieee' previously defined here
   39 | extern "C" float __gnu_h2f_ieee(XlaF16ABIType);
      |                  ^~~~~~~~~~~~~~
tensorflow/compiler/xla/service/cpu/runtime_fp16.cc:117:42: error: 'XlaF16ABIType' was not declared in this scope
  117 | float ABSL_ATTRIBUTE_WEAK __gnu_h2f_ieee(XlaF16ABIType hf) {
      |                                          ^~~~~~~~~~~~~
tensorflow/compiler/xla/service/cpu/runtime_fp16.cc:141:1: error: 'XlaF16ABIType' does not name a type
  141 | XlaF16ABIType ABSL_ATTRIBUTE_WEAK __truncdfhf2(double d) {
      | ^~~~~~~~~~~~~
Target //tensorflow/tools/pip_package:build_pip_package failed to build

@cfRod @nSircombe @penpornk

Introduced by https://github.com/tensorflow/tensorflow/commit/579c67bd60220fe66a29318191f0850ff0aa825f

@d0k

@d0k on AARCH64 the availability of _Float16 depends on if the target CPU includes the support or not. The documentation is sadly lacking in explaining this. So the default compile settings of '--copt=-march=armv8-a' result in the above failure. But if targetting a CPU with the needed support by '--copt=-march=armv8-a+fp16' then _Float16 is available and the code would compile, but then generate an illegal instruction trap if run on a CPU without the support.

The purpose of this code is providing the proper ABI for fp16 function calls when the hardware doesn't support fp16 natively. On x86 fp16 used to be passed in GPR but now moved to floating point registers. I think on aarch64 it's always an fp register, but I don't know that architecture well.
The main problem here is that GCC is essentially lying about the type, which also contradicts the documentation. I'll put
in a hack to not use _Float16 on ARM64 GCC. Everything seems to be working fine with ARM64 Clang, which doesn't define __FLT16_MANT_DIG__ when the type isn't actually there.
Note that there is no _Float16 arithmetic done here, XLA just needs to match the ABI between JIT-compiled code and runtime functions. Compilers making type availability dependent on hardware features and then providing no macros
for detecting those features is really awful.


Feature request: Sending Folders on iOS app

Is your feature request related to a problem? Please describe.
No known problem that I know of
Describe the solution you'd like
I downloaded the app from the IOS app store and I got everything working correctly except for sending folders from my Iphone. When I try I am getting "No permission. You have not granted the necessary permissions. Please grant them in the settings." How can I grant these settings, I checked the app settings in my general settings but no success. Would like some support
Describe alternatives you've considered
No response
Additional context
No response

Hi, thanks for this wonderful cross-platform app.
I see that we have an option to select folder to send on iOS. But, when we select the folder and the control returns to localsend for transferring it, the app crashes immediately.
Thanks🙏

You will not be able to press on a folder in My files to send it but ..
i tried this and worked for me and files was in good hands
long press on the FOLDER and press Compress
it will be compressed into a ZIP folder
Hold click on the Zip folder and press share the arrow upward icon
then look for LocalSend
it will be sent without any problems and no permissions needed and you can UnZIP it on any device easily


total_reward never changed

Thanks for your work.
I am facing a problem of training the project.
I found the total_reward never changed when I modified to use DDPG algorithm to train the project.

Can you give me some suggestions?
Thanks a lot!

Please try to reproduce existing examples, get some experience first, which may help a lot here.  It seems this may be due to some simple errors, and other may not experience it.

@XiaoYangLiu-FinRL
Thanks for your reply.
I changed the learning rate, it worked.


Collection Objects filter task= prolonged issues downloading CSV

I want to preface this by saying that I understand most of these issues I will mention are a work in progress and will be updated eventually, but wanted to draw much needed attention to them. I am not an API person so shortcuts that way would be difficult for me.
Quite regularly I need to download data in CSV from CO filter task.

For example, currently I need to download all data on Trichoptera specimens collected from the years 2017-2020. I can do this MOST EFFICIENTLY by adding "Trichoptera" as a Taxon name under Determinations, followed by entering "2020" in the Collecting Event field under Buffered since entering dates under Date Range is not useful. This usually spits out the specimens I need, with a few sprinkled in that do not apply to that year, but have "2020" somewhere in their buffered label, be it coordinates or collecting times. This results in having to delete rows of data that are not needed in the CSV after it is downloaded.

If there are more than 500 records per page, the CVS will not include more than those first 500 records, despite changing the number of records per page to 1000.

I can get around this bug by downloading each page of 500 records separately and condensing them into one file on Numbers or Xcel.

Lastly, if you reference my last screenshot image above, you will notice that Catalogue numbers in the CSV are omitted for containerized specimens or specimens that need to be reindexed. This is by far the issue that gives me the most grief! I have to go back and manually enter each catalogue number in the spreadsheet before I send it to my supervisor.

What could take just a few minutes of data retrieval ends up taking me days to finish due to these issues. I worry that someone who is not familiar with Taxonworks would become greatly discouraged if they wanted to search specimens to maybe request a loan or just generally gather data.
Thank you in advance!

Sorry I forgot to annotate each image.
First: shows 1000 records per page
Second: shows the catalogue number of the last record for that page
Third: shows the last records of the downloaded CSV list, which do not reflect what is in TW because only the first 500 are included on that list.

@jlpereira the CSV is client side right?
@lvhart2 we'll try and prioritize the easy stuff.  Some of this will be improved/is dealt with on other branches, particularly the DwC export branch.
Thanks very much for adding this, and don't hesistate to make these requests when issues are systematic for you.

https://github.com/SpeciesFileGroup/taxonworks/blob/e74814dc6d97eb0b32f3d92671ab81a613ae93a2/app/controllers/collection_objects_controller.rb#L357
That must be preventing the CSV download to be larger than 500. Handcrafting the request with a larger per param allows to download much more.
Would be reasonable to disable pagination at backend if explicitly requested by frontend (with some new param)?

https://github.com/SpeciesFileGroup/taxonworks/blob/e74814dc6d97eb0b32f3d92671ab81a613ae93a2/app/controllers/collection_objects_controller.rb#L357
That must be preventing the CSV download to be larger than 500. Handcrafting the request with a larger per param allows to download much more.
Would be reasonable to disable pagination at backend if explicitly requested by frontend (with some new param)?

Related: https://github.com/SpeciesFileGroup/taxonworks/issues/2121
If i remember well, the last time i tried to increment the records per page, the maximum i got was 20.000 records.

@lvhart2 Can you provide a little more information about what you're doing with the data?  Are there just a couple columns that you need?  What is your boss doing with the data?  Things that take days really shouldn't, we need to address the workflow specifically I suspect.

I work for Ed DeWalt. He wants to be able to download data of past collecting years so he can have an idea of what he found and where.
He usually has me do all of the TW work for him, but I know we need all of the columns except for
-collection_objects-id
-dwc_occurrence_id
-dwc_occurrence_object type
-institutional code & id
-nomenclatural code
He has also mentioned that it would be helpful for the individual count to reflect the actual biocuration as well. Currently I have to manually enter that data if he needs it.
And possibly to be able to search by trip code/accession code for collection objects and have that as a column in the downloaded csv.
Feel free to reach out to him directly<EMAIL_ADDRESS>Thanks!

@lvhart2 I believe we might be able to close this issue with the new DwC download functionality.  Let us know if so.

@mjy status? do we need @lvhart2 to test using the DwC download function?

I brought this issue to the meeting on 2/21/24 with the difference that I was generating search results under Filter Nomenclature and need to be able to download the entirety of the resulting data, no matter how many entries it may involve. Currently each download is limited to 2500 results so I have to download separate pages of 2500 and then combine the downloads into one document.

@hhopkins77 I'm glad you brought this up because twice this week I have tried to download CO data as a DwC file and it never seems to load. Months ago I filtered data and downloaded it no problem- it was really to download in a couple minutes. What I need is Plecoptera with descendants from the state of Arkansas. The file contains 2721 records. I don't think I'm being impatient. Will update once the file is ready to download.

@lvhart2 there is a known bug in the downloader that we should have fixed tomorrow, sorry for this.  Indeed 2k records should be available in the hub in 1 minute or two, if they are not please open bugs here in the future.

@mjy @lvhart2 is this still an issue or have all these problems been fixed?

Yes these have all been fixed. Thanks!


fix files.add promise test

Detects bug found on https://github.com/ipfs/js-ipfs-api/issues/308#issuecomment-230104544


installing rviz_visual_testing_framework config

Addresses ros2/rviz#372
Feature:

Moved config files to a config folder
updated path in cpp
installed config folder

Tested:

Compilation and laser scan test on 18.04
colcon build --merge-install  --cmake-args -DEnableVisualTests=True -DBUILD_TESTING=1
colcon test --merge-install --packages-select rviz_default_plugins --ctest-args -R laser

build/rviz_default_plugins/test_results/rviz_default_plugins/laser_scan_display_visual_test.gtest.xml: 1 test, 0 errors, 0 failures, 0 skipped
Summary: 1 test, 0 errors, 0 failures, 0 skipped

config files in install/share folder locally

Looks like this PR went stale and the issue was resolved separately in https://github.com/ros2/rviz/pull/487


pglookout: support explicit failover priorities

Support explicit prioritization between instances. This can be configured via failover_priorities key, and will be consulted upon picking up the standby that should do the promotion in cases where multiple nodes have a matching replication position.
Previously, and also as the current default, the selection was based on the sorting order of the remote nodes.
The configuration option allows some additional flexibility, and supports e.g. topologies where we have more favorable and less desirable standbys in multiple different network locations.

How are we going to synchronize the configuration for failover priorities ?
It seems quite dangerous to me that we could have separate nodes favoring themselves, leading to split brain scenarios.

In Aiven case, this would be based on the configuration delivered to the services. The priorities help to replace the implicit prioritization via instance identifies with explicit order, but both list are indeed generated from a common config.

The pipeline fails for a rather silly reason:
  test/test_lookout.py:1:0: C0302: Too many lines in module (1302/1300) (too-many-lines)

The pipeline fails for a rather silly reason:   test/test_lookout.py:1:0: C0302: Too many lines in module (1302/1300) (too-many-lines)

Thank you. I added a quick commit to move some helper functions from test/test_lookout.py into test/utils.py in order to compact the test file a bit.


plugin readme update

I had to run "docker-compose build" after modifying Dockerfile.
Unless doing so, plugin was never added.
And I thought adding this step will help others with limited knowledge of Docker.

first of all I just want to say what an awesome job on this docker image. not sure if this is the right place to post this question but I'm trying to get the X-pack stuff going and I download the X-pack branch and ran it and everything comes up just fine except that Kibana never ask me login with a default user of any kind. I'm wondering if I have to enable that feature somehow before I startup the stack with docker-compose.  Any insight into this issue would be greatly appreciated.

docker-elk/elasticsearch/config/elasticsearch.yml has
xpack.security.enabled: false
You have to change that to true and Kibana will ask you for password.

@arubalcaba it's not the best place indeed :) It would be better to open a new issue or ask your question on Gitter.
You don't have to change anything if you checked out the x-pack branch. If you run:
git checkout x-pack
docker-compose build

you will end up with the expected images, and see a login screen.