Some misbehaving SSL servers hang at the end instead of responding wi…

…th an SSL close_notify alert.  the docs for SSL_shutdown say 'If a unidirectional shutdown is enough (the underlying connection shall be closed anyway), this first successful call to SSL_shutdown() is sufficient.' which seems to fix the problem.  Also capture Errno's during close instead of bubbling them up, same as IO::Errors. #7136
Sorry I couldn't format the commit message easily on the command line...

Can we implement a misbehaving server in Crystal for the spec?

OK added a spec that repro's it, thanks!

I believe that's the reason it was put in the "spec/manual" directory,
basically it's unstable specs "go there".  But feel free to pull it if
desired, now that we have a working local spec it isn't as necessary.
On Sat, Nov 16, 2019 at 10:48 AM Sijawusz Pur Rahnama <
<EMAIL_ADDRESS>wrote:

@Sija commented on this pull request.
In spec/manual/https_client_spec.cr
https://github.com/crystal-lang/crystal/pull/8481#discussion_r347100651:

@@ -12,4 +12,8 @@ describe "https requests" do
break
end
end

it "can fetch from IIS servers that don't shutdown gracefully" do
HTTP::Client.post("https://login.microsoftonline.com/common/oauth2/token", form: {"hi" => "a"}).body

This imo shouldn't be added - we don't want to rely on 3rd party services
in tests.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/crystal-lang/crystal/pull/8481?email_source=notifications&email_token=AAADBUHDNYJTRHVLOLAN7HTQUAW5DA5CNFSM4JODZ4Q2YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCL2AEKA#pullrequestreview-317981224,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAADBUE2GNT4QX4AZGLIWUTQUAW5DANCNFSM4JODZ4QQ
.

I think this is ready to merge if somebody would like to help me out with it sometime.  Then I could move onto other OpenSSL stuff.  Thanks!

This imho would perfectly fit into 0.32.0 :)

Thanks!
On Thu, Nov 28, 2019 at 11:21 AM Stephanie Hobbs<EMAIL_ADDRESS>wrote:

Merged #8481 https://github.com/crystal-lang/crystal/pull/8481 into
master.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/crystal-lang/crystal/pull/8481?email_source=notifications&email_token=AAADBUBWRLYENAKEQEH3XUDQWADZHA5CNFSM4JODZ4Q2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOVFBFUWA#event-2839698008,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAADBUC6XY7KZPTWCASPSLTQWADZHANCNFSM4JODZ4QQ
.

FYI it seems the CI is not always happy with this new spec. Any clues?
https://travis-ci.org/crystal-lang/crystal/jobs/623864676?utm_medium=notification&utm_source=github_status
Unhandled exception in spawn: Error writing to socket: Broken pipe (Errno)
  from src/socket.cr:82:13 in 'unbuffered_write'
  from src/io/buffered.cr:135:14 in 'write'
  from src/openssl/bio.cr:31:7 in '->'
  from ???
  from BIO_write
  from ???
  from BIO_ctrl
  from ???
  from ???
  from ???
  from SSL_do_handshake
  from src/openssl/ssl/socket.cr:57:15 in 'initialize'
  from src/openssl/ssl/socket.cr:54:5 in 'new:sync_close'
  from src/openssl/ssl/server.cr:63:5 in 'accept'
  from spec/std/openssl/ssl/socket_spec.cr:46:25 in '->'
  from src/fiber.cr:255:3 in 'run'
  from src/fiber.cr:47:34 in '->'
  from ???

Yes it's #8559 but I haven't tidied it all up yet...
On Thu, Dec 12, 2019 at 11:32 AM Brian J. Cardiff<EMAIL_ADDRESS>wrote:

FYI it seems the CI is not always happy with this new spec. Any clues?
https://travis-ci.org/crystal-lang/crystal/jobs/623864676?utm_medium=notification&utm_source=github_status
Unhandled exception in spawn: Error writing to socket: Broken pipe (Errno)
from src/socket.cr:82:13 in 'unbuffered_write'
from src/io/buffered.cr:135:14 in 'write'
from src/openssl/bio.cr:31:7 in '->'
from ???
from BIO_write
from ???
from BIO_ctrl
from ???
from ???
from ???
from SSL_do_handshake
from src/openssl/ssl/socket.cr:57:15 in 'initialize'
from src/openssl/ssl/socket.cr:54:5 in 'new:sync_close'
from src/openssl/ssl/server.cr:63:5 in 'accept'
from spec/std/openssl/ssl/socket_spec.cr:46:25 in '->'
from src/fiber.cr:255:3 in 'run'
from src/fiber.cr:47:34 in '->'
from ???
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/crystal-lang/crystal/pull/8481?email_source=notifications&email_token=AAADBUAXCUYDD4TZCUOTVXLQYJ7VPA5CNFSM4JODZ4Q2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGXS5VQ#issuecomment-565128918,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAADBUGJKJ36XW43RLVGWVDQYJ7VPANCNFSM4JODZ4QQ
.


HTTP POST Request Example

Hello Team,
This doesnt qualifiy as an issue but more of an example request. I am trying to build a Rust client for making a http api call which is based on no-std. I am new to
Please can give me an example for the following api calls with httparse crate.

Call to get Ouath Token in reponse passing username and password:
curl -d "username=hello&password=world" -H "Content-Type: application/x-www-form-urlencoded" -X POST http://localhost:8080/login

Call to get data using the token

curl --get http://localhost:8080/user/api/data --header "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJyZW5hdWx0IiwibmJmIjoxNjMxNzEzMTcyLCJleHAiOjE2MzE3OTk1NzIsInVzZXJJZCI6IjEiLCJhdXRob3JpdGllcyI6IlVTRVIiLCJ1c2VybmFtZSI6InJlbmF1bHQifQ.Ktsg_084LPg8KSZnKqdloRjdHBQzEeuBGAka8CHcrUIA6kubvMGBq03qWYKhUP-_FrBZKOd5eb2DuUt24K0TLcaM-meGNtUvSDU-0wVZIxEwgSTHbVZ2QRf9eNuSkcW7s1QHg29hzxZ2_f2KHNZWqVjSs4JxqExXYPkxLhidYT8d_22oLWcDMtnfdUZ6fhmsRZ-jV0h-sB_zV0z3dBY9ZNL_KduYhdCGzXpGPjfpieYJAieDqc2P1Gy2N1gk88eCKvsYAs011egSBmhRGy-fJuU_Y4rlvdxa5I6pjec_vvWBMVMxGtyxLjHCFn8VQW59DnUA5hEOVRzgv7l_IiZrvA"
Thanks for the help.

Hey there! I'm not sure what exactly you're looking for... You mentioned make a client, and showed some options to make a request with curl, but httparse is only for parsing, so a client would only need to parse a response.
Since this isn't a bug, I'm going to close. I'd recommend asking with the extra details in a place better suited, such as StackOverflow or the r/rust weekly "easy question" thread.


Update integration-ec2-job-parameters.md

Change FSx ID to DNS name.
You need the DNS name for this to work, see https://github.com/awslabs/scale-out-computing-on-aws/blob/master/source/soca/cluster_node_bootstrap/ComputeNodePostReboot.sh#L90
It wouldn't be hard to switch this to take the FSx id instead which imho would make a lot more sense, but that's outside of the scope of this PR.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Merged internally


angular-animate doesn't work

Error message:

TypeError: Cannot read property '' of undefined
at lookup (angular-animate.js:619)
at animationRunner (angular-animate.js:683)
at performAnimation (angular-animate.js:1286)
at angular-animate.js:961
at angular-animate.js:541
at Scope.$digest (angular.js:15717)
at Scope.$apply (angular.js:15935)
at done (angular.js:10356)
at completeRequest (angular.js:10528)
at XMLHttpRequest.requestLoaded (angular.js:10469)

Solution:

Remove angular-anima.js import from index.html
Remove 'phonecatAnimations' injection in app.js

have same problem. It happened for me after updating Angular from 1.3.x to 1.4.1

+1
After updating to 1.4.1

fixed it updating angular-animate

I am using angular-1.4.2 and I am using the lastest version of angular-animate on Github and it still not working.


TypeError: object of type 'numpy.int32' has no len()

When i run testing i get the below error
(vinayak) C:\Users\vinayak\Documents\github\CRNN-1\CRNN>python run.py -ex ../sam
ples --test --restore
Restoring
Checkpoint is valid
0
Loading data
examples 10
Traceback (most recent call last):
File "run.py", line 118, in
main()
File "run.py", line 111, in main
args.restore
File "C:\Users\vinayak\Documents\github\CRNN-1\CRNN\crnn.py", line 53, in in
it
self.__data_manager = DataManager(batch_size, model_path, examples_path, max
_image_width, train_test_ratio, self.__max_char_count)
File "C:\Users\vinayak\Documents\github\CRNN-1\CRNN\data_manager.py", line 26,
in init
self.test_batches = self.__generate_all_test_batches()
File "C:\Users\vinayak\Documents\github\CRNN-1\CRNN\data_manager.py", line 108
, in __generate_all_test_batches
(-1)
File "C:\Users\vinayak\Documents\github\CRNN-1\CRNN\utils.py", line 17, in spa
rse_tuple_from
indices.extend(zip([n]*len(seq), [i for i in range(len(seq))]))
TypeError: object of type 'numpy.int32' has no len()

The images in the samples/ folder are the images displayed in the README.md, not valid testing examples as they do not have a ground truth in the filename.

You can use this dataset to test https://node1.belval.org/test_dataset.zip or generate a new one using https://github.com/Belval/TextRecognitionDataGenerator.

i found a workaround. I commented sparse_tuple_from call from __generate_all_test_batches and used only batch_x and batch_y to run test. Anyways we just need batch_x only to do the prediction.
And yes, i had modified the filenames in the samples folder
Thanks

hello，i also have this problem during training the new model.
the command i used is :
python run.py -m /save/ -ex F:/code/ICDAR/2015part/ch4_training_word_images_gt/ --train
and then i get the same error message as you
Traceback (most recent call last):
File "run.py", line 119, in
main()
File "run.py", line 99, in main
args.restore
File "C:\Users\CRNN\CRNN\crnn.py", line 53, in init
self.__data_manager = DataManager(batch_size, model_path, examples_path, max_image_width, train_test_ratio, self.__max_char_count)
File "C:\Users\CRNN\CRNN\data_manager.py", line 28, in init
self.test_batches = self.__generate_all_test_batches()
File "C:\Users\CRNN\CRNN\data_manager.py", line 112, in __generate_all_test_batches
batch_dt = sparse_tuple_from(np.reshape(np.array(raw_batch_la),-1))
File "C:\Users\CRNN\CRNN\utils.py", line 20, in sparse_tuple_from
indices.extend(zip([n] * len(list(seq)), [i for i in range(len(list(seq)))]))
TypeError: 'numpy.int32' object is not iterable
i use this model on dataset icdar2015,and the filename is just like the samples.
could you please help me to solve this problem ,thanks a lot.
@vinayakkailas  @Belval


View the SAP BTP from 10,000 Meters

Tutorials: https://developers.sap.com/tutorials/cp-explore-cloud-platform.html
The text and the action a user should do are not matching 100% in step 4.
There is a link to the Data Center map: "You can open this map when you click on the globe icon on this page to see a complete list of the types of centers and the services provided at each data center."
which goes to: https://discovery-center.cloud.sap/viewServices
In my case, the page opens with the filter bar minimized. Therefore, the globe icon is not visible. To see it, I had to click on filter to expand the filter bar and to see the globe icon.
This link:
https://discovery-center.cloud.sap/viewServices?provider=all&regions=all&showFilters=true
opens the page with the filter bar expanded, showing the globe icon.

Thanks Tobias. You're right, the page layout changed. I replaced the link and also updated the screenshots.
The changes should be live in the next few minutes.
Have a great weekend,
Marius


make python examples using_qiskit* and other examples run as a test

What is the expected enhancement?
Since the using_qiskit are to be use case examples we need to make them part of the tests. We would have found out that job.done was used and I'm hoping user did note make examples based on this when we make this release.

I think this should not be part of Terra but the mega test. My main point is in the last update things were changing and the examples were not. Since these define how different users use qiskit we need to know when they break. They are more like a full integration test.
I just feel we keep talking about it but I want a full integration to have a green light that says tests pass.

Sure, where the test lives doesn't actually matter too much, especially if it's like what I pasted before. That would just need the relative path updated to work from anywhere. Although, if we do have it in an external test suite we will have to fix that examples/ in terra isn't actually in the sdist so it may not be present if it's just pip installed (but we can solve that by just adding it to the manifest file).

I agree. I am not sure what the best thing is to do. I let you decide. the workflow is more how do we make sure that things dont break for Aer<->Terra<->IBMQ and standard user tasks.

For the record, @diego-plan9 wrote https://github.com/Qiskit/qiskit-tutorial/blob/master/utils/test/test_tutorials.py to help testing the tutorials. Perhaps we should include it as part of the regular tests.

Maybe once we finish separateing  the tutorial from ones that teach quantum using qiskit from those that teach qiskit.  This is going to take a long time  (your welcome to help there) so I would much rather a solution using these examples.

@mtreinish is there any update on this? I think what you pasted above works in the sense that it makes sure no error is raised while executing the examples. Later on we may want to test some correctness aspects as well, but for now just need to make sure they're not broken.


Find maintainer with more active use

I started this package mostly as an experiment. While it has worked pretty well so far, I don't actively work on it, and I don't have any projects I work on that depend on it right now.
If someone else is willing to maintain this package, I'm happy to pass the baton.
@brev, @claydiffrient, and @chrisjaure have all contributed or at least shown interest in the project. Are any of you willing to take ownership?

@thetalecrafter thanks for the offer! I'm not currently using it either, but I'd be willing to maintain it. I think stateless function components will make this pretty nice :)

If you aren't really using it either, then it probably would just be more work for you, and not necessarily any help to people who are using it. If no one is using it then I suppose it doesn't matter who maintains it. ;)

I'm starting to use this project right now. I've noticed that the Component.default fix is not on the published version on npm. It would be awesome if you could release a new version with it.

@magalhas Sorry about that. I've published it now as 1.0.1.


Handle bytes identifier as UTF-8 strings

Previously, we visited either strings or bytes so that the visitor had to handle both cases.  With this change, we always visit strings so that the bytes visitor function can be optimized out. This significantly reduces binary size.

Yes, at least serde.  We just need to create a nice example demonstrating the problem.


Fix unusual build problems

Fix building tests in release mode.
Fix building the package with library stability enabled. (This configuration is unsupported, and this fix does not constitute a promise that this package has stable ABI.)

Checklist

[X] I've read the Contribution Guidelines
[X] My contributions are licensed under the Swift license.
[ ] I've followed the coding style of the rest of the project.
[ ] I've added tests covering all new code paths my change adds to the project (if appropriate).
[ ] I've added benchmarks covering new functionality (if appropriate).
[ ] I've verified that my change does not break any existing tests or introduce unexplained benchmark regressions.
[ ] I've updated the documentation if necessary.

@swift-ci test


Jupyterhub stops all docker containers when it is shutdown

Hi guys,
I just want to understand if it is required that all docker containers being stopped when jupyterhub is shutdown?
If we only want to stop jupyterhub but let all containers continue to run,, can we reattach jupyterhub to all existing running containers?
Thanks!

This is an option. If you specify c.JupyterHub.cleanup_servers = False in your config, shutting down the Hub will leave the servers up and reconnect on resume.

@minrk  Perfect! I didn't know we have this config. Thanks for letting me know!


Apps do not stick to same zone if app has multiple windows steam + steam chat or Firefox + picture in picture

Microsoft PowerToys version
0.74.1
Installation method
PowerToys auto-update
Running as admin
Yes
Area(s) with issue?
FancyZones
Steps to reproduce
Snap picture in picture 2e zone browser to 1e zone, close picture in picture re open picture in picture is now in zone1 instead of last zone2
✔ Expected Behavior
If picture in picture is closed in zone 2e zone it should re open in 2e zone, regardless of firefox snapping previously to zone 1
❌ Actual Behavior
If previously firefox was snapped to zone 1 and picture picture is on zone 2 and closed and re opent it re opens in zone 1
Other Software
Firefox 118.0.1
Steam

Appears to be the same in v0.75.  Firefox v119's PiP window now always snaps to the same zone as the Firefox window where the original video was displayed.
It used to be that the PiP window would stay at the position where it was last moved to, regardless of which window contained the original window.
My guess is that the PiP window used to be identified as a popup-class window and now is not.  When it was a popup...

...you could move the PiP window using the snap keys
...its position would be remembered by Firefox
...subsequent opening of the PiP would retain the saved position

Now that PiP is not treated as a popup...

...FancyZones snaps the PiP window to its 'parent' window
...you can move the PiP window using the snap keys
...but FancyZones continues to override the PiP window position

Could Firefox's PiP window perhaps be identified by WindowClass GUID and be included in a fixed override list?

turned back on automatic window positions again see if it is now fixed, only reason i use it is cos Whatsapp always defaults to wrong location else it be non issue but thats different issue entirely

Oke enough testing seems to still be an issue with steam chat and friends docked in 1 window

I'm unclear on what the intended behaviour should be.

 Current behaviour:
Open main window: fullscreen --> Open friends or settings window: fullscreens.

Open main window: size and zone of the ^child window.

For Steam, I'd like the main window and each sub/child windows to have different sizes & zones. However regardless of the "Allow child windows snapping" option all new windows belonging to the application will resize to the last zone and size of the most recently opened window.
Instead of using the exclusion list to remove FancyZones from the windows is there a way to treat the child windows so that they are unique.

Am going to add a couple of additional things to the mix...

The size of the Firefox Picture-In-Picture window used to be constrained to wholly fit within the zone.  For example, I have a zone at the bottom of my (vertical-oriented) screen deliberately sized to match the 16x9 aspect ratio.  If a video is 4x3, the PiP window used to fit wholly in the 16x9 zone and scale the width down; now the video stretches the width to fit the zone and the height is now allowed to stretch beyond the bottom of the zone (over the taskbar and off the screen).

Before...				Now...
+----------------+			+----------------+
|1               |			|1               |
|                |			|                |
|                |			|                |
|                |			|                |
|                |			|                |
+----------------+			+----------------+
|2   |///////////|			|////////////////|
|    |////PIP////|			|////////////////|
|    |///////////|			|//////PIP///////|
+----------------+			+////////////////+
[#]         12:00|			|////////////////|
+----------------+			+////////////////+

Sometimes I move an ongoing MS Teams call into the same 16x9 zone, and this usually works fine.  However, in Teams, if you minimize the in-call window, it is actually replaced by a smaller window (i.e. it doesn't actually "minimize" to the taskbar).  This smaller window used to position itself in the bottom-right of whatever screen it was on (i.e floating above other windows, unconstrained by the zone).  Now, it positions itself at the top-left of whatever zone it was placed in.

Before...


Rename, change "refs" field from int to "resurrected" bool.

See https://github.com/go4org/intern/pull/4#issuecomment-749932892
/cc @danderson

I like your docs in general but they're a bit too verbose and spread over too many places (and pushing the code too far apart).
You want to send just the docs as a follow-up change to this one, keeping them all together in one place?

Sure.


How to do Row selection and adding buttons to columns?

I tried on render and added an a href on some columns but it seems it is html encoding the string.
How do I add custom Columns, on rows that respond to click event?

You either render HtmlContent which is not encoded by default or render a string with Encoded option set to false. You see encoding examples here.

Did you got it working?

Thanks! it solved my issue!


build: use docker.io/ceph/ceph:v15 as BASE_IMAGE

Use a qualified image name, including the registry where it should come
from. This makes it possible to pull the image from the right location,
and consume it in CI jobs without trying to pull again.

Show available bot commands
These commands are normally not required, but in case of issues, leave any of
the following bot commands in an otherwise empty comment in this PR:

/retest ci/centos/<job-name>: retest the <job-name> after unrelated
failure (please report the failure too!)
/retest all: run this in case the CentOS CI failed to start/report any test
progress or results

@mergifyio rebase

/retest ci/centos/mini-e2e-helm/k8s-1.19

/retest ci/centos/mini-e2e-helm/k8s-1.19

#1740 has been posted to address the failure while deploying Vault.


🎨 Update to new Anchor UAL plugin.

(Jesta says it resolves some UX issues with the last version.)

<EMAIL_ADDRESS>


Use Jinja2 templates.

@timabbott please review. There is only one commit, even though this is a big commit but I think the changes are trivial.
Changes in templates:

Change block.super to super().
Remove load tag because Jinja2 doesn't support it.
Use minified_js() instead of {% minified_js %}. The safe filter is
explicitly applied as well.
Use compressed_css() instead of {% compressed_css %}. The safe filter
is explicitly applied as well.
forloop.first -> loop.first.
Use {{ csrf_input }} instead of {% csrf_tokesn %}.
Use {# ... #} instead of {% comment %}.
Create css.jinja template for Django Pipeline.
Use url() instead of {% url %}.
Use _() instead of {% trans %} because in Jinja trans is a block tag.
Use {% trans %} instead of {% blocktrans %}.
Add check to user_profile for undefined in templates/zerver/base.html.

Changes in tools:

Check for trans block in check-templates instead of blocktrans

Changes in backend:

Create custom render_to_response function which takes request objects
instead of RequestContext object. There are two reasons:

RequestContext is not compatible with Jinja2
RequestContext in render_to_response is deprecated

Add Jinja2 related support files in zproject/jinja2 directory. It
includes a custom backend and a template renderer, compressors for js
and css and Jinja2 environment handler.
Enable slugify and pluralize filters in Jinja2 environment.
Add Django 1.8 compatible TEMPLATES setting and remove TEMPLATE_*
settings.

Automated message from Dropbox CLA bot
@umairwaheed, it looks like you've already signed the Dropbox CLA. Thanks!

The /integrations  page throws an error when one tries to load it related to the verbatim tag -- can you investigate it?

The /integrations page throws an error when one tries to load it related to the verbatim tag -- can you investigate it?

Sure

Great, thanks @umairwaheed!  This is looking pretty good.  What have you done in terms of testing so far?  The things I'm thinking we should do are:

Checking that it generates the same .po files as before when using ./manage.py makemessages -- I think that would be a great way to confirm the translation strings process worked correctly.
Checking that the various static content pages /api, /integrations/, /features, /hello, /login, /analytics, the main logged-in app (/), etc. have the same HTML content before and after the migration so we can be confident this isn't breaking anything.  Is this something you can do?

The other thing I'm a bit worried about is creating XSS issues with the changes to how user content is escaped in the templates -- do you have thoughts on how we can confirm we're not introducing any with this changeset?

Thanks @timabbott. Mainly, I have just relied on the already written tests. Sure, I'll be glad to check both .po and static pages. I think I did check /login and /hello but I'll go over them once more from the perspective of QA.
Hmm, let me do a bit of research on XSS.

@timabbott I have confirmed that the .po files have same translations strings. I had to update the makemessages command because default version doesn't capture the trans tag of Jinja2. While comparing the .po files I am ignoring the comments, I am using:
diff -I "^#" django.po django_master.po

The comments in the .po files include references to the lines where the translation strings exist, we are removing some lines like {% load i18n %} due to which line references don't match.

I have confirmed that the following links work now:

/integrations
/features
/hello
/login
/activity (there was a problem in this one)
/realm_activity/zulip.com/
<EMAIL_ADDRESS>

About XSS. In Jinja2 everything is already escaped by default. The places where the problem can be are where we are explicitly marking the content as safe or are turning the auto escaping off using {% autoescape off %}. I guess we can just review the code which marks the content safe. I'll also try to find out how else we can test it.

@timabbott I think we can also test if we have introduced any XSS vulnerability by scraping text off the html pages and check that the text is same as before. I can write a Selenium based solution.

Hmm.  Not sure if it's worth writing a Selenium suite for that.  But one thing I am still worried about is that we use the Django mark_safe function in a bunch of places in the code:
(zulip-venv) tabbott@zaset:~/zulip$ git grep mark_safe
analytics/views.py:from django.utils.html import mark_safe
analytics/views.py:    content = mark_safe('
' + output + '
')
analytics/views.py:    return mark_safe(email_link)
analytics/views.py:    return mark_safe(realm_link)
zerver/forms.py:from django.utils.safestring import mark_safe
zerver/forms.py:                raise ValidationError(mark_safe(u'That user does not exist at MIT or is a mailing list. If you want to sign up an alias for Zulip, <a<EMAIL_ADDRESS>us.'))
zerver/forms.py:        raise ValidationError(mark_safe(SIGNUP_STRING))
zerver/forms.py:            raise ValidationError(mark_safe(error_msg))
zerver/lib/upload.py:from django.utils.safestring import mark_safe
zerver/lib/upload.py:    return mark_safe(re.sub('[-\s]+', '-', value, flags=re.U))

does that work correctly with Jinja2, or do we need to do something different?

@timabbott I have updated the code to use Markup() which is the equivalent of mark_safe in Jinja2, for the time being I am doing from jinja2 import Markup as mark_safe so that we don't have to change the code. I think mark_safe from Django was also working alright but it's safer to use Markup from Jinja2.
I also did some manual testing and there were few bugs that I uncovered, mainly relating to the fact that Jinja2 expects () at the end of a function so some forms were not working.
I have also tested the following processes:

Registration
Confirm registration. I just copied the link from the email and pasted in the browser to complete registration.
Invitation
Mass invitation when you create a new realm.

I can continue to do manual testing.

Some more testing:

Chat between two users both private and through streams works
In case a user is offline, emails are being sent.
Offline/online/away status of the user is correct.

Awesome!  Yeah I think given how many bugs we've been finding in the templates, I think maybe it's worth trying to do manual testing of every template; I think you've already done more than half of them already.  If there are any you're not sure how to test, ping me and I'm happy to take care of them.

Hey @timabbott, I haven't found a way to test other authentication backends. Actually, is there a way to test production environment using Vagrant?

I can take care of testing the other auth backends; by and large they don't have much UI.
There's no way to test the production environment in Vagrant right now.
One thing I'm a bit confused by is that the Jinja2 version seems to be slower than the old version.  I did some testing watching the server log while I repeatedly reloaded the /integrations page, and here's what I get on master:
2016-04-29 18:50:16,022 INFO     <IP_ADDRESS>       GET     200  58ms /integrations/ (unauth via ?)
2016-04-29 18:50:17,526 INFO     <IP_ADDRESS>       GET     200  42ms /integrations/ (unauth via ?)
2016-04-29 18:50:18,260 INFO     <IP_ADDRESS>       GET     200  39ms /integrations/ (unauth via ?)
2016-04-29 18:50:18,834 INFO     <IP_ADDRESS>       GET     200  46ms /integrations/ (unauth via ?)
2016-04-29 18:50:19,316 INFO     <IP_ADDRESS>       GET     200  17ms /integrations/ (unauth via ?)
2016-04-29 18:50:19,937 INFO     <IP_ADDRESS>       GET     200  44ms /integrations/ (unauth via ?)
2016-04-29 18:50:20,427 INFO     <IP_ADDRESS>       GET     200  25ms /integrations/ (unauth via ?)
2016-04-29 18:50:21,058 INFO     <IP_ADDRESS>       GET     200  39ms /integrations/ (unauth via ?)
2016-04-29 18:50:21,578 INFO     <IP_ADDRESS>       GET     200  47ms /integrations/ (unauth via ?)
2016-04-29 18:50:22,379 INFO     <IP_ADDRESS>       GET     200  19ms /integrations/ (unauth via ?)
2016-04-29 18:50:23,004 INFO     <IP_ADDRESS>       GET     200  49ms /integrations/ (unauth via ?)
2016-04-29 18:50:24,909 INFO     <IP_ADDRESS>       GET     200  43ms /integrations/ (unauth via ?)

vs. with this branch:
2016-04-29 18:52:00,938 INFO     <IP_ADDRESS>       GET     200  54ms /integrations/ (unauth via ?)
2016-04-29 18:52:01,581 INFO     <IP_ADDRESS>       GET     200  50ms /integrations/ (unauth via ?)
2016-04-29 18:52:02,119 INFO     <IP_ADDRESS>       GET     200  59ms /integrations/ (unauth via ?)
2016-04-29 18:52:02,630 INFO     <IP_ADDRESS>       GET     200  70ms /integrations/ (unauth via ?)
2016-04-29 18:52:03,082 INFO     <IP_ADDRESS>       GET     200  33ms /integrations/ (unauth via ?)
2016-04-29 18:52:03,794 INFO     <IP_ADDRESS>       GET     200  56ms /integrations/ (unauth via ?)
2016-04-29 18:52:04,442 INFO     <IP_ADDRESS>       GET     200  58ms /integrations/ (unauth via ?)
2016-04-29 18:52:05,268 INFO     <IP_ADDRESS>       GET     200  55ms /integrations/ (unauth via ?)
2016-04-29 18:52:09,757 INFO     <IP_ADDRESS>       GET     200  38ms /integrations/ (unauth via ?)
2016-04-29 18:52:20,825 INFO     <IP_ADDRESS>       GET     200  53ms /integrations/ (unauth via ?)
2016-04-29 18:52:21,561 INFO     <IP_ADDRESS>       GET     200  51ms /integrations/ (unauth via ?)
2016-04-29 18:52:22,189 INFO     <IP_ADDRESS>       GET     200  61ms /integrations/ (unauth via ?)
2016-04-29 18:52:22,721 INFO     <IP_ADDRESS>       GET     200  62ms /integrations/ (unauth via ?)
2016-04-29 18:52:23,203 INFO     <IP_ADDRESS>       GET     200  33ms /integrations/ (unauth via ?)
2016-04-29 18:52:23,770 INFO     <IP_ADDRESS>       GET     200  85ms /integrations/ (unauth via ?)
2016-04-29 18:52:24,391 INFO     <IP_ADDRESS>       GET     200  58ms /integrations/ (unauth via ?)
2016-04-29 18:52:25,045 INFO     <IP_ADDRESS>       GET     200  53ms /integrations/ (unauth via ?)

which seems clearly different.  I get the same observation looking at the homepage (/).  Any ideas what could be going on there?  In theory, I thought Jinja2 was supposed to be faster.  We ran into a similar issue with #545, so I wonder if the pieces you backported from a newer version of pipeline are involved.

@timabbott, it seems that the problem is related to pipeline because when I disable pipeline code and compare Django templates and Jinja2, Jinja2 performs better:
Jina2
2016-05-01 02:02:43,922 INFO     <IP_ADDRESS>       GET     200 737ms (+start: 267ms) /integrations/ (unauth via ?)
2016-05-01 02:02:47,311 INFO     <IP_ADDRESS>       GET     200  17ms /integrations/ (unauth via ?)
2016-05-01 02:02:48,788 INFO     <IP_ADDRESS>       GET     200  13ms /integrations/ (unauth via ?)
2016-05-01 02:02:50,450 INFO     <IP_ADDRESS>       GET     200  22ms /integrations/ (unauth via ?)
2016-05-01 02:02:52,320 INFO     <IP_ADDRESS>       GET     200  27ms (+start: 7ms) /integrations/ (unauth via ?)
2016-05-01 02:02:54,100 INFO     <IP_ADDRESS>       GET     200  25ms /integrations/ (unauth via ?)
2016-05-01 02:02:55,866 INFO     <IP_ADDRESS>       GET     200  17ms /integrations/ (unauth via ?)
2016-05-01 02:02:57,635 INFO     <IP_ADDRESS>       GET     200  15ms /integrations/ (unauth via ?)
2016-05-01 02:02:59,058 INFO     <IP_ADDRESS>       GET     200  14ms /integrations/ (unauth via ?)
2016-05-01 02:03:00,700 INFO     <IP_ADDRESS>       GET     200  18ms /integrations/ (unauth via ?)
2016-05-01 02:04:16,292 INFO     <IP_ADDRESS>       GET     200  18ms /integrations/ (unauth via ?)
2016-05-01 02:04:22,916 INFO     <IP_ADDRESS>       GET     200  36ms /integrations/ (unauth via ?)
2016-05-01 02:04:24,185 INFO     <IP_ADDRESS>       GET     200  14ms /integrations/ (unauth via ?)
2016-05-01 02:04:25,336 INFO     <IP_ADDRESS>       GET     200  16ms /integrations/ (unauth via ?)
2016-05-01 02:04:26,507 INFO     <IP_ADDRESS>       GET     200  18ms /integrations/ (unauth via ?)
2016-05-01 02:04:27,896 INFO     <IP_ADDRESS>       GET     200  28ms /integrations/ (unauth via ?)
2016-05-01 02:04:29,157 INFO     <IP_ADDRESS>       GET     200  64ms /integrations/ (unauth via ?)
2016-05-01 02:04:30,466 INFO     <IP_ADDRESS>       GET     200  22ms (+start: 6ms) /integrations/ (unauth via ?)
2016-05-01 02:04:31,640 INFO     <IP_ADDRESS>       GET     200  27ms /integrations/ (unauth via ?)
2016-05-01 02:04:33,232 INFO     <IP_ADDRESS>       GET     200  24ms /integrations/ (unauth via ?)

Django template
2016-05-01 02:06:43,378 INFO     <IP_ADDRESS>       GET     200 874ms (+start: 313ms) /integrations/ (unauth via ?)
2016-05-01 02:06:46,199 INFO     <IP_ADDRESS>       GET     200  46ms (+start: 8ms) /integrations/ (unauth via ?)
2016-05-01 02:06:48,107 INFO     <IP_ADDRESS>       GET     200 119ms /integrations/ (unauth via ?)
2016-05-01 02:06:49,385 INFO     <IP_ADDRESS>       GET     200  62ms /integrations/ (unauth via ?)
2016-05-01 02:06:50,647 INFO     <IP_ADDRESS>       GET     200 102ms /integrations/ (unauth via ?)
2016-05-01 02:06:51,972 INFO     <IP_ADDRESS>       GET     200  59ms (+start: 15ms) /integrations/ (unauth via ?)
2016-05-01 02:06:53,445 INFO     <IP_ADDRESS>       GET     200  89ms /integrations/ (unauth via ?)
2016-05-01 02:06:54,931 INFO     <IP_ADDRESS>       GET     200 180ms /integrations/ (unauth via ?)
2016-05-01 02:06:56,428 INFO     <IP_ADDRESS>       GET     200  80ms /integrations/ (unauth via ?)
2016-05-01 02:06:58,074 INFO     <IP_ADDRESS>       GET     200  92ms (+start: 6ms) /integrations/ (unauth via ?)
2016-05-01 02:07:19,723 INFO     <IP_ADDRESS>       GET     200  49ms /integrations/ (unauth via ?)
2016-05-01 02:07:21,999 INFO     <IP_ADDRESS>       GET     200  54ms /integrations/ (unauth via ?)
2016-05-01 02:07:24,093 INFO     <IP_ADDRESS>       GET     200  92ms (+start: 10ms) /integrations/ (unauth via ?)
2016-05-01 02:07:26,311 INFO     <IP_ADDRESS>       GET     200  37ms /integrations/ (unauth via ?)
2016-05-01 02:07:28,325 INFO     <IP_ADDRESS>       GET     200  61ms /integrations/ (unauth via ?)
2016-05-01 02:07:29,776 INFO     <IP_ADDRESS>       GET     200  75ms /integrations/ (unauth via ?)
2016-05-01 02:07:31,311 INFO     <IP_ADDRESS>       GET     200  63ms /integrations/ (unauth via ?)
2016-05-01 02:07:33,138 INFO     <IP_ADDRESS>       GET     200  76ms (+start: 6ms) /integrations/ (unauth via ?)
2016-05-01 02:07:35,046 INFO     <IP_ADDRESS>       GET     200  49ms /integrations/ (unauth via ?)
2016-05-01 02:07:36,926 INFO     <IP_ADDRESS>       GET     200  78ms (+start: 10ms) /integrations/ (unauth via ?)

Hey @timabbott, now this branch performs better. I have just used the code from the current version of Pipeline.

Great, thanks for tracking that down!  Remind me why you were upgrading the version of pipeline in this PR in the first place?
I think this is looking pretty good; I'm hoping to planning this after I do the Zulip 1.3.11 release (in the next couple days) since it seems like the kind of feature that would benefit from not going in right before a release.  Can you cleanup the history for this branch (maybe just makes sense to squash everything into one commit)?
Regardless, I think at some point we're going to need to debug why the newer version of pipeline is so much slower since we eventually need to upgrade to support Python 3 (or migrate to using a different library for managing our static assets, I suppose; see e.g. #693).  Can you at least report a bug upstream about this performance regression in newer pipeline?

Thanks. We were upgrading to a newer version of Pipeline because newer versions contained extensions which supported Jinja2 templates out of the box. Now, I have just used the template tag code to render the output.
Sure, I'll squash to one commit.
Yeah, I agree. I'll report the bug.

@timabbott, for some reason it now fails in Travis CI

@timabbott I am getting this failure in my dev environment on upstream/master.

I think the issue is we added a new dependency to requirements.txt.
-Tim Abbott (mobile)
On May 3, 2016 5:22 AM, "Umair Khan"<EMAIL_ADDRESS>wrote:

@timabbott https://github.com/timabbott I am getting this
https://www.dropbox.com/s/zpg0vzrvjgs0l2x/Screen Shot 2016-05-03 at 5.12.17 pm.png?dl=0
failure in my dev environment on upstream/master.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/zulip/zulip/pull/699#issuecomment-216511487

The production test suite failure is because we need to add python-jinja2 to the list of apt packages in puppet/zulip/manifests/app_frontend_base.pp so it'll be installed in production as well.
Hopefully the version available in Ubuntu Trusty is new enough; worth checking that and maybe pinning our requirements.txt to that version so we're testing the same thing we'll run in prod.

thanks @timabbott, I was going crazy with this one. I'll check the Jinja version in Trusty

@timabbott all good now. Squashed and rebased.

I have also changed the version of Jinja2 to 2.7.2 and of Markupsafe to 0.18 to reflect the production environment.

OK, great, awesome work @umairwaheed!  I'm going to start trying this out on my test server so I can see if the authentication backends work.  There's a few more things I would like to see to happen in order to merge this:

It would be nice to split the refactoring of zproject/settings.py to use the TEMPLATES dict out as a commit before the Jinja2 conversion commit (so the Jinja2 conversion commit would mostly just be adding the new entry to TEMPLATES).  The first commit we can merge immediately while we work out any additional bugs with this branch.
What's the story with the changes you have in the analytics/ code (e.g. with total_at_risk_count)?  Seems like those might be fixing some pre-existing bug.  Can those be broken out into a separate commit from the main conversion commit?
Also, can you add comments explaining the origin of the other two new files under zproject/jinja2/?  If you wrote them, that's great, just want to have correct documentation of where things came from.
Can you open a bug with django-pipeline about the performance issue (if you haven't already) and post the bug ID on this thread?
We need to update the documentation to mention that we're using jinja2.  I think docs/new-feature-tutorial.rst aka http://zulip.readthedocs.io/en/latest/new-feature-tutorial.html and docs/directory-structure.rst at least could use small tweaks on that front).
I think it would also be valuable to add a new section in docs/translating.md about the correct way to add strings to templates for the two templating systems so they'll be included for translating properly.  I think these can be brief, just explaining the trans/blocktrans and _/trans distinctions and linking to the docs for the respective systems.  This isn't a blocked for merging this PR but would be reasonable to include as an extra commit in it.

Does that sound reasonable @umairwaheed ?  Thanks again for all your work on this, I'm very excited about it.

I temporarily upgraded my test production machine to run this, and got this traceback on trying to load the homepage:
2016-05-05 23:39:44,910 ERROR    Internal Server Error: /favicon.ico
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 87, in get_exception_response
    response = callback(request, **param_dict)
  File "/usr/lib/python2.7/dist-packages/django/utils/decorators.py", line 110, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/django/views/defaults.py", line 22, in page_not_found
    body = template.render(context, request)
  File "/home/zulip/deployments/2016-05-05-23-34-36/zproject/jinja2/backends.py", line 53, in render
    return self.template.render(context)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 969, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 742, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/zulip/deployments/2016-05-05-23-34-36/zproject/../templates/404.html", line 1, in top-level template code
    {% extends "zerver/portico.html" %}
  File "/home/zulip/deployments/2016-05-05-23-34-36/zproject/../templates/zerver/portico.html", line 1, in top-level template code
    {% extends "zerver/base.html" %}
  File "/home/zulip/deployments/2016-05-05-23-34-36/zproject/../templates/zerver/base.html", line 26, in top-level template code
    {{ compressed_css('common')|safe }}
  File "/home/zulip/deployments/2016-05-05-23-34-36/zproject/jinja2/compressors.py", line 14, in compressed_css
    return CompressedCSSNode(package_name).render({package_name: package_name})
  File "/usr/lib/python2.7/dist-packages/pipeline/templatetags/compressed.py", line 33, in render
    return self.render_css(package, package.output_filename)
  File "/usr/lib/python2.7/dist-packages/pipeline/templatetags/compressed.py", line 45, in render_css
    return render_to_string(template_name, context)
  File "/usr/lib/python2.7/dist-packages/django/template/loader.py", line 98, in render_to_string
    template = get_template(template_name, using=using)
  File "/usr/lib/python2.7/dist-packages/django/template/loader.py", line 35, in get_template
    return engine.get_template(template_name, dirs)
  File "/usr/lib/python2.7/dist-packages/django/template/backends/django.py", line 30, in get_template
    return Template(self.engine.get_template(template_name, dirs))
  File "/usr/lib/python2.7/dist-packages/django/template/engine.py", line 167, in get_template
    template, origin = self.find_template(template_name, dirs)
  File "/usr/lib/python2.7/dist-packages/django/template/engine.py", line 139, in find_template
    for loader in self.template_loaders:
  File "/usr/lib/python2.7/dist-packages/django/utils/functional.py", line 60, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/usr/lib/python2.7/dist-packages/django/template/engine.py", line 94, in template_loaders
    return self.get_template_loaders(self.loaders)
  File "/usr/lib/python2.7/dist-packages/django/template/engine.py", line 99, in get_template_loaders
    loader = self.find_template_loader(template_loader)
  File "/usr/lib/python2.7/dist-packages/django/template/engine.py", line 122, in find_template_loader
    loader_instance = loader_class(*args)
TypeError: __init__() takes exactly 3 arguments (2 given)

@umairwaheed any ideas what's wrong there?

@timabbott on it.

It would be nice to split the refactoring of zproject/settings.py to use the TEMPLATES dict out as a commit before the Jinja2 conversion commit (so the Jinja2 conversion commit would mostly just be adding the new entry to TEMPLATES). The first commit we can merge immediately while we work out any additional bugs with this branch.

Done.  I have not included the Jinja2 related settings.

What's the story with the changes you have in the analytics/ code (e.g. with total_at_risk_count)? Seems like those might be fixing some pre-existing bug. Can those be broken out into a separate commit from the main conversion commit?

Yeah, so Jinja2 throws an exception if a variable is not defined which made me look in the code and I thought that code was missing the calculation of total_at_risk_count.

Also, can you add comments explaining the origin of the other two new files under zproject/jinja2/? If you wrote them, that's great, just want to have correct documentation of where things came from.

I wrote them myself but I have added more comments.

Can you open a bug with django-pipeline about the performance issue (if you haven't already) and post the bug ID on this thread?

https://github.com/jazzband/django-pipeline/issues/567

We need to update the documentation to mention that we're using jinja2. I think docs/new-feature-tutorial.rst aka http://zulip.readthedocs.io/en/latest/new-feature-tutorial.html and docs/directory-structure.rst at least could use small tweaks on that front).

Done

I think it would also be valuable to add a new section in docs/translating.md about the correct way to add strings to templates for the two templating systems so they'll be included for translating properly. I think these can be brief, just explaining the trans/blocktrans and _/trans distinctions and linking to the docs for the respective systems. This isn't a blocked for merging this PR but would be reasonable to include as an extra commit in it.

Done

@timabbott for the exception can you please to debug remove the Django backend from the TEMPLATES setting in the settings file and then check? I am not sure but it seems that Django backend is kicking in while we need Jinja2 backend to process this. It seems that the error is on /favicon.ico does that mean there you were able to see home page?

i'll try the production version on my side as well.

Regarding that exception: I wasn't able to see the homepage

I merged the prerequisite commits that you broke out (plus a couple more, e.g. adjusting the jinja2 version in requirements.txt).  There will be some small but easy to resolve merge conflicts when you next rebase onto master.  I tested this out on my development environment, and it seems to work pretty well there, so it must be a production-specific problem...

(I pushed my rebase to the jinja2 branch of my repo in case you want to skip resolving merge conflicts)

Commenting out the Django TEMPLATES block gives a new traceback:
2016-05-07 20:52:04,499 ERROR    Internal Server Error: /
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 132, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/zulip/deployments/2016-05-07-20-48-43/zerver/decorator.py", line 216, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/zulip/deployments/2016-05-07-20-48-43/zerver/views/__init__.py", line 884, in home
    request=request)
  File "/home/zulip/deployments/2016-05-07-20-48-43/zproject/jinja2/__init__.py", line 15, in render_to_response
    response = render_to_string(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/django/template/loader.py", line 99, in render_to_string
    return template.render(context, request)
  File "/home/zulip/deployments/2016-05-07-20-48-43/zproject/jinja2/backends.py", line 66, in render
    return self.template.render(context)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 969, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 742, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/zulip/deployments/2016-05-07-20-48-43/zproject/../templates/zerver/index.html", line 1, in top-level template code
    {% extends "zerver/base.html" %}
  File "/home/zulip/deployments/2016-05-07-20-48-43/zproject/../templates/zerver/base.html", line 26, in top-level template code
    {{ compressed_css('common')|safe }}
  File "/home/zulip/deployments/2016-05-07-20-48-43/zproject/jinja2/compressors.py", line 14, in compressed_css
    return CompressedCSSNode(package_name).render({package_name: package_name})
  File "/usr/lib/python2.7/dist-packages/pipeline/templatetags/compressed.py", line 33, in render
    return self.render_css(package, package.output_filename)
  File "/usr/lib/python2.7/dist-packages/pipeline/templatetags/compressed.py", line 45, in render_css
    return render_to_string(template_name, context)
  File "/usr/lib/python2.7/dist-packages/django/template/loader.py", line 98, in render_to_string
    template = get_template(template_name, using=using)
  File "/usr/lib/python2.7/dist-packages/django/template/loader.py", line 46, in get_template
    raise TemplateDoesNotExist(template_name)
TemplateDoesNotExist: pipeline/css.html

OK I figured out the __init__ traceback -- it turns out the syntax for using the cached LOADERS is a bit different from what we thought.  Here's the fix: 73b3f7a26efa3dc93b1c0ab3e9a8606581faff0c

OK, the jinja2 branch in my repository now works in a production environment; I assume the TemplateDoesNotExist: pipeline/css.html was just caused by commenting out the Django templating system not being correct.
@umairwaheed on my server I was able to test Google auth works correctly.
Is there anything else you want to investigate or test before we merge this branch?

Also, inspired by this being a bit difficult to debug, I'm working on adding to the production test suite checks for whether the server is working, since that would have caught this issue in testing (see #598).

Is there anything else you want to investigate or test before we merge this branch?

No, I think it's good to go.

@timabbott push your branch to this PR.

OK, I did a bit more cleanup on the docs and commit message, and broke out the few remaining small changes that aren't part of the overall transition, and merged this (from my branch).  Thanks @umairwaheed for all your work on it!  This has a significant impact on performance.
I'll email the zulip developers' list so folks know to watch out for template issues over the next few weeks :)

Cool :)


[bug] sheets gets blank on input focus (native expo)

Happens on native android at least. Focusing on input text inside a sheet dialog makes the whole sheet go blank.
To Reproduce
Just paste the example from the doc https://tamagui.dev/docs/components/dialog/1.0.0
Run on expo android for example, open the dialog and press on the Name input
Sheet goes blank
Demo:
https://user-images.githubusercontent.com/26435/215271937-b10db209-b846-417f-b7e3-b46b7718d7e8.mp4
System Info:
Output of npx envinfo --system --npmPackages --binaries --browsers
System:
OS: Linux 5.15 Ubuntu 22.04.1 LTS 22.04.1 LTS (Jammy Jellyfish)
CPU: (8) x64 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz
Memory: 1.63 GB / 31.06 GB
Container: Yes
Shell: 5.1.16 - /bin/bash
Binaries:
Node: 19.3.0 - ~/.nvm/versions/node/v19.3.0/bin/node
Yarn: 3.2.3 - ~/.nvm/versions/node/v19.3.0/bin/yarn
npm: 9.2.0 - ~/.nvm/versions/node/v19.3.0/bin/npm
Browsers:
Brave Browser: beta
Chrome: 108.0.5359.124
Firefox: 109.0
npmPackages:
@babel/plugin-proposal-export-namespace-from: ^7.18.9 => 7.18.9
@babel/runtime: ^7.18.9 => 7.20.7
@ethersproject/providers: ^5.7.2 => 5.7.2
@ethersproject/shims: ^5.7.0 => 5.7.0
@expo/vector-icons: ^13.0.0 => 13.0.0
@expo/webpack-config: ^0.17.2 => 0.17.4
@gnosis.pm/safe-modules-deployments: ^1.0.0 => 1.0.0
@gnosis.pm/safe-react-gateway-sdk: ^3.5.2 => 3.5.2
@manypkg/cli: ^0.19.1 => 0.19.2
@react-native-async-storage/async-storage: ^1.17.11 => 1.17.11
@safe-global/safe-core-sdk-types: ^1.8.0 => 1.8.0
@safe-global/safe-gateway-typescript-sdk: ^3.5.5 => 3.5.5
@tanstack/query-async-storage-persister: ^4.22.4 => 4.22.4
@tanstack/react-query: ^4.22.4 => 4.22.4
@tanstack/react-query-persist-client: ^4.22.4 => 4.22.4
atob: ^2.1.2 => 2.1.2
base-64: ^1.0.0 => 1.0.0
date-fns: ^2.29.3 => 2.29.3
eslint: ^8.21.0 => 8.31.0
ethers: ^5.7.2 => 5.7.2
expo-barcode-scanner: ^12.1.0 => 12.2.0
expo-camera: ^13.1.0 => 13.1.0
expo-constants: ^14.0.2 => 14.1.0
expo-modules-core: ^1.1.1 => 1.1.1
node-libs-react-native: ^1.2.1 => 1.2.1
prettier: ^2.7.1 => 2.8.2
react-hook-form: ^7.42.1 => 7.42.1
react-native-get-random-values: ^1.8.0 => 1.8.0
react-native-svg-transformer: ^1.0.0 => 1.0.0
semver: ^7.3.8 => 7.3.8
turbo: ^1.4.2 => 1.7.0
typescript: ^4.7.4 => 4.9.4
wagmi: ^0.10.10 => 0.10.10

tested in latest version and it's fixed so closing this. Thanks!

nvm, still happening

To be precise, it doesn't happen anymore when I remove the  tag from the dialog
-      <Adapt when="sm" platform="touch">
-        <Sheet zIndex={200000} modal dismissOnSnapToBottom>
-          <Sheet.Frame padding="$4" space>
-            <Adapt.Contents />
-          </Sheet.Frame>
-          <Sheet.Overlay />
-        </Sheet>
-      </Adapt>
-

if you add this on your Input ?
<Input
  onFocus={(e) => e.stopPropagation()}
  ...
/>

I know that is not good solution but work and is not blocking

Hello, same issue here, with the documentation code exemple (like @patcito)
@lobor Your solution doesn't work on my app.
@patcito What do you mean by removing tag from the dialog?

I recently encountered another variation of this issue in my app. In my case, instead of an <Input> inside of a Sheet, I have a <WebView> inside a Sheet containing a page with a text input.
As soon as the user focuses on an input in the webview and the keyboard appears, the webview refreshes itself -- behaving almost as if that instance of the component had been invalidated by a new key prop.
I'm only able to reproduce this issue on Android (not on iOS) and only when the sheet has the modal property set to true. Non-modal sheets behave normally on all platforms.
This makes me think that the problem is somehow related to the custom native implementation of portal (@tamagui/portal->Portal.native.tsx)? Still just a hunch at this stage though. I'm going to try to find some time to dig a bit deeper into finding out what ultimately is the root cause(s).

Hey there,
Similar to what @nderscore reported.
I'm using  inside of a Sheet. When the input is focused the keyboard appears, the sheet shifts up on the screen and shortly after (with no further interactions) it resets: keyboard closes + the sheet shifts back down.
I would agree that the issue occurs "when the sheet is in a flex container which flex-grows to the height of the screen". Wrapping the sheet in a Stack with the height set to the height of the window completely resolves the issue.
Tested on android

yeah same issue here, when putting the input text in the sheet

temporarily you can add this in expo configuration (app.json / expo.json)
    "android": {
      "softwareKeyboardLayoutMode":"pan"
    }

I think we added some docs on how to avoid this on the Sheet docs page in a different way too (if I understand correctly).
@nderscore let me know if you found anything here.

@natew The sheet being blanked was caused by the sheet conditionally un-mounting its contents during a resize.
That's been fixed in #787

I think the only other concern here was how to handle keyboard avoiding for <Sheet modal>
I saw that moveOnKeyboardChange was added as a Sheet prop recently, which I believe solves for that case, but admittedly I haven't tested it out yet. For my use case, I ended up using a non-modal sheet and doing my own keyboard avoiding on the parent container.

https://user-images.githubusercontent.com/6276406/234624861-68b4dc61-1d1d-4c74-9f27-b2f0623a8f37.mov
So i currently have an input in a dialog, when modal is set to true. I get what is shown in the video, when I set modal to false its completely fine.

i think this is fixed, but re-open with new repro if that other issue still exists


Skip Intro disappears when ignored once

I'm submitting a ...

[x] bug report
[ ] feature request
[ ] support request

General infomration
Prerequisites

[x] Are you running the latest version?

Description
After ignoring the Skip Intro feature for an episode subsequent episodes fail to offer the option to skip the intro.
Steps to Reproduce

Play an episode without skipping the intro
Play another episode
The Skip Intro button fails to appear

Expected behavior:
I expect the skip Intro feature to appear every time, regardless of earlier (non)actions
Context (Environment)
Kodi Leia nightly running on OSMC, Vero 4k+
Installation

[ ] I installed the plugin via zip from the Releases page
[x] I´m using the Netflix Repo
[ ] I´m using a different source (Please tell which)

Can confirm this issue persists in v0.13.21. No hurry though, it's just a minor bug.


CBMC: Update to Version 6

Bump CBMC to freshly released version 6.

@potsrevennil @mkannwischer Out of curiosity I added the benchmark tag here, but it didn't trigger the CI. Is this deliberate?

This is wierd, this should not happened. I expected that after adding the benchmark label, click rerun job, then the bench workflow should just run. I'll investigate this tmr.

This is wierd, this should not happened. I expected that after adding the benchmark label, click rerun job, then the bench workflow should just run. I'll investigate this tmr.

Here is another think to look at: https://github.com/pq-code-package/mlkem-c-aarch64/actions/runs/9595340954 reports:
MLKEM512
keypair cycles=139579
encaps cycles=173250
decaps cycles=223791
CRYPTO_SECRETKEYBYTES: 1632
CRYPTO_PUBLICKEYBYTES: 800
CRYPTO_CIPHERTEXTBYTES: 768

MLKEM768
keypair cycles=0
encaps cycles=0
decaps cycles=0
CRYPTO_SECRETKEYBYTES: 2400
CRYPTO_PUBLICKEYBYTES: 1184
CRYPTO_CIPHERTEXTBYTES: 1088

MLKEM1024
keypair cycles=377669
encaps cycles=428755
decaps cycles=512516
CRYPTO_SECRETKEYBYTES: 3168
CRYPTO_PUBLICKEYBYTES: 1568
CRYPTO_CIPHERTEXTBYTES: 1568

@cothan Let's avoid merge commits if possible, and instead rebase PRs.

@hanno-becker , got it. I wish there is a rebase button at the Github UI.

I have reproduced all proofs here from this branch, and all is well, so happy.

@hanno-becker , I change the github setting, there is a button to rebase the branch in Github UI.

@hanno-becker I have a question maybe not that related to this PR.
I noticed that in here
https://github.com/pq-code-package/mlkem-c-aarch64/blob/95615fc0c742cf4b3ace32e0578d830d9447751a/.github/workflows/ci.yml#L117-L122
Since we are using bash with the -e flag, seems like if the previous proof failed, the following proof will not be executed, is this as expected ?

@potsrevennil I don't think this was intentional, but upon consideration, it may be the right call -- those CBMC proofs tend to be computationally expensive, and each line still conducts all proofs for a fixed K=2,3,4.

@hanno-becker OK, make sense! Other parts LGTM.
There is still some issue for the benchmarking where @mkannwischer pointed out, but it can be further discussed in other issue or pr I think.

@potsrevennil Yes, I agree we don't want merge commits (see comments above). I rebased now.

@hanno-becker Thanks!


Translate: Detect if array was used as input

Resolves #2505

LGTM. Will merge after green CI.

Coverage remained the same at 100.0% when pulling ca2adf83bbdcf3754c680d23f80a9615c9ed8871 on landrito:translate-array-fix into d0fe4db5a415c7a2fe6428140038a1826c7b8739 on GoogleCloudPlatform:master.


Add group name in task metrics

The subsystem name metric label improves observability of task CPU usage in Grafana - https://github.com/paritytech/polkadot/issues/4198
Added two new spawn methods to SpawnNamed trait. These allow the caller to also specify a subsystem name when starting a task. While the change doesn't break any existing API, it changes the way metrics are logged by adding an extra label. All code using the original spawn methods will default to having subsystem set to substrate-unspecified.
Polkadot companion: https://github.com/paritytech/polkadot/pull/4239

What do you think of naming it group instead of subsystem? Because in the end it is something to group different tasks?

Sounds better, indeed we just want to group by.

Synced offline with @bkchr and decided to drop the new trait methods and just add a the group to the existing spawn() methods.

Same as paritytech/polkadot#4239 (comment)
You might also need cargo +nightly fmt.

Yep, this seems odd, there is a version mismatch of some sort. I'll rerun.

bot merge

bot merge


Fixed vector access for prereq reads

Tested before and after modification. Before, any feat with prereq values (or even attempting to write in there manually) crashed the code. After, game fully reaches main menu and plays without error.

I also changed the unwield function in python layer to be able to have an optional argument to drop item instead of putting in inventory. Tested ingame and it works.

Changed the call to an optional integer flag and added a new overload for drop to use it with. Tested in game and it drops, didn't get any build errors or anything so unsure why I've got the red x above.


Add page context to error logging in rendering

Add logging of the errors/warnings which rst2html outputs to its stderr
stream when rendering rst documents. Note that rst2html outputs warnings
and errors to stderr but it also adds them to the generated html. ->
hugo logs everything in stderr as error.
Add / complete adding page context (path to file being rendered) to
anything logged by getRstContent and getAsciidocContent.
See #2570

Exemplary output with the changes in 7a5c7dee22e89995ed266d37db674c845cbce0c3:
<snip/>
INFO: 2016/10/17 12:14:52 content.go:608: Rendering post/test2.rst with /usr/bin/rst2html ...
INFO: 2016/10/17 12:14:52 content.go:608: Rendering post/demo.rst with /usr/bin/rst2html ...
ERROR: 2016/10/17 12:14:52 content.go:620: post/test2.rst:7: (WARNING/2) Definition list ends without a blank line; unexpected unindent.
ERROR: 2016/10/17 12:14:52 content.go:620: post/demo.rst:547: (WARNING/2) Definition list ends without a blank line; unexpected unindent.
ERROR: 2016/10/17 12:14:52 content.go:620: post/demo.rst:89: (ERROR/3) Undefined substitution referenced: "problematic".
<snip/>


[kong] Add option to enable HorizontalPodAutoscaler

Default disabled

@hbagdi Would you like mt to update this PR with .Capabilities?

@yasn77 Yes. We need to make it compatible with v1 and v2 version of HPA before we can merge this in.

@hbagdi Updated to support both autoscaling/v1 and autoscaling/v2beta2

@hbagdi Branch updated. Thanks for your comments :+1:

I was about to approve but this is missing CI test.
Can you update one of the ci test files and set autoscaling.enabled: true?

I was about to approve but this is missing CI test.
Can you update one of the ci test files and set autoscaling.enabled: true?

@hbagdi Added and all tests :green_heart:
Thanks

@yasn77 Thank you for your contributions!
As a token of appreciation of your efforts on this and other PRs to our Helm Chart, we would like to send some swag your way if you fill out the following form:
https://docs.google.com/forms/d/e/1FAIpQLSfacQHac-PYIDIrDOi_W2l7cWxBDRIVBs6xE5HnlojPvdhL9g/viewform

@hbagdi That's really appreciated.
And thanks again to the Kong team!


ACNA-1756 | Config Schema Changes for Integration Type ApiKey

Description
Issue:
Expected Behaviour

I expect the cli to generate a app builder project
Actual Behaviour

 ›   Error: Missing or invalid keys in config: [
 ›     {
 ›       "keyword": "required",
 ›       "dataPath": ".project.workspace.details.credentials[0].oauth2",
 ›       "schemaPath": "#/definitions/oauth2/required",
 ›       "params": {
 ›         "missingProperty": "client_secret"
 ›       },
 ›       "message": "should have required property 'client_secret'"
 ›     },
 ›     {
 ›       "keyword": "format",
 ›       "dataPath":
 ›   ".project.workspace.details.credentials[0].oauth2.redirect_uri[0]",
 ›       "schemaPath":
 ›   "#/definitions/oauth2/properties/redirect_uri/items/format",
 ›       "params": {
 ›         "format": "uri"
 ›       },
 ›       "message": "should match format \"uri\""
 ›     }
 ›   ]

Implementation:
New response will not have oauth2 object block for integration_type = apiKey in credentials array
Instead, it will have apikey = client_id_val directly in the schema
New response
{
  "workspace": {
    "details": {
      "credentials": [
        {
          "id": "Project_id",
          "name": "Project_NAME",
          "integration_type": "apikey",
          "apikey": "client_id_value"
        }
      ]
    }
  }
}

Related Issue
ACNA-1756
Motivation and Context
This is a very critical bugfix to patch breaking app dev
How Has This Been Tested?

yarn test
Local link with aio-cli

Screenshots (if appropriate):

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] I have signed the Adobe Open Source CLA.
[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

Converted to Draft because this shouldn't get pulled in yet until the backend changes are ready. Once those are ready, we can change the status of this PR.


Allow specifying custom class and styles

Allows to specify classes on <Card>
Allows to specify styling on <Card>
Adds card__front global class (not strictly necessary as .card:not(.card__back) works also)

Also, .card conflicts with .card in bootstrap, an extremely common library.  I suggest changing it to .playingcard.

Good comments. I’ll update the PR tomorrow. Thanks!

Let me know if you suggest renaming the front/back classes also; otherwise I’ll leave that as-is

Yes, I think front / back classes should be updated accordingly, e.g. react-playing-card__front

Done.  I also ran npm install and it recompiled the files in the lib folder.


:bug: fix minor bug in FPU MUL instruction

Fix invalid instruction logic (if any operand is sNAN).
Bug identified by @racodond in #1024.
@mikaelsky, maybe this is also interesting for you :wink:

Interesting.. wonder why it was caught by the compliance coverage test. Will add it to my internal test list.


No launchers seem to be installing

This should be hard to mess up, right? Launched desktop file, from desktop. Selected rockstar social club, ps plus and ubisoft connect. Click 'Ok'. I see a progress bar of things downloading and then i see steam restart. Nothing new appears in non-steam games. Tried this about 4 times, even just rockstar by itself, no dice. (dear god why is it so impossible to install non-steam games to SD cards kill me)

@blaeeeeek alot of Code was moved around this past week, try to use the .sh file from 3.7

I had the same problem and removed Firefox explorer from the system and installed Chromium. After that the launchers were installed and appeared in Steam.

Hi. I's not launching for me either. attaching my log. i can't find it in the steam library
NonSteamLaunchers-install.log

I had the same problem and removed Firefox explorer from the system and installed Chromium. After that the launchers were installed and appeared in Steam.

DId you install Chrome or Chromium? Would Brave or Ungoogled Chromium serve the same purpose?

@gitscosh for some reason the logic of the code of the Google Chrome installation in the nsl script is blocking the installation of the launchers, it shouldn't be that way I still need to fix this

@gitscosh for some reason the logic of the code of the Google Chrome installation in the nsl script is blocking the installation of the launchers, it shouldn't be that way I still need to fix this

Ok. The Battlenet launcher install worked just fine, but I can't get the GOG Galaxy or Xbox installs to work properly. No entries in Steam. I'll wait for an update.

@gitscosh send your log that could be a number if reasons

@gitscosh send your log that could be a number if reasons

Sure thing. On the Steamdeck, which log do you want? Just the main one in the top level folder? Do you wanted it pasted here?

@gitscosh upload the nsl log in the downloads folders

NonSteamLaunchers-install.log

@gitscosh still waiting for all the fixes and approval on the decky team

This is now resolved as of today (3.8.7 was released today), but I had to start from scratch and clean out the old stuff, and I used the Deck Plugin. Now all I have to do is figure out how to get the mouse to work in GOG Galaxy... or find another way to navigate. Any ideas?

@gitscosh let's goooo baby!! I'm glad it's working ... NSL used to supply three controller configuration, and we still do but it's in the process of rewriting, so for now just use a template in the controller setting called Web Browser that's what I use for the launchers. It works very good.


fix(core): fix a deadlock issue on coroutine mutex

Summary

fix(core): fix a deadlock issue on coroutine mutex
Coroutine mutex uses semaphore to ensure only one job
is running for a specific event (e.g. reconfigure_handler).
However, if the coroutine job is wrapped within a light thread
(e.g. worker event's event_thread) and the associated thread
was killed in the middle of the job, semaphore post followed would
be skipped.
Light threads would be killed when it cannot connect to the worker
0 which is common if the worker 0 is OOM Killed by OS.
All subsequent jobs of the same event would wait and timeout, as
there is not any semaphore resource available.
This deadlock currently affects DB-less mode as Hybrid mode has
the safeguard at https://github.com/Kong/kong-ee/blob/054b204b378c8a4c985ffac06ccef6fbd15d16cf/kong/clustering/data_plane.lua#L300.
Please refer to the analysis here https://konghq.atlassian.net/browse/FTI-5930?focusedCommentId=131903.
Checklist

[ ] The Pull Request has tests
[x] A changelog file has been created under changelog/unreleased/kong or skip-changelog label added on PR if changelog is unnecessary. README.md
[ ] There is a user-facing docs PR against https://github.com/Kong/docs.konghq.com - PUT DOCS PR HERE

Issue reference

Fix #FTI-5930

Need a change log?

Need a change log?

Done.

which is common if the worker 0 is OOM Killed by OS.

In this case, isn't the whole worker restarted, aka no deadlock as everything is gone. I feel like this issue must be related more to not that worker is killed but to fact that if read or write thread dies, the events thread may be prematurely killed. In which case the worker is not restarted and the lock is retained.

@outsinre, @chronolaw, @samugi, what do you think about this:
https://github.com/Kong/lua-resty-events/pull/44

@outsinre, @chronolaw, @samugi, what do you think about this: Kong/lua-resty-events#44

I just checked the PR over there. It treats the read_thread specially. But the other two threads may have similar issues.
I think the PR on Kong Gateway side can still serve as a safeguard.

which is common if the worker 0 is OOM Killed by OS.

In this case, isn't the whole worker restarted, aka no deadlock as everything is gone. I feel like this issue must be related more to not that worker is killed but to fact that if read or write thread dies, the events thread may be prematurely killed (that is in middle of event execution). In which case the worker is not restarted and the lock is retained (and thus deadlocking).

The case is if worker 0 was dead (restarted), it would lock other workers.

The case is if worker 0 was dead (restarted), it would lock other workers.

When we are talking about coroutine mutexes, they cannot lock other workers. Period.

There are different scenarios the read/write threads might fail, not sure if it is a good idea we should let the event thread still running.

I have a feeling that you don't understand the issue.

I think the PR on Kong Gateway side can still serve as a safeguard.

It is not a safe guard. It is a bug.

thread 1 starts and gets coroutine mutex, the available mutexes will be 0
thread 2 starts and sees that mutex is <= 0 so it posts to it, then starts waiting for lock and it gets its own mutex (the thread 1 is still running)

@outsinre, @chronolaw, @samugi, what do you think about this: Kong/lua-resty-events#44

I just checked the PR over there. It treats the read_thread specially. But the other two threads may have similar issues.

No. There is a reason why it treats read_thread specially. On any thread quit, we want to stop receiving new events ASAP. The thread read does nothing else than synchronous non-yielding event push.
But it is okay to say that there might still be interdependencies between events thread and write thread. Or event events queue. E.g. event may generate more events, which we can ask that do those need to be written when the read thread has already quit. Or that should we process all the events that have been queued before quitting the events thread? And there are more edge cases like this.

@outsinre, @chronolaw, @samugi, what do you think about this: Kong/lua-resty-events#44

I just checked the PR over there. It treats the read_thread specially. But the other two threads may have similar issues.

No. There is a reason why it treats read_thread specially. On any thread quit, we want to stop receiving new events ASAP. The read thread does nothing else than synchronous non-yielding event queue push.
But it is okay to say that there might still be interdependencies between events thread and write thread. Or event events queue. E.g. event may generate more events, which we can ask that do those need to be written when the read thread has already quit. Or that should we process all the events that have been queued before quitting the events thread? And there are more edge cases like this. (I have not checked what happens to queue when threads are restarted).

When the threads are restarted, the events are gone. Kong loses them. This has been an issue since https://github.com/Kong/lua-resty-worker-events.

I think the PR on Kong Gateway side can still serve as a safeguard.

It is not a safe guard. It is a bug.
1. thread 1 starts and gets coroutine mutex, the available mutexes will be 0

2. thread 2 starts and sees that mutex is <= 0 so it posts to it (making available mutex count == 1), then starts waiting for mutex and it gets its own mutex that it just posted (the thread 1 is still running -> and now thread 1 and thread 2 are executing at the same time, just what the concurrency tries to protect)

For "safeguard", I mean Kong Gateway should be robust enough to consider the lock bug, putting aside the worker library. Let's say, other libs or modules might trigger the same issue, Kong Gateway should consider that.

Currently, the thread only post after the pcall(fn) (e.g. fn is reconfigure_handler).

thread 2 starts and sees that mutex is <= 0 so it posts to it

That is the point of this PR, we must let a thread to post 1.

Closed in favor of https://github.com/Kong/kong/pull/13019.


Importing dates

Bug description
I've got a handful of posts (in a .csv) with the date formatted like this:
postDateTime
2020-04-01 12:39:00
2020-06-15 14:03:00
2020-05-05 18:27:00
2020-12-09 15:43:00
2020-12-10 8:36:00

postDateTime from my csv is mapped to date in Statamic
The date in my blueprint looks like this
            handle: date
            field:
              type: date
              required: true
              default: now
              validate:
                - required
              time_enabled: true
              time_seconds_enabled: true

When I run the import, I get this error:
local.ERROR: Unexpected data found.
Unexpected data found.
Not enough data available to satisfy format {"exception":"[object] (Carbon\\Exceptions\\InvalidFormatException(code: 0): Unexpected data found.

The import works if I remove the date mapping.
How to reproduce
Try to import dates in this format
postDateTime
2020-04-01 12:39:00
2020-06-15 14:03:00
2020-05-05 18:27:00
2020-12-09 15:43:00
2020-12-10 8:36:00

Logs
No response
Environment
Environment
Application Name: Statamic
Laravel Version: 11.29.0
PHP Version: 8.3.8
Composer Version: 2.6.5
Environment: local
Debug Mode: OFF
URL: vocal.test
Maintenance Mode: OFF
Timezone: Europe/London
Locale: en

Cache
Config: NOT CACHED
Events: NOT CACHED
Routes: NOT CACHED
Views: CACHED

Drivers
Broadcasting: log
Cache: redis
Database: sqlite
Logs: stack / single
Mail: smtp
Queue: redis
Session: file

Statamic
Addons: 3
Sites: 1
Stache Watcher: Enabled (auto)
Static Caching: Disabled
Version: 5.35.0 PRO

Statamic Addons
jonassiewertsen/statamic-jobs: 1.5.0
statamic/importer: 1.0.2
visuellverstehen/statamic-anchor-navigation: 1.0.0

Additional details
No response

I updated my blueprint, removed time_enabled: true and  time_seconds_enabled: true
            handle: date
            field:
              type: date
              required: true
              default: now
              validate:
                - required

and changed postDateTime to just date
postDateTime
--
2020-04-01
2020-06-15
2020-05-05
2020-12-09
2020-12-10

It works now. Would be handy to be able to import date and time in that format though.

Looks like this is because of 2020-12-10 8:36:00. It's 8: but probably should be 08:.
We might need handling to define what date format is being used in the csv fields.


Vector type fix for fit prior

Also includes option to add a seperate bspoly_params for d.

still need to fix smth so the test doesn't fail


Can't open https://www.rutracker.org

As I can judge it happens because of www.., but I have no idea why...

I have the same thing happening. http://rutracker.org it doesn't open and it's not related to www. When I enter the address, I get a stub about the blocking from the provider. Firefox, Ubuntu 20.04, proxy server is enabled in the extension.


Spec basic curation table that will work for both versions of pipeline.  @dosumis

Switch to using IRIs based on accessions.
Equivalent class expressions split out into a separate table.
Simplified curation table (e.g. https://github.com/obophenotype/brain_data_standards_ontologies/blob/master/src/templates/CCN202002013_class.tsv)
Various aliases should be rolled into a comment in this template.  Comment can make clear type of alias.

STATUS: IN PROGRESS
This aim simplifies the current curation tables, e.g. https://github.com/obophenotype/brain_data_standards_ontologies/blob/master/src/templates/CCN202002013_class.tsv
Aim: A partially filled robot template tsv that can be used to manually curate labels, classification and properties of classes defined with reference to nodes in the taxonomy.  Classes and properties will be linked via subClassOf axioms.
A second template will provide  Equivalent Class definitions for the same terms following one of two patterns
Pattern1 : Following https://github.com/obophenotype/brain_data_standards_ontologies/blob/master/src/templates/CCN202002013_class.tsv
Pattern2: Link To ticket TBA
Robot template spec:
ID -> IRI of form  http://www.semanticweb.org/brain_data_standards/AllenDendClass_{accession e.g. CS201810310_1}
label: curated field
synonyms_from_taxonomy: Pre-rolled text list field (multiple entries separated by |). Source fields.
curated_synonyms: curated text list field (multiple entries separated by |)
classification: Curated field
comment: Pre-rolled text from the fields in taxonomy. Recipe
classification_comment: Curated text field
classification_pub: curated text list field (multiple entries separated by |)

@hkir-dev - please review

Quesitons on curation table:

Does "content: curated ..." means template field will be left empty for manual curation or get data from some existing manually curated table?
How should we handle empty strings for Synonyms_from_taxonomy, should we ignore? Such as cell_set_additional_alias sometimes has "" as value.

curation table updated: new table
Equivalent classes template with the reification pattern added.


Update composition.md

I made the following edits to this page:
Removed repeated sentence.
Corrected spelling

Closing this PR as I have made another pr to contribute the fix.
thank you @kazupon


[WIP] MAUI Interop

What does the pull request do?
Add support for using avalonia controls inside maui. Currently, only android is supported.
What is the current behavior?

What is the updated/expected behavior with this PR?

How was the solution implemented (if it's not obvious)?

Checklist

[ ] Added unit tests (if possible)?
[ ] Added XML documentation to any related classes?
[ ] Consider submitting a PR to https://github.com/AvaloniaUI/Documentation with user documentation

Breaking changes

Obsoletions / Deprecations

Fixed issues

This is very interesting and unexpected 👍

Closing this, as project will be moved to it's own repository


Register-PSRepository needs to support -Scope AllUsers

Summary of the new feature / enhancement
We have a secure environment and all of registries are private proxies which have to be registered with PowerShell in order to install approved packages.  So, everyone in out environment need to register the same repositories.  It would be more efficient if we could register or configure those proxies for all users of a system so individuals don't need to do that.
If there are registries registered for all users, then the default nuget registry should not need to be installed by a user in order to install packages.
Proposed technical implementation details (optional)
No response

Thanks for reaching out @RobCannon, this is a duplicate issue of #84 so closing this issue. We will add this feature in soon.


Avoiding the testcase workaround

Hello,
as you're aware, hiding the code generation in testcases is a bit of a hack and probably annoying for anyone actually using their tests for other matters. Like testing.
If you haven't seen it, https://github.com/Wulf/tsync is taking a different approach: Instead of generating a meaningful #[derive(TS)] at build time, it will parse your source code independently of the compiler. But both tsync and ts-rs use the syn crate, and will parse a syn::Item::Struct/::Enum into type information.
Their approach has a few advantages:

doesn't mess with test cases
no build overhead to generate meaningful #[derive(..)] (though that is probably minor)
it's easier to regenerate .d.ts files on demand (just run tsync instead of all your tests)

But let's not hide the disadvantages:

they currently require global installation of a tsync binary. This could be avoided by letting users add a one-liner [[bin]] or [[example]] to their project and calling that. (an example has the advantage that you can put its deps into dev-dependencies)

I was wondering if it'd be useful to better abstract the type parsing away from the type consumption. We create a function taking a syn::Item and returning a struct DerivedTS (or something more generic than it).
Using that function, we could implement both the actual #[derive(TS)] and testcase generation, but we could also implement an external parser similar to tsync for projects where that's a better fit.
Do you think that'd be useful? Am I overlooking something important? Is this something I should hack on the next slow weekend, or over the holidays?

Hey!
I've just taken a look at tsync, and it's a completely different beast than ts-rs. It naively takes parsed idents and tries to convert them to TS. Here is an excerpt of the source:
match identifier.as_str() {
  "i8" => "number".to_string(),
  "u8" => "number".to_string(),
  "i16" => "number".to_string(),
  "u16" => "number".to_string(),
  ...
  "NaiveDateTime" => "Date".to_string(),
  "DateTime" => "Date".to_string(),
  ...
  _ => identifier.to_string()
}

While this probably works for simple usecases, it's really limiting since the tool has no idea what the types are - it just echoes their identifier to typescript.
This approach can't support  things like renaming types, inlining types, flattening types and many more.
Also, it's easy to break - Type aliases or shadowed identifiers would just result in wrong ts being generated.
ts-rs takes a different approach, defering the actual generation of typescript to runtime. With this, we can make use of rusts name resolution/type checking to properly support all those features. It lets you flatten, inline, rename, ... your types and doesn't get confused when you use type aliases or shadow identifiers.
More concretely: DerivedTS is basically the information needed to generate an impl TS for T block. So at this point, there are no actual bindings generated, but just something like
#[derive(TS)]
#[ts(rename = "SomeType")]
struct T {
  a: i32,
  #[ts(flatten)]
  b: Something,
}

impl TS for T {
   ...
  fn name() -> String {
    "SomeType".to_owned()
  }
  fn inline() -> String {
    format!("interface T { a: {}, {} }", <i32 as TS>::name(), <Something as TS>::inline_flattened())
  }
  ...
}

So, tldr: ts-rs depends on doing stuff at runtime, and for good reason.
However, I'd love to make the workaround more paletable - Maybe we should gate the tests which export typescript behind a feature flag or some other way of easily disabeling them/running them. (That's currently possible too by excluding export_typescript_* tests, but it's not super obvious).

Good points, thanks for taking the time to explain. You aren't just parsing the tokens, you're re-inserting the tokens into the source code to get the compiler to do important work on them. So the external parser won't work.
If we cannot pull type generation out of the code, I'll try to find a workaround with feature gates. The goal is that

running tests doesn't generate typescript
generating typescript doesn't run other tests
no deps or features are introduced into the app. proc-macros and dev-dependencies are fine, as cargo can isolate them.

I'll play around with it as time permits and check back with you.

Awesome, thanks. Really apreciate it.
Regarding point 1 & 2: Filtering by name (currently the tests start with export_bindings i think), using #[ignore] and cargo test -- --ignored, or some conditional compilation mechanism comes to mind.

Success.

ts-rs-macros gets a feature called export-bindings. If not set, generate_export_test does nothing, and no tests are generated.
ts-rs gets the same feature, which is passed through to ts-rs-macros. If not set, TS::export does not exist.

Running cargo test does not generate bindings, as the feature isn't set. Keeping this working without requiring additional parameters is good for those using the cli.
Running cargo test --features "ts-rs/export-bindings" export_bindings_ runs just the exports, but not any other tests. That's a bit long, but it would go into a build script.
dprint-plugin-typescript can be made optional, depending on export-bindings. That might conflict with #58 - I don't think we can enable a dependency only when two other features (export-bindings + pretty-print) are enabled. But for the goal of keeping dprint out of the app's dependency graph, maybe this is enough?
This works best if the feature "export_bindings" is not set by default, but that's a breaking change. Otherwise we'd have to tell users to manually disable the feature when specifying the ts-rs dep.
But I'm wondering, how far should we go here? Right now, the macro derives are still generated outside of tests. We could disable that to avoid the compile time penalty. Are you aware of any users consuming the traits other than for exporting? Depending on the answer, we might turn the whole crate inert when export-bindings is off, or we could gate trait generation behind a second feature flag. Or we could leave it as is.

You can also opt for a more manual approach and write lines of format!("export {}", MyType::decl()) into a file yourself in any function that you like, without any export annotations. I wrote the following macro to help out with that and it seems to work fine.
macro_rules! export {
    ($target:expr, $($types:ty),*) => {
        {
            let target = $target;
            fn _export(target: &mut impl ::std::io::Write) -> ::std::result::Result<(), ::std::io::Error> {
                $(
                    writeln!(target, "export {}\n", <$types as ::ts_rs::TS>::decl())?;
                )*
                Ok(())
            }
            _export(target)
        }
    };
}

fn some_function() {
    let mut target = std::fs::File::create("my-bindings.ts").unwrap();
    let _ = export! {
        &mut target,

        SomeType,
        AnotherType,
        /// ...
    }
}

@Heliozoa with #[ts(export)], you get quite a bit more than format!("export {}", MyType::decl()), like imports and formatting.

This has been inactive for a very long time and there have not been any PRs aiming at implementing this.
Also, this can somewhat be achieved by adding a feature flag to your own crate and hiding #[derive(TS)] (and even the whole ts-rs crate) behind it, so I think it's safe to close this issue


Fix failing tests due to new versions of sklearn and scipy

Affected classes:

CClassifierRidge: fails due to default parameter max_iters that should be integer
CExplainerInfluenceFunctions: fails due to pinv2 function being deprecated

Fixed in v0.15.2


Dust3D

Project description
Making 3D Model Has Never Been So Easy
Auto UV unwrapping, auto rigging with PBR Material support, pose and motion authoring all in one. 3D Modeling in a blink, try it now!
Metadata

homepage URL: https://dust3d.org/
source URL: https://github.com/huxingyi/dust3d
license: Dust3D's source code is available under the MIT license, however, some third party libraries we use are not MIT license compatible, such as CGAL. You can use the released Dust3D software freely no matter for personal or for commercial purpose, and you can integrate the source code of Dust3D to your project as MIT licensed, but with the third party libraries on the specified license respectively.
platforms: linux, darwin, windows

Very usefull to easily create models that can be exported into Sweet Home 3D...

CGAL shouldn't be an issue for people wanting to use the program, as the parts are either dual licensed GPL+commercial or LGPL+commercial, so the combination of dust3d and cgal can be used under GPL. It's also already packages in nixpkgs.
The weirdly licensed dependency is actually instant-meshes, which appears to be BSD 3-clause with an added paragraph.

I had a quick look at packaging this. Frankly, it's a massive pain, as many of the dependencies are "vendored", for example, instant-meshes doesn't build a static or shared library which can be linked against by dust3d, so dust3d needs the entire instant-meshes source tree in a subdirectory. instant-meshes itself has at least six vendored dependencies of its own, with no option to use system libraries instead, even if they exist.

Yeah, seems complicated. They also provide appimage, is it an option to use this in nixpkgs? I tested it and it works, well, I guess it is missing icons, but I am not sure as I use i3 and I am not used to see much icons anyway :)


Increase code coverage

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@fea5c7c). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master       #3   +/-   ##
=========================================
  Coverage          ?   99.61%
=========================================
  Files             ?        1
  Lines             ?      259
  Branches          ?        0
=========================================
  Hits              ?      258
  Misses            ?        1
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fea5c7c...a756fff. Read the comment docs.


Pull image by default, if not build

Description

Pull the image by default

Today, even if you not specify the option --build, if not existent the image will be built by default. The idea is to pull instead of build

Related Issues:

closes #39

It sounds there is a way to "control the precedence of pull over building", but it is not clear how to do that. Did you try to tweak the yaml file (maybe pull_policy)?

No, you are right, this should be the right place to fix it, I will look into it


Delayed type checking for Type Hinting python 3.7+

added delay type checking syntax for python 3.7 and above for 3.19.13 Conditional Imports

Closed with reference to https://github.com/google/styleguide/pull/490#issuecomment-568363980


PaginatedDataTable2 shows wrong number of total rows on last page

Let' s say, my datasource contains 15 records and 10 of them are displayed once in a while. Then the footer displays "11 - 20 of 15" on the last page, which is quite confusing. I suggest the following change in footerWidgets.addAll:
Text(

        localizations.pageRowsInfoTitle(

          _firstRowIndex + 1,

          math.min(_rowCount, _firstRowIndex + _effectiveRowsPerPage), // take the lower number

          _rowCount,

          _rowCountApproximate,

        ),

      ),

That's how the SDKs PaginatedDataTable works, shows empty rows to fill the page. For consistency that behavior will be preserved.
There was also a PR with one of the parts containing a toggle for that behavior, should it eventually be completed the capability to override that behavior will be added

My issue was not about showing empty rows, but the displayed number within the footer.

Those are related, you get number in the future corresponding to the number of rows displayed, empty rows counted in

OK, but that is kind of weird in my opinion.

It's Flutters or Material design default way of displaying paginated tables...

PaginatedDataTable2.renderEmptyRowsInTheEnd property has recently been added, try setting it to false, should fix your issue

Thank you very much. Meanwhile I reported this issue to the flutter-team itself: https://github.com/flutter/flutter/issues/116569


focus issue

nice start on this window manager.  I like that its written in python.  One issue I have noticed is that when a window steals focus there doesn't seem to be a way to focus the other opened windows.  A way to fix that would be to add the ability to cycle windows (alt-tab) so that you can force focus to other windows.  You could also fix it by forcing focus to under mouse pointer.

Thank you for your involvement.
We will definitely implement some kind of keyboard focus control.
Currently, the focus should be always under the mouse pointer.

The focus follows mouse is working for some programs (alacritty, thunar, etc),  but there are programs that steal focus and won't allow the other windows to be focused (falkon, discord, etc).  I'm not sure how other window managers deal with this but I haven't seen this issue on other tiling window managers such as dwm or qtile.

I looked into the issue a bit more.  Focus follows mouse works properly until certain programs are clicked on causing that program to steal focus and not allowing other open programs to have focus back when the mouse pointer is back on them.   Changing workspaces allows the focus to work properly again until the problem program is clicked on again.  It may be necessary to have a click to focus method along with the focus follows pointer to have it all work properly.

Which programs have you experienced this issue with?

I've noticed it most with Falkon and Discord, falkon is especially bad at stealing focus.

I didn't notice that before.
Thank you very much!


Fix typescript error in i18n

Updated NumberFormatOptionsStyleRegistry to align with latest version of typescript.

/snapit

Tophatted in /web in https://github.com/Shopify/web/pull/138778


examples: fix for example instructions

FIX Updates the app.py installation instructions

I'm closing this PR as the #17 is a further improvement of it.