Codegen imports unused classes

While I was playing around with the generator, I figured out that for all generated classes the same mandatory imports are generated - no matter what:
https://github.com/smallrye/smallrye-mutiny-vertx-bindings/blob/dda664c3fb94bbeeb0e763fd088ff6fb357d1a4b/vertx-mutiny-generator/src/main/java/io/smallrye/mutiny/vertx/codegen/lang/ImportDeclarationCodeWriter.java#L24-L34
Unfortunately, I have an interface called Publisher which I annotated with @VertxGen and it clashes with org.reactivestreams.Publisher - even if it is not used in the generated class.
Any ideas of how to fix this (besides renaming my Interface)?

Another relating issue is:
import redundant classes/interfaces that are declared in origin class but are in package level access => complied failed

Can you give an example of that?

Origin class JooqxBuilder. SQLExecutorBuilder is the internal interface (package level)
import org.jetbrains.annotations.NotNull;
import org.jooq.DSLContext;

import io.github.zero88.jooqx.datatype.DataTypeMapperRegistry;
import io.vertx.codegen.annotations.Fluent;
import io.vertx.codegen.annotations.GenIgnore;
import io.vertx.codegen.annotations.VertxGen;
import io.vertx.core.Vertx;
import io.vertx.sqlclient.Pool;
import io.vertx.sqlclient.Row;
import io.vertx.sqlclient.RowSet;
import io.vertx.sqlclient.Tuple;

/**
 * Represents a builder that constructs {@link Jooqx}
 *
 * @since 2.0.0
 */
@VertxGen
public interface JooqxBuilder extends SQLExecutorBuilder {

Generated class
import java.util.Map;
import java.util.stream.Collectors;
import io.smallrye.mutiny.Multi;
import io.smallrye.mutiny.Uni;
import java.util.function.Consumer;
import org.reactivestreams.Subscriber;
import org.reactivestreams.Publisher;
import io.smallrye.mutiny.vertx.TypeArg;
import io.vertx.codegen.annotations.Fluent;
import io.smallrye.common.annotation.CheckReturnValue;
import io.github.zero88.jooqx.SQLExecutorBuilder;
import io.github.zero88.jooqx.datatype.DataTypeMapperRegistry;
import org.jooq.DSLContext;
import io.github.zero88.jooqx.SQLErrorConverter;

/**
 * Represents a builder that construct {@link io.github.zero88.jooqx.mutiny.Jooqx}
 *
 * <p/>
 * NOTE: This class has been automatically generated from the {@link io.github.zero88.jooqx.JooqxBuilder original} non Mutiny-ified interface using Vert.x codegen.
 */

@io.smallrye.mutiny.vertx.MutinyGen(io.github.zero88.jooqx.JooqxBuilder.class)
public class JooqxBuilder {

This issue does not happen with vertx-rx3 and vertx-rx2

I will fix the Publisher issue, but the visibility issue is more complex. I've to see what the other generators are doing, but yes, for now, it's a Mutiny generator limitation (do not forget that this generator is very different from the RX ones and does a lot more).
Can you open a new issue with to address the visibility issue?


Prepare repository for Threat Bus 2021.12.16

The usual monthly changelog rotation.

Let's go!

Just waiting for the VAST CI to upload the Docker image... 🥱


Myntra Fashion Product Analysis using Image Processing

Deep Learning Simplified Repository (Proposing new issue)
:red_circle: Project Title : Myntra Fashion Product Analysis using Image Processing
:red_circle: Aim : The aim of this project is to analyze the images of the fashion products using image processing methods.
:red_circle: Dataset : https://www.kaggle.com/datasets/djagatiya/myntra-fashion-product-dataset
:red_circle: Approach : Try to use 3-4 algorithms to implement the models and compare all the algorithms to find out the best fitted algorithm for the model by checking the accuracy scores. Also do not forget to do a exploratory data analysis before creating any model.

📍 Follow the Guidelines to Contribute in the Project :

You need to create a separate folder named as the Project Title.
Inside that folder, there will be four main components.

Images - To store the required images.
Dataset - To store the dataset or, information/source about the dataset.
Model - To store the machine learning model you've created using the dataset.
requirements.txt - This file will contain the required packages/libraries to run the project in other machines.

Inside the Model folder, the README.md file must be filled up properly, with proper visualizations and conclusions.

:red_circle::yellow_circle: Points to Note :

The issues will be assigned on a first come first serve basis, 1 Issue == 1 PR.
"Issue Title" and "PR Title should be the same. Include issue number along with it.
Follow Contributing Guidelines & Code of Conduct before start Contributing.

:white_check_mark: To be Mentioned while taking the issue :

Full name :
GitHub Profile Link :
Email ID :
Participant ID (if applicable):
Approach for this Project :
What is your participant role? (Mention the Open Source program)

Happy Contributing 🚀
All the best. Enjoy your open source journey ahead. 😎

Full name : Akshat Mishra
GitHub Profile Link : https://github.com/Adm-2005
Email ID<EMAIL_ADDRESS>Approach for this Project : Keras Sequential Model with 3 additional layers.
What is your participant role? : Swoc 24 contributor

Can you come up with 2 more models which you want to implement?
@Adm-2005

Sure. If I were to make a list the other two models would be ResNet and FashionNet. The reason keras sequential was my first choice was because FashionNet sometimes due to its specific purpose sometimes results in overfitting.

Full name : Titiksha Agrawal
GitHub Profile Link : https://github.com/AgrawalTitiksha
Email ID<EMAIL_ADDRESS>Approach for this Project : After data preprocessing and visualization, using PCA algorithm for normalizing and standardizing the images, then to use LDA , SVM (all 3 kernel) , and CNN (VGG16) for analyzing the images
What is your participant role? (Mention the Open Source program) : @Contributor 2024

Cool, @Adm-2005 use all the three models for this project and make a comparison of the models based on the accuracy scores to find out the best fitted model for this dataset/project.
Issue assigned to you. You can start working on it.

Sure, I'll do that.

Hey , @abhisheks008
Full name : Aaradhya Singh
GitHub Profile Link : https://github.com/kyra-09
Email ID<EMAIL_ADDRESS>Participant ID (if applicable):
Approach for this Project :
1.) I want to use Keras API to implement various pre-trained models and optimise them to make better accuracy for mentioned dataset .
2.) Comparing between used models with various performance metrics such as - f1 score , accuracy , confusion matrix etc.
What is your participant role? (Mention the Open Source program) - Contributor/GSSOC-2024
Kindly assign this issue to me

Hi @kyra-09 one issue at a time please!

As you have previously contributed in the repository, can you find out a dataset with the same problem statement which will be compatible with the deep learning methods?
@CoderOMaster

Sorry my bad it has images, found out after downloading since kaggle didn't show overview.anyways I will use to classify what type.of.clothing accesories it is based on CNN, resnet,vgg etc models

Hey you didn't assign me the issue

Oops! Sorry for that.

@abhisheks008 I am unable to find a conclusive result since it has huge no of labels to predict anything.at the moment I tried using NLP to predict whether product is expensive or not by providing description of that dress/product. Then I am using cv to use images with brands as labels from CSV but brand names are too huge to implement and get good accuracy.very poor results as brands are itself 1000+ to be categorised.Should I publish my results anyways?

@abhisheks008 I am unable to find a conclusive result since it has huge no of labels to predict anything.at the moment I tried using NLP to predict whether product is expensive or not by providing description of that dress/product. Then I am using cv to use images with brands as labels from CSV but brand names are too huge to implement and get good accuracy.very poor results as brands are itself 1000+ to be categorised.Should I publish my results anyways?

I'd like to suggest you something. Can you look for a better dataset than this which suits the existing problem statement.

@abhisheks008 I found something after extensive research.I hope this will be suffice although this dataset is also similar to that so used single column for classification that too which had over 36 classes


Regression: TODOs treated as errors by dartanalyzer as of 1.16.0-dev.1.0

Given:
// TODO(ochafik): hello.
main() {}

Then dartanalyzer --strong ~/analyzer.dart yields:
Analyzing [/Users/ochafik/analyzer.dart]...
[error] TODO(ochafik): hello. (/Users/ochafik/analyzer.dart, line 1, col 4)
1 error found.

dartanalyzer --version
dartanalyzer version 1.16.0-dev.1.0

Works fine in 1.16.0-dev.0.0 (emitted as hint instead of error)

To clarify, does this regression only happen with strong-mode enabled?

@jwren I have strong mode enabled – yes

@jwren ...and disabling strong mode eliminates the squiggles.

@munificent Is this related to your error message changes?

Gah, it must be. I'll take a look.

Sorry about that. Accidentally made strong mode REALLY STRONG mode. :metal: :angry: :metal:

AUTHORITARIAN mode FTW! (todo -> thou shalt do it now... would solve so many issues!)
Thanks for the quick fix!

With 1.16.0-dev.2.0 Unsound implicit cast (strong_mode_down_cast_composite) reports as an error. Was warning with 1.15.0.

@munificent

I'll take a look.

@munificent different bug, no? The TODO issue has not regressed, right?

If it is an issue, it's probably still caused by the same severity promotion code.

https://codereview.chromium.org/1833553002/


Make the round tripper use the dialer's context

this is useful in getting https://pkg.go.dev/net/http/httptrace events related to DNS when using the round tripper to make a request. Other events related to the connection continue not to work.

this addresses the DNS part of https://github.com/quic-go/quic-go/issues/3342

@deweerdt Are you still planning to work on this PR?

Closing due to inactivity. @deweerdt Feel free to reopen once you have the bandwidth to push this through.


Fix logout bug

Please check if the PR fulfills these requirements:

[x] The PR title and message describe the solution briefly.
[ ] Documentation has been updated/added if relevant
[x] I am not raising a spam PR or a PR that fixes spellings or whitespaces for Hacktoberfest
[x] I know that if this PR is found to be spam, It will be labeled and reported immediately

What is the current behavior?

On clicking logout button, 'dispatch' not defined error used to occur.
What is the new behavior?
Fixed the logout dispatch not defined issue. Now it correctly logs out the user.
Other information?

@anshulporwal26 this is again a duplicate and already being fixed

Fixed in #79


Make "fixlengths" command ignore trailing empty fields in automatic m…

…ode.

This was inspired by a public dataset (warning: big) https://s3.amazonaws.com/nyc-tlc/trip+data/yellow_tripdata_2016-07.csv
It is apparently a sloppy expert in the form of:
foo,bar,baz
a1,b1,c1,,
a2,b2,c2,,

I squashed and merged this in https://github.com/BurntSushi/xsv/commit/64b49169f8d4368ad07062a80ee401be8a65004f


WIP: add KHI setup from eth zurich

This is the setup used by Fjordholm et al. to test convergence. Example uses a KHI setup with a sharp interface, that gets randomly perturbed. For the random pertubation, StableRNGs package is used to enable CI testing.

Some tests fail because of upstream issues - should be resolved by #1170.

Ok, thanks - I was scratching my head, because the tests locally worked out...

This is the setup used by Fjordholm et al. to test convergence

Out of curiosity: in which sense is the setup used to test convergence?

This is the setup used by Fjordholm et al. to test convergence

Out of curiosity: in which sense is the setup used to test convergence?

The idea is the following: you make many many realisations (they use 400 in the paper) and then average the solution.
Next, you refine the grid and do the same procedure again (random perturbations, 400 realisations, compute the average).
Then this average of the solution seems to converge - although the setup is extremely chaotic.
However, the average basicallx smears out everything in this example - here I computed 100 realisations and the average. I am not 100% sure, how useful this convergence is for applications.

@gregorgassner Thank you for the explanation. From the "averaged" picture above, it seems to be clear that 100 realizations is not enough,since - at the very least - you would expect symmetry between the top and the bottom half, wouldn't you?
But it's an interesting approach!

I am not 100% sure, how useful this convergence is for applications.

IIRC, the convergence is from a quite theoretical angle using some kind of measure valued solution from a particular space. i think these measure valued solutions were an (somewhat hopeful) avenue to prove things like well-posedness for nonlinear problems.


ToilBox: Make Toil version configurable

via command line option

This was done in https://github.com/BD2KGenomics/cgcloud/commit/5e79540e99cedb06f4f3329abc74590e150ceb99


Arm64 arch in website dev environment

This change was made on the build-hugo workflow in #1583, but we will need to make the change here as well

When this build passes, we'll have multi-arch Hugo images:
https://github.com/fluxcd/website/actions/runs/5681642492
The hugo-support image addressed by this PR, uses Dockerfile in the local repository, and represents all of our own requirements atop of Hugo itself. I've tested all of these builds locally, the only thing that remains here is I think to update the remaining GHA workflow, and run the image builds by hand.
A separate PR might still add the capability to automatically trigger the necessary hugo and hugo-support builds on change to any of the files that can necessitate a rebuild. I haven't worked all that out yet, we've only needed these images a very few times (as most of the needs for testing are already being met by the Netlify preview feature!)

I tested the consolidated workflow here:
https://github.com/kingdonb/website/commit/dbd7c1aff076e754e491099a291115bc9cf5a869
It's not a full test, but it verifies that the new path-based trigger works, which has been the only part I cannot test locally.

Thanks! The build for the base image succeeded
https://hub.docker.com/r/fluxcd/website/tags
^ shows we have multiarch images for the hugo base now, will have them for hugo-support next 🚀


No958

Já vem a onde ??
Vai vim ou não ??

It seems you are working on a GitHub issue in the choosealicense.com repository. Below are the details you provided:

Issue: #1225
Branch: patch-9
Base: gh-pages

Here's a step-by-step guide to address the issue and create a Pull Request (PR) from patch-9 to gh-pages.
Step-by-Step Guide
1. Clone the Repository
If you have not already done so, clone the repository locally:
git clone https://github.com/github/choosealicense.com.git
cd choosealicense.com

2. Checkout the Branch
Check out the patch-9 branch:
git checkout patch-9

3. Identify and Fix the Issue
Review the contents of issue #1225. The issue should contain a description of the problem and any relevant details. Based on this information, you will need to:

Identify the part of the project that has the problem.
Make appropriate fixes or enhancements.

Example Change
Let's assume the issue is related to updating licensing information in a particular file. Open the file and update the content accordingly.
Example: If the file is _licenses/mit.txt and needs an update:
# Open the file
vim _licenses/mit.txt

# Make necessary updates to the content

4. Add and Test Changes
Save your changes and ensure everything works as expected:

Add Changes: Stage the changes for commit.
Run Tests: If the repository has tests, run them to ensure nothing is broken.

# Add changes to the staging area
git add _licenses/mit.txt

# Run tests (the exact command might differ)
# Typically, for a Jekyll site, you would run:
bundle exec jekyll serve
# Then visit http://localhost:4000 to test locally

5. Commit Your Changes
Once you are confident the changes are correct, commit them:
git commit -m "Fix issue #1225: Update licensing information"

6. Push Changes
Push the changes to GitHub:
git push origin patch-9

7. Create a Pull Request
Now, create a pull request (PR) on GitHub from the patch-9 branch to the gh-pages branch.

Navigate to the Repository: Go to the repository on GitHub.
New Pull Request: Click on New Pull Request.
Base and Compare: Ensure the base branch is gh-pages and the compare branch is patch-9.
Fill PR Details: Provide a title and description for the PR, including details of the fix you’ve made.

Example PR
Title: Fix issue #1225: Update licensing information
Description:
This pull request resolves issue #1225 by updating the licensing information in _licenses/mit.txt. The new information reflects the latest standards and resolves discrepancies mentioned in the issue.
Review and Merge
After creating the PR, it will need to be reviewed and approved by other collaborators. Address any feedback or required changes. Once approved:

Merge PR: Click the Merge pull request button.
Delete Branch: Optionally, delete the branch after merging.

Alternatively, you can merge and clean up using the command line:
# Ensure you are on the right branch
git checkout gh-pages

# Pull the latest changes
git pull origin gh-pages

# Merge the patch-9 branch
git merge patch-9

# Push the changes back to GitHub
git push origin gh-pages

# Delete the patch-9 branch locally and remotely
git branch -d patch-9
git push origin --delete patch-9

Following these steps ensures that you have fixed issue #1225 in the choosealicense.com repository and created a successful pull request from branch patch-9 to gh-pages. If further assistance is needed, feel free to ask!


completion on openfl project is broken

Compiler based completion defines display,
and some type are hidden from it which cause "type not found" errors
Like https://github.com/openfl/openfl/blob/develop/openfl/_internal/renderer/RenderSession.hx
And some type are different in display mode like https://github.com/openfl/openfl/blob/develop/openfl/utils/ByteArray.hx
Last but not least the flash extern, I don't even get it why are they mapped to openfl classes on display?!
https://github.com/openfl/openfl/blob/develop/externs/core/flash/flash/Memory.hx
Besides some very specific cases, mostly involving macro, #if display shouldn't be used.

On the subject of externs what are https://github.com/openfl/openfl/tree/develop/externs/core/openfl and https://github.com/openfl/openfl/tree/develop/externs/extras/ used for @jgranick?
According to the include.xml they are pulled in when in display mode (and the first one on flash build too)

The externs (and -Ddisplay) are used for multiple things.
It is used for code completion, used for documentation, and used for (in the future) compiling lightweight applications (which would rely on an openfl.js or lime.js, for example)
The openfl._internal package not should be exposed to display. All platforms should use the same files for code completion -- the public API for OpenFL is intended to be unified, it prevents frustrating "cannot merge" errors when generating documentation, and gives us a single place to add ASDoc documentation.
Are we missing an #if !display import somewhere?

But it should, I'm not talking about whether to add @:noCompletion or not, but an actual error from the compiler because the type isn't defined while in display mode.
Completion shouldn't use some extern definition, it gives wrong information on type hover, go to definition and things like that.
Totally get why it's done for documentation, but then let's use a documentation flag ant abuse the display flag.
Also for openfl.js why not just use --macro exclude("openfl") it'll make the type extern and assume it'll be available at runtime.

On the asdoc issue, is it really a problem?
Is there more than the openfl package/are there classes duplicated?
Assuming we don't use the openfl externs.
Do doc merge issues come from flash vs non flash?
Also I'd really like to know why the flash externs point to openfl classes on display, display shouldn't be called on them, there's a remap flash -> openfl right?

I do not want the internals of OpenFL's implementation to be exposed to code completion -- otherwise, developers may also end up making unnecessary assumptions about our implementation, leading to upgrade problems later (when the public API still works the same)
RenderSession is not a public API, being hidden makes development of OpenFL itself a little more cumbersome, but I do not want this ever to be exposed to other users.
There is no Flash -> OpenFL remap, this would break support for Flash-only classes that are not implemented in OpenFL. We do customize certain types (such as enum values) and also the openfl.* classes are the ones with ASDoc comments -- we definitely don't want to duplicate those

Alright then when I remove the display guard I'll add some @:noCompletion, but being in **_**internal that should already be enough.
Oh so that's why the extern redirect, for the documentation, probably ok I guess.
I think I got it, but I'll still make the changes in a separate branch ;)

You cannot add @:noCompletion to a class, only fields. This format also means we don't have to clutter our internal code with a @:noCompletion on everything we define

It also means that completion doesn't work,
the compiler isn't ok with types not being found.

What is asking for RenderSession? Without referencing our internal implementation, there shouldn't be code referencing it. That's why I wondered, though (say, in the special case of Tilemap on the Flash target) if we were missing an #if !display around an include

Also, is there an easy way to test if code completion is working properly? Is there a way we could extend our Travis unit tests to be sure that completion works?

No idea, @Gama11 is the one with the bug, but it's not on latest so can't be Tilemap.
But all that seems like a hack, there should be a better way to hide things from completion,
maybe haxe could support @:noCompletion on class? (cc @Simn)
Although I'm pretty sure classes in packages starting with an underscore are already hidden.

We could support @:noCompletion on classes, but then it is only going to work on latest Haxe version so that's probably not a satisfactory solution.
The whole "hide the internal implementation in display mode" business seems to be a bad idea. Where do these classes actually show up?

For the record, this is the error I was getting:
 C:/HaxeToolkit/haxe/lib/openfl/3,6,1/openfl/_internal/renderer/opengl/utils/SpriteBatch.hx:36: characters 19-32 : Type not found : RenderSession

Internal fields can be defined as @:allow(openfl) private. It would be even stricter than @:noCompletion, which doesn't really prevent people from using the internal fields.
Ideally one can define the internal class itself as private, but @:allow doesn't expose private types (https://github.com/HaxeFoundation/haxe/issues/5731). However, if all fields are private, people can't use the class anyway.

@:allow(openfl) is a bad idea - I made that mistake in Flixel. It means that all subclasses of classes in the openfl package have access as well.

Good point. That's tricky since people are expect to extends Sprite...

The issue also highlight the need for really private fields/functions,
really useful in any code where the user is expected to subclass.

I also wish that is real private fields. But, hey, ruby and python don't have real private fields either...

It may seem unconventional, but it is important to me that the underlying implementation for OpenFL is hidden. I believe this results in better code completion, and reduces the cases where updates to OpenFL break older projects.
We can discuss this more on Slack or the forums, but this is a desired behavior for now :smile:


Double-clicking a view splitter should equally resize the 2 adjacent views

When double-clicking here, I would like the search view and results view resized to take up the space equally

Additionally, I would love it if double-clicking on a view header would "maximize" that view -- either collapse all other views or just shrink all other views so that the double-clicked view takes up a must space as it can. Honestly it would be nice to have both of those behaviors with a modifier key to switch between them, because sometimes I just want to expand the current view (probably should be the default) and other times I want to collapse everything to focus on 1 view (with modifier)

What if there are 3 views? Would you expect both adjacent views to get the same size, or all 3 views to have the same size?

Additionally, I would love it if double-clicking on a view header would "maximize" that view -- either collapse all other views or just shrink all other views so that the double-clicked view takes up a must space as it can. Honestly it would be nice to have both of those behaviors with a modifier key to switch between them, because sometimes I just want to expand the current view (probably should be the default) and other times I want to collapse everything to focus on 1 view (with modifier)

Why not a maximize action icon to the right of each header? cc @misolori
@sbatten I would do this at the view pane container layer. The splitview exposes the onDidSashReset such that clients can do what they want.

I'd just want it to it to equally size the 2 adjacent views (splitting the space they both are currently using). But I know that for editors we resize them all (which I honestly don't like).


externalresources(engine): support s3 as storage (Part 2)

What problem does this PR solve?

Issue Number: ref #7114
What is changed and how it works?

broker

Create localFileManager and s3FileManager in NewBroker
If S3 is configured, a dummy resource will be persisted by broker to indicate that its temporary files have not been cleaned, which is useful to prevent resource leaks.
Asynchronously remove temporary files created by the worker in OnWorkerClosed
If an Executor exits normally, the corresponding broker will attempt to clean up all temporary files before exiting. If this step fails, the dummy record is stored in Meta, which will be cleaned up by the GCCoordinator.

manager

DefaultGCCoordinator: SetGCPending when job offline. GCExecutors when executor offline.
DefaultGCRunner: implement GCExecutors

Check List
Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)

Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note
`None`.

/run-engine-integration-test

Broker integration tests verified in https://ci2.pingcap.net/blue/organizations/jenkins/engine_ghpr_integration_test/detail/engine_ghpr_integration_test/902/pipeline/86

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/cc @amyangfei
/cc @gozssky

/run-all-tests

Codecov Report

Merging #7204 (14e8a1b) into master (d8fff7a) will increase coverage by 0.0368%.
The diff coverage is 57.1895%.

Additional details and impacted files

Flag
Coverage Δ

dm
51.8878% <ø> (+0.0503%)
:arrow_up:

engine
60.4603% <55.2662%> (-0.3979%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.
@@               Coverage Diff                @@
##             master      #7204        +/-   ##
================================================
+ Coverage   59.6043%   59.6411%   +0.0368%
================================================
  Files           790        796         +6
  Lines         91089      91571       +482
================================================
+ Hits          54293      54614       +321
- Misses        32051      32167       +116
- Partials       4745       4790        +45

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/cc @amyangfei
/cc @gozssky

/run-all-tests

/run-all-tests

/cc @gozssky

/run-all-tests

/run-all-tests

/merge


JSON decode: keep value types, do not convert to string

To test this, we have to change in the tests (utils, transform_*_test.go the appropriate fields to integers (instead of strings) and verify that everything still works.

@KalmanMeth I think that test like https://github.com/netobserv/flowlogs2metrics/pull/85/files#diff-06c34b5e9ba491d8ba67f577931652caacddf9fa65c3a177a2d5bafb0ecae68cR50 is showing exactly that

There was some failure occurring when we previously used integers rather than strings, and it had to do with converting json to a map and back. Try changing the tests in transform to use integers for port numbers, bytes, etc, and see if they pass.

@KalmanMeth I changed the tests to use numeric (int) values for ports and protocol (as a number) and the code is working as expected == parsing the ports correctly and the protocol.
I looked at the code and this looks like this:: portNumber, err := strconv.Atoi(fmt.Sprintf("%v", outputEntries[rule.Input]))  --- this makes things very generic. We do not care if this is a string or a number the result will always be the port number as int.


Support python 3.7

Description
Update python version from 3.6 to 3.7
Described in issue #2236
Rationale
Fixes #2236
Use case(s) / visualization(s)

Nice work! Thank you for this. I'll take a closer look later this week, but it looks OK at first glance 🎉

Nice!


add concurrency to the workflow

This PR adds concurrency to the smoke test, ensuring that only the most recent commit of a pull request is tested and prevent job execution on outdated commits.

Done @jayanth-kumar-morem


New-AzDisk Error: Required parameter 'storageAccountId' is missing

Used PS commands mentioned at and ran into error mentioned at https://github.com/Azure/azure-powershell/issues/12719#:~:text=Description New-Azdisk command fails requiring a storageaccountId parameter,fails New-AzDisk%3A Required parameter 'storageAccountId' is missing (null).
Had to make sure to pass the storage account ID in order to get the New-AzDisk command working.
It would be great to add that parameter too in commands mentioned.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 940f5217-9f40-0563-27d2-d9e7f7e42005
Version Independent ID: e088e366-77aa-6358-2540-2c937c6474ec
Content: Azure support for generation 2 VMs - Azure Virtual Machines
Content Source: articles/virtual-machines/generation-2.md
Service: virtual-machines
Sub-service: sizes
GitHub Login: @ju-shim
Microsoft Alias: jushiman

@smoop-msft
Thanks for your feedback! We will investigate and update as appropriate.

Thanks for your dedication to our documentation. Unfortunately, at this time we have been unable to review your issue in a timely manner and we sincerely apologize for the delayed response. The requested updates have not been made since the creation of this issue, and the timeline for resolution may vary based on resourcing, so we've created an internal work item to incorporate your suggestions. We are closing this issue for now, but feel free to comment here as necessary. #please-close


Support a callback function to detect text changes

Is your feature request related to a problem? Please describe.
It would be useful to detect changes in the text content of the box.
Describe the solution you'd like
I propose adding an onChangeText prop that would receive the text contents of the control when those contents change
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
It would be accessed like this:
  <DragTextEditor
        ...
        onChangeText={(text: string) => console.log(text)}
      />

It would require the following code to be added to DragTextEditor.js:
componentDidUpdate(prevProps, prevState){
    if(this.state.text !== prevState.text){
      if(this.props.onChangeText){
        this.props.onChangeText(this.state.text);
      }
    }
  }

Additional context
I can submit a PR if this works for you @eneskarpuz

Thanks @reggie3. I agree, it would be useful. I'm waiting for your PR to merge but you'd better check these before you submit a PR;
1- I'm not sure about using type annotations like (text: string) .
onChangeText={(text) => console.log(text)}

isn't it better ?
2- You can add your Callback Function to onText function on DragTextEditor.js like;
onText=(text)=>{
  this.setState({text});
     if(this.state.text && this.props.onChangeText ){
        this.props.onChangeText(this.state.text);
   }
}

Sorry about the string annotation, I copied and pasted the code sample from a typescript file.
It makes sense to use the functionality you already have to implement the callback.  I'll submit a PR based on what you said above.

@eneskarpuz How about this:
onText = (text) => {
    this.setState({ text }, () => {
      if (text && this.props.onChangeText) {
        onChangeText(this.state.text);
      }
    });
  };

This way, the data sent back from the callback is in sync with what is in the component state.

Thanks @reggie3! Looks like a good idea. I think it'd be better to talk this on a PR.

Closing since this feature has been merged.


Can't Not Work in linux wtih vim 8.1

Unknown function: popup#add
Undefined variable: g:popup
Invalid expresssion: g:popup.ftmenus
Sad..

Please see the last document, APIs is changed.

@luzhlon  I don't know which vim you are using.  I found some code in popup.vim :
let g:popup.menus = {}
let g:popup.ftmenus = {}
let g:popup.last = ['', '']

I don't think there is struct feature with vim variable. Here is error message.
Undefined variable: g:popup

Yeah, g:popup is inited in plugin/popup.vim in older version.
Now, I moved it to autoload/popup.vim.

It works, thank you for your working!


Better User Experience for agent delegation

As @xingyaoww pointed out in https://github.com/OpenDevin/OpenDevin/pull/2103#pullrequestreview-2088328179, agent delegation doesn't give perfect user experience:

when it finishes, the frontend will repeat the message (one from the Browsing Agent, one from the delegator), It even got a little empty box in the middle:

I assume the little empty box would be easier to solve, but how not to make user feel like the agent is repeating itself, is a tough task.

Thought:
In the frontend, we should figure out a way to mark the "source" of messages from which agent (e.g., the parent CodeActAgent, or the child browsing agent).
We could display all messages from all agents -- as long as on the top of each bubble we should the source of that conversation.
Then, when a child agent finishes execution (i.e., in Boxuan's example, the BrowsingAgent returns results to CodeActAgent), we can collapse the messages from the child agent into a "Collapsible" (e.g., https://getbootstrap.com/docs/4.3/components/collapse/) - the user can click to expand the messages from Browsing Agent. By default, it collapses and only displays the last messages returned by that child agent.

Collapse sounds like a great idea!


[Web][CanvasKit]: Locale text with punctuation such as full stop, comma position is misaligned.

Code as below

The same phenomenon occurs in Japanese.
This is a pretty big problem.
For example
"こんにちは、Flutter。"
Then
"、" And "。" move to the middle of the line.

@shenruisi
Can you provide flutter doctor -v and provide the code in formatted text instead of screenshot, so that we can directly use it to verify this issue ?
Thanks.

Hi @shenruisi and @darshankawar.
Perhaps I have a similar problem.
So I will also submit my report as one of the sample cases.
Notably, recent Mac OS updates aren't the cause.
Also, there is a beta version of Flutter 1.x that does not contain this bug.
flutter doctor -v
[✓] Flutter (Channel stable, 2.0.2, on macOS 11.3 20E232 darwin-x64, locale ja-JP)
• Flutter version 2.0.2 at /Users/morimasahide/documents/flutter
• Framework revision 8962f6dc68 (7 weeks ago), 2021-03-11 13:22:20 -0800
• Engine revision 5d8bf811b3
• Dart version 2.12.1
[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
• Android SDK at /Users/morimasahide/Library/Android/sdk
• Platform android-29, build-tools 29.0.3
• Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS
• Xcode at /Applications/Xcode.app/Contents/Developer
• Xcode 12.5, Build version 12E262
• CocoaPods version 1.10.1
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 4.1)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
[✓] IntelliJ IDEA Ultimate Edition (version 2021.1)
• IntelliJ at /Applications/IntelliJ IDEA.app
• Flutter plugin version 55.1.5
• Dart plugin version 211.6693.108
[✓] Connected device (1 available)
• Chrome (web) • chrome • web-javascript • Google Chrome 90.0.4430.85
• No issues found!

Run the following code to find the bug.
Thank you.
class _MyHomePageState extends State<MyHomePage> {

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
          child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          crossAxisAlignment: CrossAxisAlignment.center,
          children: <Widget>[
            Text(
              'こんにちは、Flutter。（Bug）',
              style: TextStyle(
                fontSize: 24,
              ),
            ),
            Text(
              'Hello, Flutter. (Correct display)',
              style: TextStyle(
                fontSize: 24,
              ),
            ),
          ],
        )
      ),
    );
  }
}

Postscript: The result is the same even if flutter upgrade.
I feel like there is a bug in Flutter 2.x.
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 2.0.5, on macOS 11.3 20E232 darwin-x64, locale ja-JP)
[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
[✓] Xcode - develop for iOS and macOS
[✓] Chrome - develop for the web
[✓] Android Studio (version 4.1)
[✓] IntelliJ IDEA Ultimate Edition (version 2021.1)
[✓] Connected device (1 available)

@MasahideMori-SimpleAppli
Thanks for the code sample and detailed report. I tried it using latest master and believe is an issue with canvaskit rendering, because using html rendering, we see the full stop at correct position as below:

with html rendering : flutter run -d chrome --web-renderer html:

With canvaskit rendering: flutter run -d chrome:

flutter doctor -v
[✓] Flutter (Channel master, 2.3.0-1.0.pre.143, on Mac OS X 10.15.4 19E2269
    darwin-x64, locale en-GB)
    • Flutter version 2.3.0-1.0.pre.143 at
      /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision aecd5e0302 (6 hours ago), 2021-04-28 00:53:57 -0300
    • Engine revision 92ef7b59c8
    • Dart version 2.14.0 (build 2.14.0-48.0.dev)

[!] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    ! CocoaPods 1.9.3 out of date (1.10.0 is recommended).
        CocoaPods is used to retrieve the iOS and macOS platform side's plugin
        code that responds to your plugin usage on the Dart side.
        Without CocoaPods, plugins will not work on iOS or macOS.
        For more info, see https://flutter.dev/platform-plugins
      To upgrade see
      https://guides.cocoapods.org/using/getting-started.html#installation for
      instructions.

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.55.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-x64     • Mac OS X 10.15.4 19E2269
      darwin-x64
    • Chrome (web)    • chrome • web-javascript • Google Chrome 90.0.4430.85

! Doctor found issues in 1 category.

Dear @darshankawar
Thank you for letting me know.
It helped me a lot!

@darshankawar
Sorry for reply late. If i use html render, the punctuation seems in the right position. And waiting for fixed with CanvasKit.

@MasahideMori-SimpleAppli Thanks for the code sample and detailed report. I tried it using latest master and believe is an issue with canvaskit rendering, because using html rendering, we see the full stop at correct position as below:

with html rendering : flutter run -d chrome --web-renderer html:

With canvaskit rendering: flutter run -d chrome:

flutter doctor -v

no, this is not the solution, because if i change to html render , the text style and color style will be change, please fix this problem quicklly

For Japanese texts, I was able to solve this problem.
It could be a problem with the default font.
When using the Noto Sans Japanese font, it could be displayed without any problem.
However, this solution requires font downloads, which is costly.

We're looking into this. Need to chat w/ our friends working on Skia (CanvasKit)

The font fallback rules in the engine are prioritizing a font that renders these characters at a different position (see https://github.com/flutter/engine/blob/main/lib/web_ui/lib/src/engine/canvaskit/font_fallbacks.dart)
I added logging to _getEffectiveFontFamilies and tried rendering this text.  The resulting family list was:
[Roboto, Ubuntu, Cantarell, DejaVu Sans, Liberation Sans, Arial, Roboto,
Noto Sans TC 0, Noto Sans TC 1, Noto Sans TC 2, Noto Sans SC 0, Noto Sans SC 1, Noto Sans
JP 0, Noto Sans JP 1, Noto Sans HK 0, Noto Sans HK 1, Noto Sans HK 2]

with the additional downloaded fallback fonts added in reverse alphabetical order.
If I patch _getEffectiveFontFamilies to move the Noto Sans JP fonts to the head of the list, then the characters render at the desired position.

I'm unsure of whether this is directly relevant to the above comment or how downloaded fallback fonts work... However,
because of the number of languages and characters we need to support, it is not practical for us to package the fonts into our app. Relying on user system fonts is best. Noto Sans JP is 4.5MB for a single font style alone.
Unsure of what downloaded fallback fonts do to the application loading times.

@mdrideout Be aware that in CanvasKit mode we have to download fonts even if you do not supply your own fonts. The fonts we download are sliced into many small files, so we don't fetch 4.5MB all at once, but instead we load them dynamically on-demand. The app loading time is therefore not affected, but some previously unseen characters may not render correctly or render as boxes while waiting for the font to arrive. We're looking to improve this in the future by better predicting the fonts the user may need (e.g. based on the navigator.language setting), upcoming browser APIs, and potentially adding API for developers to provide explicit signals to the Flutter engine.

Hi everyone
@kevmoo, is Skia team working on it?

An engineer on our team is looking at it. Just not sure which one, @EGabriel12

I believe this issue is fixed now:


io:filewriteCSV produces incorrect order when structurally equivalent different order record types

Description:
Consider the below example,
import ballerina/io;

type Base record {|
    int base1;
    int base2;
|};

type Derived1 record {|
    int derived1;
    int derived2;
    int base1;
    int base2;
|};

type Derived2 record {|
    *Base;
    int derived1;
    int derived2;
|};

public function main() returns error? {
    Derived1 d1 = {base1: 1, base2: 2, derived1: 3, derived2: 4};
    Derived2 d2 = {derived1: 3, derived2: 4, base1: 1, base2: 2};
    Derived2 d3 = {base1: 1,  derived1: 3, derived2: 4, base2: 2};
    record {|
        int base1;
        int derived1;
        int derived2;
        int base2;
    |} d4 = {base1: 1, derived1: 3, derived2: 4, base2: 2};
    Derived1[] arr = [d1, d2, d3, d4];

    check io:fileWriteCsv("derived1.csv", arr);
}

The order of the CSV output file is not correct. we need to consider filenames/keys of the record/map to preserve the correct order.

This is caused as we cannot guarantee the order of fields inside a record unlike string[]. This bug can be solved by maintaining the same order of fields in the first element while writing the other elements  of the array. As the field order can be changed from the input, the headers has to be written by default when the record[] is used as the input.

For the case of appending to a existing CSV file, we can check if there is a existing file and read the headers. If the headers match, it will append the new records according to the existing order. If there is no such a CSV file, it will create a new csv file to append the data.

In case of overwrite, I dont think there is a need to check for headers,

Overwrite:
Do not consider existing records or not. Headers based on the field names are added by default.

2 Append.
File does not exist or does not consist of any rows, it is similar to an overwrite scenario.
If records exist, the first row is considered as headers and processed to see if the field name matches.
If it does not match, returns error
If it matches, follow the order to append records.
I think this covers all use cases. @daneshk WDYT?

For the fileWriteCsvFromStream() we will use the same behavior as above to write the CSV stream.

Overwrite:
Do not consider existing data records or not. Headers based on the field names are added by default.

2.Append
File does not exist or does not consist of any rows, it is similar to an overwrite scenario.
If records exist, the first row is considered as headers and processed to see if the field name matches.
If it does not match, returns error
If it matches, follow the order to append records.


Low reliability of VT replies

Windows Terminal version
Latest source
Windows build number
10.0.19045.4780
Other Software
No
Steps to reproduce
I want to query some terminal state, in particular the latest and greatest palette (#17729).
To do so, I build a bunch of those "\x1b]4;N;?\x1b\\", send them in one go and wait for the reply.
Since I want to support not only the latest nightlies and do not want the older versions to deadlock on unknown queries, I prepend and append another query, the one that even prehistoric versions understand: "\x1b[c".
If there is something between those DA replies, it must be the one I need.
If there are only two DA replies, then it is probably not supported.
This approach works flawlessly in OpenConsole.
In WT, however, not so much: way, way too often random parts of the reply are simply missing.

Compile the code:

Code
#ifndef _CRT_SECURE_NO_WARNINGS
#define _CRT_SECURE_NO_WARNINGS
#endif

#ifndef _UNICODE
#define _UNICODE
#endif

#ifndef UNICODE
#define UNICODE
#endif

#include <algorithm>
#include <format>
#include <exception>
#include <iostream>
#include <optional>
#include <ranges>
#include <string>
#include <string_view>

#include <string.h>

#include <windows.h>

using namespace std::literals;

auto win32_error()
{
	return std::runtime_error(std::format("Error 0x{:08X}, look it up"sv, GetLastError()));
};

auto invalid_response()
{
	return std::runtime_error("Invalid response");
}

auto invalid_response(const size_t Index)
{
	return std::runtime_error(std::format("Invalid response at #{}"sv, Index));
}

auto printable(const std::wstring_view Str)
{
	std::wstring Printable;
	std::ranges::transform(Str, std::back_inserter(Printable), [](wchar_t Char){ return Char < L' ' ? Char + L'\u2400' : Char; });
	return Printable;
}

void write(const std::wstring_view Str)
{
	DWORD Written;
	if (!WriteConsole(GetStdHandle(STD_OUTPUT_HANDLE), Str.data(), static_cast<DWORD>(Str.size()), &Written, {}))
		throw win32_error();
}

void write(const std::string_view Str)
{
	write(std::wstring(Str.begin(), Str.end()));
}

std::wstring query(const std::wstring& Command)
{
	const auto Dummy = L"\x1b[c"s;

	write(Dummy + Command + Dummy);

	// Sleep(1);

	std::wstring Response;

	std::optional<size_t>
		FirstTokenPrefixPos,
		FirstTokenSuffixPos,
		SecondTokenPrefixPos,
		SecondTokenSuffixPos;

	const auto
		TokenPrefix = L"\x1b[?"sv,
		TokenSuffix = L"c"sv;

	while (!SecondTokenSuffixPos)
	{
		wchar_t ResponseBuffer[8192];
		DWORD ResponseSize;

		if (!ReadConsole(GetStdHandle(STD_INPUT_HANDLE), ResponseBuffer, ARRAYSIZE(ResponseBuffer),  &ResponseSize, {}))
			throw win32_error();

		Response.append(ResponseBuffer, ResponseSize);

		if (!FirstTokenPrefixPos)
			if (const auto Pos = Response.find(TokenPrefix); Pos != Response.npos)
				FirstTokenPrefixPos = Pos;

		if (FirstTokenPrefixPos && !FirstTokenSuffixPos)
			if (const auto Pos = Response.find(TokenSuffix, *FirstTokenPrefixPos + TokenPrefix.size()); Pos != Response.npos)
				FirstTokenSuffixPos = Pos;

		if (FirstTokenSuffixPos && !SecondTokenPrefixPos)
			if (const auto Pos = Response.find(TokenPrefix, *FirstTokenSuffixPos + TokenSuffix.size()); Pos != Response.npos)
				SecondTokenPrefixPos = Pos;

		if (SecondTokenPrefixPos && !SecondTokenSuffixPos)
			if (const auto Pos = Response.find(TokenSuffix, *SecondTokenPrefixPos + TokenPrefix.size()); Pos != Response.npos)
				SecondTokenSuffixPos = Pos;
	}

	Response.resize(*SecondTokenPrefixPos);
	Response.erase(0, *FirstTokenSuffixPos + TokenSuffix.size());

	if (Response.empty())
		throw std::runtime_error("Query is not supported"s);

	return Response;
}

void set_modes()
{
	const auto
		In = GetStdHandle(STD_INPUT_HANDLE),
		Out = GetStdHandle(STD_OUTPUT_HANDLE);

	DWORD InMode;
	if (!GetConsoleMode(In, &InMode))
		throw win32_error();

	if (!SetConsoleMode(In, (InMode & ~(ENABLE_LINE_INPUT | ENABLE_ECHO_INPUT | ENABLE_QUICK_EDIT_MODE)) | ENABLE_MOUSE_INPUT | ENABLE_EXTENDED_FLAGS | ENABLE_VIRTUAL_TERMINAL_INPUT))
		throw win32_error();

	DWORD OutMode;
	if (!GetConsoleMode(Out, &OutMode))
		throw win32_error();

	if (!SetConsoleMode(Out, OutMode | ENABLE_PROCESSED_OUTPUT | ENABLE_VIRTUAL_TERMINAL_PROCESSING))
		throw win32_error();
}

int main()
{
	try
	{
		set_modes();

		std::wstring Str;

		for (size_t i = 0; i != 256; ++i)
			std::format_to(std::back_inserter(Str), L"\x1b]4;{};?\x1b\\"sv, i);

		for (;;)
		{
			const auto ReplyData = query(Str);

			std::wstring_view Reply(ReplyData);

			if (!Reply.ends_with(L"\\"sv))
				throw invalid_response();

			Reply.remove_suffix(1);

			size_t Index = 0;

			for (const auto& Part: std::views::split(Reply, L"\\"sv))
			{
				std::wstring_view ColorStr(Part.begin(), Part.end());

				try
				{
					if (!ColorStr.starts_with(L"\x1b]4;"sv))
						throw invalid_response(Index);

					ColorStr.remove_prefix(L"\x1b]4;"sv.size());

					wchar_t* Ptr;
					const auto ReportedIndex = std::wcstol(ColorStr.data(), &Ptr, 10);

					if (Ptr == ColorStr.data())
						throw invalid_response(Index);

					if (ReportedIndex != Index)
						throw invalid_response(Index);

					ColorStr.remove_prefix(Ptr - ColorStr.data());

					const auto End = ColorStr.find(L"\x1b"sv);
					if (End == ColorStr.npos)
						throw invalid_response(Index);

					ColorStr.remove_suffix(ColorStr.size() - End);

					if (!ColorStr.starts_with(L";rgb:"sv))
						throw invalid_response(Index);

					ColorStr.remove_prefix(L";rgb:"sv.size());

					if (ColorStr.size() != L"ffff/ffff/ffff"sv.size())
						throw invalid_response(Index);
				}
				catch (const std::runtime_error& e)
				{
					Beep(500, 200);

					write(e.what());
					write(L": "sv);
					write(printable(ColorStr));
					write(L"\n"sv);
				}

				++Index;
			}

			std::wcout << L"Everything is OK"sv << std::endl;

		}
	}
	catch (const std::exception& e)
	{
		std::cerr << e.what() << std::endl;
	}
}

Run it in WT.

Expected Behavior
The program should print "Everything is OK" in a loop indefinitely and nothing else:
Everything is OK
Everything is OK
Everything is OK
Everything is OK
...and so on.

Actual Behavior
The program prints "Everything is OK" a few times, then it detects errors due to missing bits in the reply, e.g.
Everything is OK
Everything is OK
Everything is OK
Everything is OK
Invalid response at #234: c/1c1c␛
Everything is OK

Eventually it deadlocks in ReadConsole.
Adding a delay between sending the query and reading the reply improves the situation dramatically, but still not to 100% and it does not feel like the right thing to do: fixing issues with Sleep(N) never works in the long term.
Splitting it to smaller queries also helps, but again, it is guesswork.
Overall it is about 2700 characters to send and about 7000 to receive. It does not look like something humongous by modern standards.
Moving the mouse or pressing keyboard keys noticeably increases the error rate. Again, only in WT.

I believe this is caused by #16224. Since the terminal lock isn't held anymore, any thread can write into the input (and response) pipe concurrently (in our case: the VT parser thread sending VT responses and the UI thread sending cursor messages).

Thanks.

and the UI thread sending cursor messages

Just to confirm: bad replies happen even when there are no cursor messages (ENABLE_MOUSE_INPUT is not set) or any other input.
Also, I don't know if it's related or not, but every time I terminate the repro with ^C the shell crashes:

I thought maybe I've built WT wrong or something, but the fact that it fails in ReadKey is suspicious.

Just to confirm: bad replies happen even when there are no cursor messages (ENABLE_MOUSE_INPUT is not set) or any other input.

Aside from mouse messages and keyboard input, we also send focus events to ConPTY from the UI thread.

Also, I don't know if it's related or not, but every time I terminate the repro with ^C the shell crashes: [...]

Yeah, I noticed that as well. I believe it's https://github.com/dotnet/runtime/issues/88697 which points to our issue #15859. I've set it to be fixed in v1.23 now.
The weirdest thing is that this issue doesn't occur anymore after my PR. (It does break PSReadLine though, because ENABLE_MOUSE_INPUT doesn't get disabled on exit.)

Thanks for such a quick fix!


Improvement

Rather than checking for specific container classes, you could use
thing.respond_to?(:empty?)

To check if you can call empty

Hi @sandymcp, thank you for reaching out.
I am sure it can be an improvement, but what I am not sure is what to? Do you have an example?
Thank you in advance!

Closing.


update dependencies

I have image_cropper package in my project. When I try to install dart_code_linter, I get this error:
Because image_cropper >=5.0.0 depends on image_cropper_platform_interface ^5.0.0 which depends on http ^1.0.0, image_cropper >=5.0.0 requires http ^1.0.0. And because every version of dart_code_linter depends on http ^0.13.5, image_cropper >=5.0.0 is incompatible with dart_code_linter. So, because the App depends on both image_cropper ^5.0.0 and dart_code_linter any, version solving failed..
Please update your dependencies.

Same here with http package: we are using current http version 1.1.0 but dart_code_linter still requires version 0.13.5
So +1 for upgrading the dependencies!

This is fixed on the branch. @santitigaga can you please create a new version on pub.dev

Please also update uuid to the newest (4.2.1).

ping @santitigaga

I also have a dependency version problem with uuid: Because every version of dart_code_linter depends on uuid ^3.0.7 and my_project depends on uuid ^4.1.0, dart_code_linter is forbidden.

The analyzer dependency was changed to 6.0.0 which introduces many breaking changes. The project right now does not compile. I think that change should be reverted.

Hi all, sorry for the delay in replying. I reverted the change of the dependency analyzer from version 6 to version 5, adjusted the failed unit tests and added this control in the CI pipeline.  #139
Once the pull request is completed a new version will be generated.


added laravel 9

Update composer file compatible with laravel version 9

CI is broken, please fix it @Sanjay-100
Also please this in the tests strategies

Ping @Sanjay-100

Just adding the || 9.0 to composer probably won't be enough. Tests will need to be updated and we will need to make sure that there are not any high / medium impact changes https://laravel.com/docs/9.x/upgrade
One of them is going to be the requirement to be on at least php 8.0.2 so the CI/CD flows might need to be amended.
As @lucasmichot mentioned, @Sanjay-100 can you update your PR as it is broken.

Closed by #1100


Fix: Wiki links to secret notes, list numbering on Firefox, add required OpenGraph properties, css typo

Fix wiki links to secret notes.
Add missing OG properties: og:type and og:url. og:locale is optional and defaults to en_US. (closes #10)
Reference: https://ogp.me/#metadata

Also added:
Fix list numbering on Firefox
List numbering was messed up on Firefox because of this regression:
https://bugzilla.mozilla.org/show_bug.cgi?id=1548753
This fixes the issue by having the events (ul) inside the entries (li), not alongside them.
Before:

After:


[1.2-beta1] Performance issue with processing patterns and ItemStackList

Issue description:
Something in the way it manages adding items to machines using processing patterns is just hammering the server.
What happens:
Looking at the machines, including a Mekanism crusher and smelter (both elite and doing 7 at a time, and with 4 speed upgrades, set to auto-sort), and the solderer, while crafting two 64k storage disks, the progress arrows are jerky and the machines are running at far less than the speed they should be. The client is fine, but the server is reporting "Can't keep up!" messages. They ceased once the crafting operation was completed. If I manually insert a stack of iron into the solderer when no autocrafting is happening, it runs smoothly and without jerking.
So I, of course, ran some profiling with Sampler, and loaded up the .nps file in VisualVM. I see a few things taking a lot of processing time on the server. For example, ItemStackList.add(), ItemStackList.copy(), TileController.func_73660_a(), CraftingTask.update(). There's others like Comparer.isEqual() and isEqualNoQuant() that also might be taking up more time than desirable.
The .nps file can be found here: https://www.dropbox.com/s/3biafjq8kv6qlts/rs-beta-profile.nps?dl=0
You know, on the good side, it completed and never jammed up once! :D
What you expected to happen:
For it to not result in jerky, slow processing, and for it to be kind to the server.
Steps to reproduce:

Set up processing for 64k storage disks. I used a Mekanism crusher and smelter, but other machines should have the same result (as the solderer seemed also affected).
Request two 64k storage disks.
Observe the machines and notice they are operating in a jerky fashion, and the server is being hammered.

Should still happen with other types of items and patterns, but this is the one I used to test, since that would often jam up with previous versions (and I needed more storage space, anyway).
Version (Make sure you are on the latest version before reporting):

Minecraft: 1.10.2
Forge: <IP_ADDRESS>7
Refined Storage: 1.2-beta1

Does this issue occur on a server? [yes/no] Yes
If a (crash)log is relevant for this issue, link it here:
No crash or crash log, but see profiler file above.

Please try this one :)
refinedstorage-1.2-beta2-test.zip

Wow, you two are fast! Giving it a try not, thanks. :)

It's definitely improved a bit, but still an issue. Server is spitting out "Can't keep up!" messages, still. Sampler says CraftingTask.update() and ItemStackList.copy() are still taking up a fair bit.
The .nps file: https://www.dropbox.com/s/kdvdn8eglcmloe7/rs-beta-profile2.nps?dl=0
Can verify I'm using the test version on both sides: b230b930fa93b8fe0abe52941428fa60  mods/refinedstorage-1.2-beta2-test.jar

Call tree leading to it seems to be:
TileController.func_73660_a()->CraftingTask.update()->CraftingStep.canStartProcessing()->ItemStackList.copy()

Yeah, the issue is with the copy of an items stack. As that list gets big it seems to be really slow.

Just so you know, though I didn't get the extremely recent disk drive rendering code, I built it from Github to see if the performance changes helped. Everything is smooth as butter, and ancient legends say the server isn't spewing "Can't keep up!" messages either. So looks like you nailed it. Good job, way2muchnoise.  Very detailed bug report WITH profiling and you even tested from GH yourself.  Thanks!

hehe No prob. I've been a professional developer for a rather long time now. I know how to make good bug reports, at least. lol
Though I don't know much about MC modding, it'd be nice to learn, though. Was thinking this project would be a nice one to be involved in, but my lack of knowledge about MC modding is boundless. We shall see. Being able to build it from source is a good first step, though. Anyway, my numerous bug reports to many projects is one way I can contribute. :)
Thanks to you both, and the other people who have contributed!

Thx @RealGrep the profiling really helps out.  lol


Add extractComments option to be used on uglifyOptions.js

What problem does this feature solve?
A developer should have an option to extract coments into a new file when minifying js files.
In some cases this option can generate less kb.
Today it can be done by using configureWebpack:
// vue.config.js
module.exports = {
  configureWebpack: config => {
    if (process.env.NODE_ENV === 'production') {
      config.optimization.minimizer[0].options.extractComments = true
      config.optimization.minimizer[0].options.uglifyOptions.output.comments = false
    }
  }
}

What does the proposed API look like?
The new option can be named productionExtractComments  and it will work like productionSourceMap option.
When options.productionExtractComments  is true, uglifyOptions.js should receive extractComments: true and uglifyOptions.output.comments: false (it doesn't removes all the extracted comment if equals /^\**!|@preserve|@license|@cc_on/).
If options.productionExtractComments  is false, uglifyOptions.js should be default (extractComments: true and uglifyOptions.output.comments: /^\**!|@preserve|@license|@cc_on/).
It must be included in https://cli.vuejs.org/config/#vue-config-js
In the future, productionSourceMap and productionExtractComments  can be renamed and realocated inside a js or other options, like the css one.

Since this is literally done in 3 lines of code and, at waker in my experience, not a widely used feature (this isn't he first time I even hears about it), I don't think this will get its own config option.
The goal the config extensibility and plugin system are to enable users to make small changes like this themselves and extract bigger ones into their own plugin.

Agreed, since it does little difference.
I posted that here because I saw many devs asking about it on Vue Land and they don't got the answer.
I also don't know if config.optimization.minimizer[0].options is the best way to do that or if the configureWebpack is the best place. By your comment I can assume it is not wrong, but it could be easier (it take me some hours to figure out).
Since it is a faq and not following the common plugin settings way, it could/should/must be included into docs, as I said in a general way, like "you can edit uglify or js minification settings by...".
You can consider this issue as a suggestion that can save people time.

So is there a way to tweak the minification options yet?

OP already provided one right in his issue.
We won't add anything special for this.

His solution doesn't actually work. Maybe it did < Webpack 4? Regardless, inside of chainWebpack, minimizer is a function, so I can't really get at the minimizer options.
The only way I was able to extract comments or adjust any license-related comment stuff is to replace the entirety of the minimizer with a Terser plugin.
This is undesirable because I generally want to leave the Vue-cli webpack configuration alone. I just want to tweak an option here or there.

I think what I'm looking for is: Is there a better way to pass options into the uglify/minify settings without having to replace it wholesale? I just feel like it's totally overkill.

The UP used configureWebpack.
In chainWebpack it's a function, true. All webpack-chain interfaces are functions.
See here:
https://github.com/neutrinojs/webpack-chain/blob/master/README.md#config-optimization-minimizers-modify-arguments
If you need help in actually using chainWebpack, please open a topic on forum.vuejs.org

Ah, jeeze, I misread. Thank you so much! Cheers.


redhat don't have systemd --user process.

i can install systemd package ,and start systemd --user, but it need to work in more operator.
i don't know whether home-manager can integrate internal user systemd service module.
[op@my-200 ]$ sudo XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR /nix/store/4xn6z5k11ixb1sy278jn02bi40lh7aas-systemd-243.7/lib/systemd/systemd --user
[sudo] password for op:
Unit default.target not found.
Falling back to rescue target: rescue.target
Unit rescue.target not found.

it can be launched by system command easily.
sudo chmod -R o+rw /sys/fs/cgroup/systemd/user.slice/user-1000.slice
/usr/lib/systemd/systemd --user


OSError: libcuda.so.1: cannot open shared object file: No such file or directory

I exactly followed the http://mxnet.io/get_started/install.html instruction to install the GPU version in the server, however, I am unable to install it successfully.
Environment info
Operating System: CentOS release 6.9 (Final)
Compiler: gcc version 4.8.1 (GCC)
Package used (Python/R/Scala/Julia): Python
MXNet version:  mxnet-cu80
Or if installed from source: No.
MXNet commit hash (git rev-parse HEAD): NA
If you are using python package, please provide
Python version and distribution: Python 3.5.2 |Anaconda 4.1.1
If you are using R package, please provide
R sessionInfo():
Error Message:
Please paste the full error message, including stack trace.
Minimum reproducible example
if you are using your own code, please provide a short script that reproduces the error.
Traceback (most recent call last):
File "", line 1, in
File "/home/panwei/wuxx0845/.local/lib/python3.5/site-packages/mxnet/init.py", line 7, in
from .base import MXNetError
File "/home/panwei/wuxx0845/.local/lib/python3.5/site-packages/mxnet/base.py", line 52, in
_LIB = _load_lib()
File "/home/panwei/wuxx0845/.local/lib/python3.5/site-packages/mxnet/base.py", line 44, in _load_lib
lib = ctypes.CDLL(lib_path[0], ctypes.RTLD_GLOBAL)
File "/panfs/roc/msisoft/anaconda/anaconda3-4.1.1/lib/python3.5/ctypes/init.py", line 347, in init
self._handle = _dlopen(self._name, mode)
OSError: libcuda.so.1: cannot open shared object file: No such file or directory
Steps to reproduce
or if you are running standard examples, please provide the commands you have run that lead to the error.

I followed the steps as described on the website.

What have you tried to solve it?

I tried to fix the path, including PATH, USE_CUDA_PATH, LD_LIBRARY_PATH.
 I really want to install mxnet and use it. Thanks for your help in advance.

So the problem here is just that ctypes.CDLL can't find the libcuda. Could you try the following?
find / -type f -name libcuda.so.* -exec dirname {} \; 2>/dev/null

The above command prints the paths to libcuda.so.*, and you can append those to your LD_LIBRARY_PATH variable to fix the issue. Otherwise, you may want to look into installing cuda again since the libcuda isn't available on your system.

I have the same problem as you. I was trying to install mxnet-cu80 on supercomputer, when I met this problem. I also did
module load cuda/8.0
module load cudnn/5.1
But I still can't make it.

@zhangxgu
Do you solve the problems? At that time, I used Theano to do my project. I think it might be a good time to learn mxnet.

@ChongWu-Biostat did you try the proposed fix above?

@szha I don't think that works. I am unable to find libcuda.so anyway.
Thanks,
Chong

@ChongWu-Biostat you need your cuda driver and the cuda library to use the GPU version. Are you able to find and install the libcuda through other methods?

@szha  I use the university super computing source. I load all the required packages and software and with the right version, I think. How do you solve this problem? Thanks for your help in advance...
Thanks,
C

@ChongWu-Biostat you should try to get help from your university on locating the cuda driver and the cuda library, and once you find the location, adding it to LD_LIBRARY_PATH variable fixes the problem.

accroding to the LD_LIBRARY_PATH, cuda seems installed at /panfs/roc/msisoft/cuda/8.0/
you can try if you have libcuda.so on the directory, often it is located at
/panfs/roc/msisoft/cuda/8.0/lib64/stubs/libcuda.so

Yes, I find libcuda.so under the folder /panfs/roc/msisoft/cuda/8.0/lib64/stubs/libcuda.so. However, I am unable to find libcuda.so.1 under this folder.

Does creating a symlink in that path work?
ln -s libcuda.so /panfs/roc/msisoft/cuda/8.0/lib64/stubs/libcuda.so.1

@szha @mli
Thanks for your help. I solve the problems. It simply because I do not login into the right server, which contains GPUs.


Add support for parsing datetimes from strings

Description
Currently, the only way to create a datetime is to use now() or the datetime constructor with individual components explicitly provided. However, that is essentially never useful for ingesting external inputs, meaning that for any data coming in from JSON, YAML, etc., the user has to hack in a string parser themselves to be able to use time manipulation functions in any capacity. That's a big drawback for things like invoice templates.
There should be a way to parse a datetime-containing string into a datetime that's built-in. There was a previous PR in #1668 which aimed to implement parsing Unix timestamps, but it got closed in favour of a generalised implementation that could parse more formats. It doesn't seem that that implementation ever materialised, nor does there seem to be an existing discussion of it. I'm opening this issue to hopefully revive the topic.
Use Case
Any situation in which external data are ingested and contain dates that need to be manipulated. For example:

Generating invoices with due dates automatically calculated, based on the contents of a JSON file
Displaying a date value in a different format than the one given (i.e. "2024-07-01" → "1 July 2024")
Extracting a date to be displayed out of a a full timestamp given as a string
Generating a calendar for a given month, based on a value passed in through sys.inputs

Yaml support datetime. You can even do something like
#let date = yaml.decode("date: 2024-07-01").date

Not inside typst, that is actually a string. Try this:
#let date = yaml.decode("date: 2024-07-01").date
#type(date)

I don't believe YAML supports timestamps as much as it allows applications to recognise certain values and convert them to timestamps automatically. Either way, that does nothing for other formats such as JSON or sys.inputs, so the need for a parser is still there.

I was wrong, it works for toml, not yaml

I was wrong, it works for toml, not yaml, so you can define function
#let parse(s) = toml.decode("date = " + s).date

That's a neat hack! Though I still think it should be promoted to properly supported functionality :)

Hi, I am interested in this issue. Some details need discussion:

What datetime format should we support?

Some limited formats: [year]-[month]-[day], [hour]:[minute]:[second], [year]-[month]-[day] [hour]:[minute]:[second] ...
Customized formats via https://typst.app/docs/reference/foundations/datetime/#format

What happen if the string cannot be parsed?

Compiler errors (with hints)
Return none or something else

I implemented a simple version just now. It could parse datetime with default/custom formats.
= Parse with default formats

#datetime.parse("2024-05-26")

#datetime.parse("23:59:59")

#datetime.parse("2024-05-26 23:59:59")

= Parse with custom formats

#datetime.parse("2024/05/26", "[year]/[month]/[day]")

#datetime.parse("23_59_59", "[hour]_[minute]_[second]")

#datetime.parse("2024/05/26 23_59_59", "[year]/[month]/[day] [hour]_[minute]_[second]")

I don't know if it's desirable, but there's also this feature request https://github.com/typst/typst/issues/1658 @Coekjan
(Seems unergonomic to me, but who am I to judge)

I think these features could be useful when processing data from files (yaml, json, etc.). These features enable users to extract/parse datetime from their string data. For example, there is a yaml file and a  typst file:
---
date: 2024/05/26
# ...

#data = yaml("/path/to/data.yaml")
#data.date // this is a string
#datetime(data.date, "[year]/[month]/[day]").year() // we can extract year from the string

@Coekjan: thanks for taking a crack at it!
I'd say ISO 8601 should definitely be a supported (and likely default) format. Auto-discovery would be nice, but from what I understand, the underlying crate doesn't really allow that, so a couple shorthands to allow specifying well-known formats easily would be good. Especially since ISO 8601 has a couple allowed variant forms so specifying it through a manual format string is not trivial.
As for parsing failure, typst convention is generally to fail with an explicit error, so that's probably what I'd suggest. OTOH, this would also make it hard for the caller to try through several possible formats in lieu of auto-discovery, since there isn't really any error handling facility. Perhaps if the format could be passed in as a list of formats to try in order before it fails? I.e.
#datetime.parse(data.date, (none, // default format, same as "iso8601"
                            "[year]/[month]/[day]",
                            "[year].[month].[day]")).year()

Current datetime display api does not support iso 8601 directly. If we allow iso format here, it seems a little asymmetrical.
I think we had better consult the team why the display api was designed not to use iso 8601 (or something else).

@Enivex: #1658 is what led to #1668, which got closed as insufficiently general (rightly so, IMHO). However, being able to ingest unix timestamps as one of the options would be very useful, it is a real format that many things generate.

This is definitely not the way to do it, but thanks to foreign runners like pyrunner for Python (RustPython), jogs for JavaScript (QuickJS) and ctxjs, if you really need this feature immediately, you can actually achieve this in a rather detouring way by the following:
pyrunner (Python)
#import "@preview/pyrunner:0.1.0" as py

#let parse-iso-to-utc(s) = {
  let compiled = py.compile(```python
  from datetime import datetime, timezone
  def parse_iso_to_utc(iso_string: str) -> dict[str, int]:
      dt = datetime.fromisoformat(iso_string)
      dt = dt.astimezone(timezone.utc)
      return {
          "year": dt.year,
          "month": dt.month,
          "day": dt.day,
          "hour": dt.hour,
          "minute": dt.minute,
          "second": dt.second
      }
  ```)
  let (year, month, day, hour, minute, second) = py.call(compiled, "parse_iso_to_utc", s)
  datetime(year: year, month: month, day: day, hour: hour, minute: minute, second: second)
}
#parse-iso-to-utc("2024-11-10T15:00:00+09:00")

#let parse-iso-to-local(s) = {
  let compiled = py.compile(```python
  from datetime import datetime

  def parse_iso_to_local(iso_string: str) -> dict[str, int]:
  dt = datetime.fromisoformat(iso_string)
  return {
      "year": dt.year,
      "month": dt.month,
      "day": dt.day,
      "hour": dt.hour,
      "minute": dt.minute,
      "second": dt.second
  }
  ```)
  let (year, month, day, hour, minute, second) = py.call(compiled, "parse_iso_to_local", s)
  datetime(year: year, month: month, day: day, hour: hour, minute: minute, second: second)
}

#parse-iso-to-local("2024-11-10T15:00:00+09:00")

datetime(
    year: 2024,
    month: 11,
    day: 10,
    hour: 6,
    minute: 0,
    second: 0,
)
datetime(
    year: 2024,
    month: 11,
    day: 10,
    hour: 15,
    minute: 0,
    second: 0,
)

jogs (JavaScript)
#import "@preview/jogs:0.2.3" as jogs

#let parse-iso-to-utc(s) = {
  let bytecode = jogs.compile-js(```javascript
    function parseISOToUTC(isoString) {
        const date = new Date(isoString);
        return {
            year: date.getUTCFullYear(),
            month: date.getUTCMonth() + 1,
            day: date.getUTCDate(),
            hour: date.getUTCHours(),
            minute: date.getUTCMinutes(),
            second: date.getUTCSeconds()
        };
    }
  ```)
  let (year, month, day, hour, minute, second) = jogs.call-js-function(bytecode, "parseISOToUTC", s)
  datetime(year: year, month: month, day: day, hour: hour, minute: minute, second: second)
}

#parse-iso-to-utc("2024-11-10T15:00:00+09:00")

#let parse-iso-to-local(s) = {
  let bytecode = jogs.compile-js(```javascript
    function parseISOToLocal(isoString) {
        const date = new Date(isoString.substring(0, isoString.indexOf("+")));
        return {
            year: date.getUTCFullYear(),
            month: date.getUTCMonth() + 1,
            day: date.getUTCDate(),
            hour: date.getUTCHours(),
            minute: date.getUTCMinutes(),
            second: date.getUTCSeconds()
        };
    }
  ```)
  let (year, month, day, hour, minute, second) = jogs.call-js-function(bytecode, "parseISOToLocal", s)
  datetime(year: year, month: month, day: day, hour: hour, minute: minute, second: second)
}

#parse-iso-to-local("2024-11-10T15:00:00+09:00")

datetime(
    year: 2024,
    month: 11,
    day: 10,
    hour: 6,
    minute: 0,
    second: 0,
)
datetime(
    year: 2024,
    month: 11,
    day: 10,
    hour: 15,
    minute: 0,
    second: 0,
)


Compilation for iOS in release mode fails

swift-atomics doesn't compile for iOS in release mode when building with Xcode (but succeeds in debug).
Information

Package version: main / 46a69523595b5142b30b20f922a1324586bdc96f
Platform version: 11.5.2
Swift version: swift-driver version: 1.26.7 Apple Swift version 5.5 (swiftlang-13<IP_ADDRESS> clang-13<IP_ADDRESS>) Target: x86_64-apple-macosx11.0
Xcode version: 13.0 beta 4 (13A5201i)

Checklist

[x] If possible, I've reproduced the issue using the main branch of this package.
[x] I've searched for existing reports of the same issue.

Steps to Reproduce
Create a new iOS project from the App template in Xcode, add the swift-atomics package as a dependency under Package Dependencies (using branch: main) and set the build configuration in Edit Scheme... to Release.
Expected behavior
The build for iOS in release mode succeeds.
Actual behavior
The build for iOS in release mode fails with the following 2 errors:
1.)
CompileC /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Intermediates.noindex/swift-atomics.build/Release-iphonesimulator/_AtomicsShims.build/Objects-normal/i386/_AtomicsShims.o /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/SourcePackages/checkouts/swift-atomics/Sources/_AtomicsShims/src/_AtomicsShims.c normal i386 c com.apple.compilers.llvm.clang.1_0.compiler (in target '_AtomicsShims' from project 'swift-atomics')
    cd /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/SourcePackages/checkouts/swift-atomics
    export LANG\=en_US.US-ASCII
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x c -target i386-apple-ios9.0-simulator -fmessage-length\=0 -fdiagnostics-show-note-include-stack -fmacro-backtrace-limit\=0 -fmodules -gmodules -fmodules-cache-path\=/Users/lx/Library/Developer/Xcode/DerivedData/ModuleCache.noindex -fmodules-prune-interval\=86400 -fmodules-prune-after\=345600 -fbuild-session-file\=/Users/lx/Library/Developer/Xcode/DerivedData/ModuleCache.noindex/Session.modulevalidation -fmodules-validate-once-per-build-session -Wnon-modular-include-in-framework-module -Werror\=non-modular-include-in-framework-module -fmodule-name\=_AtomicsShims -fapplication-extension -Wno-trigraphs -fpascal-strings -Os -Wno-missing-field-initializers -Wno-missing-prototypes -Wno-return-type -Wno-missing-braces -Wparentheses -Wswitch -Wno-unused-function -Wno-unused-label -Wno-unused-parameter -Wno-unused-variable -Wunused-value -Wno-empty-body -Wno-uninitialized -Wno-unknown-pragmas -Wno-shadow -Wno-four-char-constants -Wno-conversion -Wno-constant-conversion -Wno-int-conversion -Wno-bool-conversion -Wno-enum-conversion -Wno-float-conversion -Wno-non-literal-null-conversion -Wno-objc-literal-conversion -Wno-shorten-64-to-32 -Wpointer-sign -Wno-newline-eof -DSWIFT_PACKAGE -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator15.0.sdk -fasm-blocks -fstrict-aliasing -Wdeprecated-declarations -g -Wno-sign-conversion -Wno-infinite-recursion -Wno-comma -Wno-block-capture-autoreleasing -Wno-strict-prototypes -Wno-semicolon-before-method-body -I/Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Products/Release-iphonesimulator/include -I/Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/SourcePackages/checkouts/swift-atomics/Sources/_AtomicsShims/include -I/Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Intermediates.noindex/swift-atomics.build/Release-iphonesimulator/_AtomicsShims.build/DerivedSources-normal/i386 -I/Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Intermediates.noindex/swift-atomics.build/Release-iphonesimulator/_AtomicsShims.build/DerivedSources/i386 -I/Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Intermediates.noindex/swift-atomics.build/Release-iphonesimulator/_AtomicsShims.build/DerivedSources -F/Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Products/Release-iphonesimulator -F/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/Library/Frameworks -iframework /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator15.0.sdk/Developer/Library/Frameworks -DXcode -MMD -MT dependencies -MF /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Intermediates.noindex/swift-atomics.build/Release-iphonesimulator/_AtomicsShims.build/Objects-normal/i386/_AtomicsShims.d --serialize-diagnostics /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Intermediates.noindex/swift-atomics.build/Release-iphonesimulator/_AtomicsShims.build/Objects-normal/i386/_AtomicsShims.dia -c /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/SourcePackages/checkouts/swift-atomics/Sources/_AtomicsShims/src/_AtomicsShims.c -o /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Intermediates.noindex/swift-atomics.build/Release-iphonesimulator/_AtomicsShims.build/Objects-normal/i386/_AtomicsShims.o

In file included from /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/SourcePackages/checkouts/swift-atomics/Sources/_AtomicsShims/src/_AtomicsShims.c:13:
/Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/SourcePackages/checkouts/swift-atomics/Sources/_AtomicsShims/include/_AtomicsShims.h:339:28: error: unknown type name '__uint128_t'
_sa_dword _sa_decode_dword(__uint128_t value) {
                           ^
1 error generated.

2.)
CompileSwiftSources normal arm64 com.apple.xcode.tools.swift.compiler (in target 'Atomics' from project 'swift-atomics')
    cd /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/SourcePackages/checkouts/swift-atomics
    export DEVELOPER_DIR\=/Applications/Xcode.app/Contents/Developer
    export SDKROOT\=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator15.0.sdk
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -incremental -module-name Atomics -O -whole-module-optimization -enforce-exclusivity\=checked @/Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Intermediates.noindex/swift-atomics.build/Release-iphonesimulator/Atomics.build/Objects-normal/arm64/Atomics.SwiftFileList -DSWIFT_PACKAGE -DXcode -sdk /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator15.0.sdk -target arm64-apple-ios9.0-simulator -g -module-cache-path /Users/lx/Library/Developer/Xcode/DerivedData/ModuleCache.noindex -Xfrontend -serialize-debugging-options -application-extension -swift-version 5 -I /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Products/Release-iphonesimulator -I /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/lib -F /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Products/Release-iphonesimulator/PackageFrameworks -F /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Products/Release-iphonesimulator -F /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/Library/Frameworks -F /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator15.0.sdk/Developer/Library/Frameworks -c -num-threads 8 -output-file-map /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Intermediates.noindex/swift-atomics.build/Release-iphonesimulator/Atomics.build/Objects-normal/arm64/Atomics-OutputFileMap.json -parseable-output -serialize-diagnostics -emit-dependencies -emit-module -emit-module-path /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Intermediates.noindex/swift-atomics.build/Release-iphonesimulator/Atomics.build/Objects-normal/arm64/Atomics.swiftmodule -Xcc -I/Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Intermediates.noindex/swift-atomics.build/Release-iphonesimulator/Atomics.build/swift-overrides.hmap -Xcc -I/Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/SourcePackages/checkouts/swift-atomics/Sources/_AtomicsShims/include -Xcc -I/Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Products/Release-iphonesimulator/include -Xcc -I/Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Intermediates.noindex/swift-atomics.build/Release-iphonesimulator/Atomics.build/DerivedSources-normal/arm64 -Xcc -I/Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Intermediates.noindex/swift-atomics.build/Release-iphonesimulator/Atomics.build/DerivedSources/arm64 -Xcc -I/Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Intermediates.noindex/swift-atomics.build/Release-iphonesimulator/Atomics.build/DerivedSources -Xcc -DSWIFT_PACKAGE -emit-objc-header -emit-objc-header-path /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/Build/Intermediates.noindex/swift-atomics.build/Release-iphonesimulator/Atomics.build/Objects-normal/arm64/Atomics-Swift.h -working-directory /Users/lx/Library/Developer/Xcode/DerivedData/ldsdhs-bzmdchssanulrqdlkqewpmdcmpmq/SourcePackages/checkouts/swift-atomics

remark: Incremental compilation has been disabled: it is not compatible with whole module optimization
Command CompileSwiftSources failed with a nonzero exit code

Ah, this is a regression introduced in #26. It seems we will need to beef up CI coverage.

Thanks so much for looking into it, much appreciated!

Submitted a PR with the fix -- apologies for not catching this earlier.

Awesome, that was a super fast fix, thanks again!


Readme Typo

Hi Yi! I know this is minor but I thought I might as well point it out: This title has an extra 's' (Should be "What's Next"). No PR because this is so tiny 😛
Thanks for making this project, it's been very nice to read through!

Hi thank you for pointing this out. Already fixed.


Drop Python 3.8

The key dependency Flask, no longer supports Python 3.8:
https://github.com/pallets/flask/blob/bc098406af9537aacc436cb2ea777fbc9ff4c5aa/pyproject.toml#L22

Will be completed by PR #374


Unable to run JAR files

Trying to run compiler.jar from Google Closure with java -jar compiler.jar results in this error message:
Error: Could not find or load main class -jar

Any jar file?
Or just that one?

I can't run any jar file too.

I suppose you have to do something like:
java -cp compiler.jar <compiler.jar's main domain>


build: remove custom variables for cmark handling

Use the imported target to handle the include path and link libraries.
This allows CMake to properly handle dependencies and removes the
custom handling logic that we have relied on now that CMark properly
supports export targets.

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

Can one of the admins verify this patch?

@compnerd I've checked out this branch locally, but it fails with this build error:
swift/lib/Markup/Markup.cpp:18:10: fatal error: 'cmark.h' file not found
#include "cmark.h"

It seems like the change from
  include_directories(
      "${CLANG_BUILD_INCLUDE_DIR}"
      "${CLANG_MAIN_INCLUDE_DIR}"
      "${CMARK_MAIN_INCLUDE_DIR}"
      "${CMARK_BUILD_INCLUDE_DIR}")

to
  include_directories(${CLANG_BUILD_INCLUDE_DIR}
    ${CLANG_MAIN_INCLUDE_DIR})

in this branch breaks that, as no replacement include directory is specified for the swiftMarkup target.
I assumed that something like
target_include_directories(swiftMarkup PRIVATE
  ${cmark_INCLUDE_DIR})

in lib/Markup/CMakeLists.txt would fix this, but it doesn't. I've also checked generated cmarkConfig.cmake and cmarkTargets.cmake, none of them seem to define anything related to include paths. Is this something that should be fixed in cmark CMake files to declare a PUBLIC header, or am I missing something?

I think that we may need to sync with upstream again?

Nothing has changed upstream in their CMake files though, at least when comparing their master to Apple's fork master: https://github.com/apple/swift-cmark/compare/master...commonmark:master

I'm seeing this both upstream and in the swift-cmark fork:
  install(FILES
    cmark.h
    ${CMAKE_CURRENT_BINARY_DIR}/cmark_export.h
    ${CMAKE_CURRENT_BINARY_DIR}/cmark_version.h
    DESTINATION include
    )

cmark.h is not in build/Ninja-ReleaseAssert/cmark-macosx-x86_64 (which I assume is what CMAKE_CURRENT_BINARY_DIR is) by the time swiftMarkup target is built, so I guess the install step for cmark hasn't happened at that time, but is required for the swiftMarkup target to get access to the header?
Sorry for such a trivial question, I'm just trying to wrap my head around how these dependencies are built, how their include paths are set up, and how target dependencies are structured in the toolchain CMake files. So I decided to start with something small such as cmark.

@compnerd another thing I wanted to try was to migrate the cmark dependency from build-script-impl to build-script or maybe even a direct CMake dependency, that's how I stumbled upon this PR. Do you think this migration would be worth trying as an exercise to get more comfortable with how CMake is used in the project? If so, should be migrated to be built from Python build-script or should be directly become a CMake dependency pulled with something like ExternalProject_Add or add_subdirectory?

The current CMake usage is not something we should be aiming to emulate.
What you’re thinking about (using cmake properly) is a good thing.  Making it built in build-script and passed in via CMark_DIR is the right approach.  However, be aware that some of us build with it unified (all of the toolchain in a single cmake invocation).
Migrating from build-script-impl to build script would be a good learning experience though, it would be welcome and good to see others get involved in the build cleanup efforts.

The unified/standalone builds is something that I saw references to in CMake files, but I still don't understand pros/cons and use cases of those. Would you elaborate please why some use unified builds then and how exactly that differs? I'd be happy to add that to docs/BuildManifesto.md or a separate doc file to bring that to a wider audience, which I hope would underline the importance of the improvements to the build system.

Well, different people have different reasons to use unified builds. If you have enough CPU cores, then it's dramatically faster than the build-script.

How do you even run it? I'm not able to find any documentation whatsoever about unified builds 🤔

If you have enough CPU cores, then it's dramatically faster than the build-script.
BTW, how many CPU cores is enough? 🙂

I don’t know how many cores are enough to appreciate the advantage of unified builds. I have 48 cores and (from memory), it’s about 50% faster than using the build script.
I also don’t know how others setup their unified builds, but for me, I just build LLVM and add swift, etc as external projects (as opposed to internal projects like clang, lld, lldb, etc)

The unified/standalone builds is something that I saw references to in CMake files, but I still don't understand pros/cons and use cases of those.

IMO the pros is that its a faster build (with my 4 core machine, it ends up being ~10-15% faster).  It also reduces the complexity of the build IMO, because the single build directory contains nearly everything (llvm, clang, lld, lldb, swiftc, and the swift runtime/standard library).  The last pro IMO is that when you update the repositories, it ensures that the dependencies fully percolate through the entire system.
As to the cons - largely that you get a single configuration type (release vs debug for the entire toolchain).  The other cons include things like not being able to do a partial incremental build (e.g. if you update and LLVM has changed, you will need to rebuild LLVM, you cannot go behind its back and do just the incremental swift build).

I also don’t know how others setup their unified builds, but for me, I just build LLVM and add swift, etc as external projects (as opposed to internal projects like clang, lld, lldb, etc)

@davezarzycki I dislike the setup I have currently in that I have swift checked out not where I would like.  In theory, you should be able to use LLVM_EXTERAL_<PROJECT>_SOURCE_DIR to avoid the checkout layout that I have (where swift is next to clang).  But, I basically build llvm with clang, lld, lldb, cmark, and swift as a single build.

Please update the base branch to main by Oct 5th otherwise the pull request will be closed automatically.

How to change the base branch: (Link)
More detail about the branch update: (Link)