Updates to disable-tls-1.0-1.1.md

Changes:

Updated the requirement for Skype for Business Server 2019 to CU1.
Updated the requirement for Skype for Business Server 2015 to CU9.

Note: These two changes might need to be reverted and added as a mentions, since all the subsequent procedures were written for Skype for Business Server 2015 CU6 HF2 and Skype for Business Server 2019.

Moved Lync Room System (a.k.a. SRSv1) to 'Out of scope'.
Moved Microsoft Teams Rooms and Surface Hub from to 'Fully tested'.
Removed SBA and SBS since already documented as 'Fully tested'.
Miscellaneous formatting changes.

@nenonix @JohanFreelancer9 Copy editing is needed for this PR. Thanks.

@LanaChin PR has been copyedited and is ready for final review, could you please check and merge? Thanks!

@robertpp - can you commit @JohanFreelancer9's copy edit changes?  After that, I'll merge the changes. Thank you.

Hi @LanaChin. All locales removal from links suggested by @JohanFreelancer9 have been committed.

Committed the locale changes proposed by @nenonix.


Allow threads to be closed in runner factory

Use lambda to capture common case
Fixes #674

Updated following discussion

That's everything


[TIR] Tir constants integration into compilation pipeline

This PR is a follow-up PR to integate tir constant nodes introduced in 8472 to compilation pipeline.

d-smirnov requested review from anijain2305, areusch, comaniac, jroesch, junrushao1994, jwfromm, kparzysz-quic, MarisaKirisame, masahi, mbrookhart, merrymercy, slyubomirsky, tqchen, vinx13, wweic, yzhliu, zhiics and ZihengJiang as code owners 12 hours ago

Oh. not sure how did I manage to do this. I am sorry for such a massive query.

@d-smirnov , Since this is a follow up PR (thus, cannot be merged before the dependent one is merged), shall we put this to draft for now ?

Summarizing our offline discussion, we decided to close #8472 and maintain a single PR which is this one.
We are still fixing few unit tests, however, we would appreciate if @junrushao1994 could take a look here.

Thank you for pinging me!! I'm happy to review this weekend :-)

@junrushao1994 I think the concerns are addressed, can you take a look please ? It would be great to progress this PR as it been a while...

Thanks @d-smirnov working so long on this to enable representation of non-scalar constants in TIR.
Thanks @giuseros for the initial work..
Thanks @junrushao1994 and @areusch for reviews!

We're seeing issues on Hexagon after this commit.  They are directly caused by float16 not being supported for embedded constants, but there are other potential problems there (difference between target and executor in fuse_ops).  See https://discuss.tvm.apache.org/t/problem-with-fuseops-and-embedded-constants-in-tir/12165 for more info.  I'm wondering about your thoughts about this.

@kparzysz-quic thanks for signaling the regression. Is there a way this problem can be triggered via a simple unit test? Perhaps we can get the authors to issue a patch, or perhaps even we could consider reverting this commit until we can resolve the issue?

We can look into this.
@kparzysz-quic , would you be able to submit a PR with an xfail that should have been broken by this PR ?
(It is bit harder to track tests that are not in tree).

They are directly caused by float16 not being supported for embedded constants

https://github.com/apache/tvm/blob/dcbdeddba290caee4f604e678a7e315aea038881/src/target/llvm/codegen_params.cc#L137-L157
https://github.com/apache/tvm/blob/dcbdeddba290caee4f604e678a7e315aea038881/src/target/source/codegen_params.cc#L214-L229
The above are called respectively in the following locations :
https://github.com/apache/tvm/blob/dcbdeddba290caee4f604e678a7e315aea038881/src/target/llvm/codegen_llvm.cc#L1485
https://github.com/apache/tvm/blob/dcbdeddba290caee4f604e678a7e315aea038881/src/target/source/codegen_c.cc#L673
(These are same code used to codegen LinkedParam node)
So, I am not following what is meant here.. Can you elaborate further ?
To understand a bit more, would you be able to share the relay.build(...) configuration (targets, executor, runtime) that you are using ? Ideally, if the target does not support -link-params, it should be able to disable it or if the target needs them linked (i.e. not handled via executor) it needs codegen tir.AllocateConst node -- for which the support has been added for all backends of TVM (with tests). By naive, look at the Hexagon codegen it seems like it is extending codegen LLVM.
So what we need to understand, what is the expectation of Hexagon backend when linked-param is used ?
(I suppose it requires LinkedParam nodes ? )
I am sensing we might need to move link-param from being executor attr to a target attr . cc : @Mousius @d-smirnov
Im a bit concerned by the suggestion to revert based non-tree tests. FYI : @u99127

We have a temporary workaround---just reset the link_params variable to false in fuse_ops, so the urgency for us is lower.
There are two issues that need to be addressed:

The link_params property is inconsistent with the CPU target in FuseOps, when invoked through FoldConstants.
Float16 not being supported in AllocateConst visitors, e.g. https://github.com/apache/tvm/blob/main/src/relay/backend/te_compiler_cache.cc#L242

Hexagon does utilize the LinkedParams function, but we can change that as needed.  So far, without any workarounds, we don't even get to the Hexagon codegen.

Hi @kparzysz-quic ,
This PR is about non-scalar constants, 2) links to scalar constants -- maybe that feature is yet to be supported anyway ?
For 1) maybe lets create an issue -- Im trying to understand the relevance to this PR, @d-smirnov thoughts ?

Right---the relevance is that I tried setting "relay.FuseOps.link_params" to False in PassContext/config, but it didn't have any effect, since FoldConstants creates its own empty context.  It also uses a new CPU target for folding, which has link_params=False.  However, FuseOps uses the Executor attribute from IR module, which has link_params taken from the original target (i.e. True).

However, FuseOps uses the Executor attribute from IR module, which has link_params taken from the original target (i.e. True).

I am still a bit puzzled.
https://github.com/apache/tvm/pull/8509/files#diff-8b95b87a2611e5e0c367ce17396b051fd868aa260c51ce0e50b94564fcb0e71fR1054-R1057
I read the above code as PassContext overriding whatever provided by the Executor.
cc : @d-smirnov

Would you be able to provide what should be the "correct" behaviour ?

However, FuseOps uses the Executor attribute from IR module, which has link_params taken from the original target (i.e. True).

I am still a bit puzzled.
https://github.com/apache/tvm/pull/8509/files#diff-8b95b87a2611e5e0c367ce17396b051fd868aa260c51ce0e50b94564fcb0e71fR1054-R1057
I read the above code as PassContext overriding whatever provided by the Executor. cc : @d-smirnov

Line 1054: If executor is present in IRModule, set link_params to the value from the executor.  This sets link_params to True, even though the current target is CPU.
Line 1057: If the current config has a key "relay.FuseOps.link_params", set link_params to that value.  Otherwise, use the value from line 1054.  Here the config is empty, since FoldConstants creates an empty one, so the default value is used (i.e. the one from 1054).

I think you mentioned that link_params should be a property of the target.  If fuse_ops got it from the target, everything would work fine, since it would get it from "CPU" instead of "hexagon".  I think that would be the correct behavior.

Hmmm, Just to understand this a bit better, what is the target string being used in your case ?
Secondly, does the hexagon backend requires the constants binded to the IR, eventually to be codegen'd or does it require them to be provided at the runtime ?

The first failing testcase is resnet50:
This is the script that reproduces the crash is:
import coremltools
import onnx
import onnxmltools
import tvm
import tvm.contrib.hexagon
from tvm import relay
from tvm.relay.transform import InferType, ToMixedPrecision

dtype_dict = {"data": "float32"}
shape_dict = {"data": [1,3,224,224]}

#input name and path for your caffe model
proto_file = './ResNet-50-deploy.prototxt'
input_caffe_path = './ResNet-50-model.caffemodel'

# Convert Caffe model to CoreML
coreml_model = coremltools.converters.caffe.convert((input_caffe_path, proto_file))
# Convert the Core ML model into ONNX
onnx_model = onnxmltools.convert_coreml(coreml_model)
onnxmltools.utils.save_model(onnx_model, 'resnet-50.onnx')
mod, params = relay.frontend.from_onnx(onnx_model, shape_dict)
mod = InferType()(mod)
mod = ToMixedPrecision("float16")(mod)

target = tvm.target.hexagon("v68", link_params=True)
config = {"relay.FuseOps.link_params":0}
with tvm.transform.PassContext(opt_level=3, config=config):
    lib = relay.build(mod, target, target_host=target, params=params, mod_name="default")

At the bottom of the crash dump you should see
  0: tvm::relay::tec::ScheduleBuilder::VisitExpr_(tvm::relay::ConstantNode const*)::{lambda(tvm::runtime::Array<tvm::tir::Var, void> const&)#1}::operator()(tvm::runtime::Array<tvm::tir::Var, void> const&) const
  File "/w/src/dmlc/tvm/src/relay/backend/te_compiler_cache.cc", line 242
TVMError: float16 not handled

The target string is hexagon -keys=hexagon -link-params=1 -mattr=+hvxv68,+hvx-length128b -mcpu=hexagonv68 -mtriple=hexagon.
We don't yet have codegen for AllocateConst specific to Hexagon, but if the LLVM codegen handles it, it will probably work for us as well.  Right now we use the link_params function, which contains all the constants in it, but is called at runtime to supply them to the model.

config = {"relay.FuseOps.link_params":0}

https://github.com/apache/tvm/blob/d9fac4f305c5462de4ab08cd220d60b42c0414f7/tests/python/relay/test_pass_fuse_ops.py#L631-L661
I think this is respected. maybe we should need to verify when Executor has link-params ?
But after having read this all again -- the actual issue link-params will be set to True if the Executor has it. It seems like hexagon backend was reliant on FoldConstant behaviour of always having link-params set to False. Is it not ?
If so the solution might be to provide link-params override for FoldConstants as well ?

Can ? / should this discussion please be moved into a proper issue ?
Ramana

Yes I think it make sense -- @kparzysz-quic lets create an issue and move it there.

@manupa-arm and @junrushao1994 can you tag me on changes to CI scripts in the future? The changes here in tests/scripts/ and tests/lint/python_format.sh broke linting CI on main in a roundabout way

Oops sorry about this! Will do
Attn : @d-smirnov

thanks! btw, it's fixed in #10469 so no action here is necessary


Recognition of the old genus Physematium (Woodsiaceae)

Author(s) of proposal
Li-Bing Zhang, Ngan Thi Lu
Name of taxon
Physematium
Rank of taxon
Genus
Approximate number of species affected
28
Description of change
Reinstatement of Physematium (1829) in Woodsiaceae to reflect the deep divergence in the phylogeny, morphological diagnosability, and geographical differentiation.
Reason for change
Woodsiaceae contain ca. 65 (instead of 30 as early estimated) species that are widely distributed in the north temperate zone and in higher elevations in the tropics, e.g., Central and South America, Africa (Angola, South Africa), and Madagascar.
Two contemporary classifications of Woodsiaceae recognized dramatically different numbers of genera. Shmakov (2015) recognized seven genera: Cheilanthopsis, Eriosoriopsis, Hymenocystis, Physematium, Protowoodsia, Woodsia, and Woodsiopsis. Recognition of Hymenocystis and Protowoodsia would make Cheilanthopsis paraphyletic. Shao et al. (2015) advocated for a single-genus classification with three subgenera: W. subg. Woodsia, W. subg. Cheilanthopsis, and W. subg. Physematium. Their W. subg. Physematium, including species of both Physematium and Woodsiopsis, is paraphyletic based on Lu et al. (2019) tree. Shao et al. (2015) did not sample any taxa of the old name, Hymenocystis. PPG I (2016) followed Shao et al.’s (2015) classification.
Based on a new phylogeny (tree below) using a largely expanded taxon sampling (122 samples representing ca. 43 species vs. 33 samples representing. 21 spp. in Shao et al. (2015)), detailed analyses of character evolution (figure below), ancient deep divergence (ca. 45 million years ago; Larsson, 2014), and geographical distribution, Lu et al. (2019) advocated a two-genus classification: Woodsia (1810) and Physematium (1829), corresponding to the two subfamilies proposed by Shmakov (2015). Geographically, ca. 37 species of Woodsia are distributed in circumboreal areas and East Asia only, while ca. 28 species of Physematium occur in the Americas, Asia, and Africa and Madagascar. Also, species of Woodsia have x = 39 or 41, while those of Physematium have x = 33, 37, 38, or 41.

Reference(s) for publication of the name
Shmakov 2015_Woodsiaceae.pdf
Shao et al 2015_Woodsiaceae.pdf
Lu et al. 2019_Woodsia.pdf
List the numbers of any related issues
No response
Code of Conduct

[X] I agree to follow the PPG Code of Conduct

Hello Li-Bing,
PPG I said it sought “to preserve existing taxa and circumscriptions that are widely accepted and consistent with our understanding of lycophyte and fern phylogeny” (p. 565).
PPG I recognised a broadly circumscribed Woodsia (with one genus in the family).  Your work and that of others shows that Woodsia sensu PPG I is monophyletic.
I believe my experience that general users of taxonomic names dislike name changes is not unusual.  Many taxonomists recognise the desirability of trying to meet the wants of general users for stability whilst constructing a taxonomic classification based on monophyletic groups (e.g., Drew et al., 2017, Taxon 66: 133-145).  This might equate to ‘be monophyletic, and make the fewest name changes to get there’.
As far as I know, PPG does not have standardised criteria for how old a genus or family should be.
Your work and that of others has greatly extended the understanding of relationships within Woodsiaceae.  But that could alternatively be represented taxonomically using infra-generic ranks, which would have the advantage of general users not having to incur name changes.
In summary, Woodsia sensu PPG I is consistent with phylogeny and if PPG wants “to preserve existing taxa and circumscriptions” as much as possible, why make a change?
More generally, if we split a monophyletic group into multiple monophyletic sub-groups (or lump multiple monophyletic groups), then when does taxonomic revision ever end?

Hi Leon, thanks for your comments. You made good point! Yes, you can use subgenera or sections for such monophyletic clades with the family. On the other hand, those infrageneric ranks are almost always ignored except by professional plant taxonomists or even only by specialists of a particular plant group.
Placing all the members of the family in one genus is convenient, but adopting two-genus classification informs science (phylogeny) better and emphasizes closer relationships within each of Physematium and Woodsia s.n. or more distant relationships between species in two genera. These two genera are deeply diverged (ca. 45 million years ago) and morphologically diagnosable anyway. Also, Physematium is an old name.
I am sure when Polypodium s.l. or Lycopodium s.l. was split decades ago, there were botanists who were against the splitting using the similar argument you mentioned (even now Kew's Plants of the World Online recognizes one genus only in Grammitidoideae). Think how many genera within each of them PPG I recognized! Similarly, future generations might find the two-genus classification of Woodsiaceae more informative and useful.  Convenience alone is not enough. A better science-informing classification will survive in the end.
Best wishes, Libing
Li-Bing Zhang, Dr. nat., Curator; Missouri Botanical Garden
www.mobot.org/Li-Bing Zhang

Thanks Li-Bing.  If only there was a way to objectively determine the most informative (for science or otherwise) classification among alternative monophyletic-based options.  But there doesn’t seem to be.  Even morphological diagnosability is open to interpretation.
Which seems to leave the choice as subjective, which is often convenience in some form or other.
In which case, should convenience for a few taxonomists outweigh the convenience of the many more general users?  I’m mindful that it’s the latter, through their taxes, paying the salaries of those of us fortunate to be professional taxonomists.

Thanks, Leon! A new classification is inconvenient for everybody who is familiar with the existing one. When inconvenience/laziness is not the case for future pteridologists (beyond the PPG II classification) or outside of the community of pteridology, will they choose a better science-informing classification or a conservative and convenient/lazy classification?
In that sense, the more aligned the PPG II classification is with science, the better the chance it will be appreciated.

This proposal was voted on during PPG Ballot 10 (voting period April 2024). A total of 65 votes were cast. There were 30 'Yes' votes (46.2%) and 35 'No' votes (53.8%). The proposal does not pass.


fix missing cluster check

Changes proposed in this PR

don't ignore static clusters when checking that there are no missing clusters for known running tasks

Why are we making these changes?
current code will incorrectly raise an error when an existing task is running on a static cluster

Will merge when green


add something in diffusion model

Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How has this been tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[x] local build the book
[ ] review the artifacts from local build
[ ] local build the slides if appliable
[ ] Unit test if appliable

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have applied tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

There are too many files being pushed that are not part of the changes, make sure you pull the latest branch, upload it again and only upload the changes!


Blue Jade - Add modifier to the game

Ok, I made a mistake. Blue Jade is not that easy to create, because of Spiky Bracers. Essentially, the way the relic is now, or you begin with two appliable cards, or it's dead weight.
...I will have to change how the relic work for this to do what it should.

Right, after (#134), now we can add the relic back. It still needs a new description, but that's not a problem.


timeout errors on Scaleway

On Scaleway we face some timeout issues and it is required to apply twice the configuration to finish the procces

timeout errors appear when mounting a volume. Indeed, when attaching a volume the instance is stopped. Need to wait the instance is back.

As a workaround, it is needed to apply twice the configuration and it works.

The Scaleway provider has changed and upgrading to the 1.16.0 version (https://github.com/kalisio/kaabah/issues/153) will fix the issue.


Extension issue

Issue Type: Bug
Extension Name: html-css-class-completion
Extension Version: 1.20.0
OS Version: Windows_NT x64 10.0.19042
VS Code version: 1.58.0

:warning: We have written the needed data into your clipboard. Please paste! :warning:

Please provide more information.


vimPlugins: update

Motivation for this change
Regular update.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

couldn't this be run as a daliy job instead ?

It could, if someone would build the infrastructure and review the PRs daily. Currently the "infrastructure" is a weekly recurring entry in my todo list.


Updates for aztables metadata

Include returned metadata in more APIs.
Make the amount of returned metadata configurable.
Fixes https://github.com/Azure/azure-sdk-for-go/issues/22438

/azp run go - aztables


Enter doesn't do anything on mobile

Using Oneplus Nord n30 5g with android 13
Brave Browser 1.52.130, Chromium 114.0.5735.198
I've also seen someone else on reddit with the same issue (Android 14, stock Pixel 6 Pro. Firefox 120.1.0 with uBlock Origin installed.
same on Chrome 119.0.6045.194)
How to recreate:

Open website
Type command
Press enter

What I expect to happen:
the command executes
What happens instead:
nothing, the website doesn't respond
I'm happy to provide any more info if needed. great project!

Hey do you have any idea on how I could test this if I don't have an Android device?
Thanks again for the feedback!

you can use an Android emulation app, such as Bluestacks or WSA if on Windows, or Waydroid if on linux.


bootsnap: various improvements.

Add HOMEBREW_NO_BOOTSNAP as well as HOMEBREW_BOOTSNAP
Use HOMEBREW_CACHE instead of HOMEBREW_TEMP for bootsnap files
Cleanup bootsnap files with brew cleanup and periodic runs
Vendor bootsnap and msgpack gems
Fix bootsnap failures on Utils.safe_fork e.g. on brew postinstall and brew test CC @alebcay
Document HOMEBREW_{,NO_}BOOTSNAP variables so we can announce them in Homebrew 3.0.0

Marking as critical to get this landed for testing early so we can release 3.0.0 ASAP (with a few other pending PRs).

Review period skipped due to critical label.

Note to self: need to remove bootsnap.bundle from the committed files, here.


Upgrade Scala 3.x versions

Change to

Scala 2.3.10 due to CVE - https://www.cvedetails.com/cve/CVE-2022-36944/
Scala 3.2.0 -- Scala team recommend using latest Scala 3 version - https://www.scala-lang.org/blog/2022/08/17/long-term-compatibility-plans.html
Scala 3.3.0 - the first LTS version of Scala 3 should be out soon

Bumping Scala 3's minor version isn't necessarily a good thing, some care is needed.
If a project is using Scala 3.1.3, bumping the Pekko dependency to a new version that uses Scala 3.2.0, also forces the project on Scala 3.2.0 — backwards compatibility is provided, but not forward compatibility. In that sense, Scala 3 upgrades behave like libraries. This is unlike upgrading from Scala 2.3.9 to Scala 2.3.10.
And care is needed because a compiler is unlike a library, as it may have bugs that don't manifest in the Pekko library's codebase, but that impacts downstream projects.
I think right now an upgrade to Scala 3.2.0 is totally fine, given the state of the Scala 3 adoption, and that of Pekko, but for the future, it's good to keep in mind that such upgrades basically force the new Scala version on downstream users, and that may not be good for Scala versions that have issues.

@alexandru thanks for the comments.
Picking out comments from the article linked above, just in case.

You shouldn’t be afraid of updating it. Bumping the patch version (e.g. 3.1.2 -> 3.1.3) is a no-brainer. Bumping the minor version (e.g. 3.1.3 -> 3.2.0) requires the same considerations as changing the minor version of any other dependency, yet is generally advisable.

So, upgrading to 3.2.0 is not zero risk but it pretty low on the scale of these things.
I'm not going to do anything with this issue yet and it is probably best to discuss this item on the dev mailing list before proceeding to the PR stage.

I think the CVE may work for a Pekko release of 1.0.x  pending on what we ultimately decide goes into 1.0.x (since its a CVE) but 3.2.0 makes sense for 1.1.x

@pjfanning

upgrading to 3.2.0 is not zero risk but it pretty low on the scale of these things.

I kind of disagree with their blog post, as it's not low risk, because new compiler versions can introduce source-level regressions. The difference between a library and the compiler, is that libraries are dynamically linked, at runtime. For this reason, a tranzitive dependency cannot break your source code, but compiler update can.
For example, here's an issue that broke my code when upgrading from 3.0.x to 3.1.2, and it was close to be considered a feature: https://github.com/lampepfl/dotty/issues/15160
Downstream users may be able to fix such instances, but if an upgrade breaks source compatibility, then in my view that's a major version, not a minor one (according to semver).

Again, 3.2.0 is fine for now, all I'm saying is that we can't be very relaxed about upgrading these versions. I would feel better if we let new versions stabilize a little.
And I also don't think we need further discussions, it's a simple update IMO, and for now the Scala 3 adoption of Akka is probably very low anyway, so might as well update it now, rather than later. But it's something to keep in mind.

I would argue against using 3.2.0 as it forces downstream users/projects to also upgrade to 3.2.0.
Last time I checked, Scala 3.1 couldn't read TASTy files of 3.2 and failed to compile using that library.

For now Scala 3 adoption is rather low (among other reasons - Akka HTTP is not released for any version of Scala 3, and apparently it uses Scala 2 macros), so these updates wouldn't affect many people. Surely none that use Akka HTTP and would like to migrate to Pekko.
According to the blog post - by the end of the year the compiler team want to release the first LTS version of Scala 3 (3.3). So the current situation is rather a temporary flux period before LTS kicks in. If 3.3 would be supported for at least 2 years as claimed, it would make sense to release Pekko for 3.3, stay on it until next LTS is adopted and then move on to next LTS.
IMHO, moving fast for now make sense. For many libraries blocker to update are macros and I notices that some important features stopped being @experimental only in 3.2.0, which would block importing certain functionalities before this version. And these libraries might slow down adoption for whole community, so I would expect for it to take off around the time that the compiler team want to release LTS, or even later.
Once LTS program goes live this whole issue with upgrading should be solved. Pekko can just use the current LTS and maybe wait a bit when new LTS is released. That should give community enough time to bump their versions.

Many things like IDE support are still very lacky for Scala 3, but we also don’t gain much by going to the newest Scala version. As long as Pekko supports Scala 2.13, we’re very limited in what we can use without having to resort to splitting source files by scala version.

Thanks @MateuszKubuszok for the detailed status update. akka-http is not yet released with Scala 3 support but the core akka libs do support Scala 3.
After I do some experimentation, I might look at adding Pekko to the Dotty Community Build.

According to the blog post - by the end of the year the compiler team want to release the first LTS version of Scala 3 (3.3). So the current situation is rather a temporary flux period before LTS kicks in. If 3.3 would be supported for at least 2 years as claimed, it would make sense to release Pekko for 3.3, stay on it until next LTS is adopted and then move on to next LTS.

If this is the case then at least to me it seems like doing the first release with Scala 3.3 seems like the best option? In terms of both it being an LTS and maybe in terms of timing (i.e. by the time pekko-http is ready to be released). I wasn't aware that Scala 3 basically has different stability guarantees, it sounds like there is a risk that we may end up having to having a matrix if we don't pick a good Scala3 version and a lot of people end up depending on it?

Scala 3.1.3 would be the safest bet for now. Anyone on Scala 3.1 or 3.2 could use it. As long as we cross compile for Scala 2.13 we can't use much Scala 3 specific features anyway.

I wasn't aware that Scala 3 basically has different stability guarantees, it sounds like there is a risk that we may end up having to having a matrix if we don't pick a good Scala3 version and a lot of people end up depending on it?

Scala 2 used epoch.major.minor version convention. Scala 3 has major.minor.patch.
So there is no 3.0/3.1/3.2/etc cross-compilation - the assumption is that:

you can compile against the same minor version with backward- and forward-compatibility: 3.1.3 dependency against 3.1.0 code, 3.0.0 dependency against 3.0.1 code, etc
within the same major version you always have backward-compatibility: 3.1.3 dependency can be used in 3.1.3 project, but also 3.2.0 project and in future against 3.3.0 project

It means that you should use the lowest version that you can get away with to not force people to update their build unnecessarily. If it would be e.g. 3.1.3 then there is a case to use 3.1.3 for you project and not update it.
At the same time, before 3.3 there is no LTS version planned. Meaning that if there are bugs, regressions, security vulnerabilities - they won't be backported to versions prior to 3.3, once 3.3 is released. If these issues appear in code compiled with e.g. 3.1.3 when 3.3 will be already released, there won't be any new release for 3.1 (or 3.2) fixing them. It all mirrors the JVM approach where you have LTS for Java 11 and Java 17 but intermediate versions are EOLed rather fast, and while you are encouraged to test with them, you are not given any long term support guarantees for them.
So while thing will be much clearer after the release of 3.3, right now we are living in the period of versions that can be used for any future development but will be supported for at best next few months.
Akka core might get away with 3.1.3 or earlier.
Akka HTTP from what I understand relies on parboiled2 for its macros - 2.4.1 was compiled with 3.2.1 and 2.4.0 was compiled with 3.1.1, so it might require some investigation.

@nvollmar

Many things like IDE support are still very lacky for Scala 3, but we also don’t gain much by going to the newest Scala version. As long as Pekko supports Scala 2.13, we’re very limited in what we can use without having to resort to splitting source files by scala version.

Just to set some expectations, in my experience, it is impossible to target both Scala 2.13 and Scala 3 without resorting to source files specific per version. And the setup can get quite complicated. Which is fine, at least we can. Only the simplest of libraries can get away with only shared code that compiles in both.
@MateuszKubuszok

Akka HTTP from what I understand relies on parboiled2 for its macros - 2.4.1 was compiled with 3.2.1 and 2.4.0 was compiled with 3.1.1, so it might require some investigation.

I was just thinking of this — we might not have a choice, but update, alongside our dependencies. For example, akka-persistence depends on jackson-module-scala, and the latest version also depends on Scala 3.2.0.
I think this idea is good — try to depend on the 3.3 LTS, when it gets released, and until then, depend on intermediary versions 👍

On Fri, Nov 4, 2022 at 9:21 AM Alexandru Nedelcu @.***>
wrote:

@MateuszKubuszok https://github.com/MateuszKubuszok
Akka HTTP from what I understand relies on parboiled2 for its macros -
2.4.1 was compiled with 3.2.1 and 2.4.0 was compiled with 3.1.1, so it
might require some investigation.
I was just thinking of this — we might not have a choice, but update,
alongside our dependencies. For example, akka-persistence depends on
jackson-module-scala https://github.com/FasterXML/jackson-module-scala,
and the latest version also depends on Scala 3.2.0.
I think this idea is good — try to depend on the 3.3 LTS, when it gets
released, and until then, depend on intermediary versions 👍

This is a good point. Another point is that Akka's Scala 3 support has so
far been experimental and incomplete so expectations on compatibility are
low. Until we make the first release, I think we should grant ourselves the
flexibility of tracking the latest version of Scala 3. Not least, because
Scala 3 has been somewhat unstable and we hit many regressions during the
akka-http migration so we might have to take whatever works...
Incidentally, Akka HTTP right now contains a copy of parboiled2 in its
sources (with some slight additions). This decision was originally made on
the grounds that Akka should not have many dependencies on Scala libraries,
so it can be released quickly after a new (incompatible) Scala versions
comes up. This point does not seem as important as before, so I'd suggest
to move upstream parboiled2 when possible.
I created https://github.com/apache/incubator-pekko-http/issues/1 to track
this.

Message ID: @.***>

So a PR is open that updates Scala 2.12 and 2.13 versions (see https://github.com/apache/incubator-pekko/pull/258). Its still open on what to do with Scala 3 , .e. should we just target Scala 3.3 LTS which is going to come out at any point now or Scala 3.2?

@pjfanning I am thinking of closing this ticket and making a specific one for Scala 3 updates since its a completely different kettle of fish with its own set of problems, wdyt?

So far, Scala 3.1 seems to be the preference. This issue has not been included in the v1.0.0 milestone.

So far, Scala 3.1 seems to be the preference. This issue has not been included in the v1.0.0 milestone.

There hasn't been a formal vote but from what I can see there doesn't seem to be a clear direction either way. I spoke about this with @jrudolph and he summarised this conundrum when using Scala 3.3 in a very succinct way,

It's a promise for the future not for the past

My argument here is that when it comes to Scala 3, Akka doesn't really have a "past". By that I mean I seriously doubt that there is any significant user base that is actually using Scala 3 (and we can even quantify this with download stats). I strongly agree with what https://github.com/apache/incubator-pekko/issues/6#issuecomment-1302472832 said here, there isn't any real point in sticking with an old version of Scala 3 if barely anyone is using Akka + Scala 3.

Made a discussion on mailing lis https://lists.apache.org/thread/mk88ybzx51k7x7x102drtbzh6l68k0xm.
I do agree that wait and see is a mature stance for now.

As far as I can tell from talking to the compiler team developers, Scala 3.3 will not introduce any feature that was absent in 3.2. Some APIs marked as experimental will become stable and that's it - so 3.3 will be a stabilization of the current API, but with a guarantee that it will not be touched for a few years.
From what I understand 3.3 is relatively near, so it makes sense to target it. Versions older than that like 3.1, or 3.0... they were basically a public beta, named otherwise to encourage any adoption and feedback loop. I don't see any point in using them, since they offer no:

bugfixes
stability
any other form of support

especially if Pekko intends to be more on a stable-side than bleeding edge-side of development.
I'd say of you'll pick LTS you could always ping compiler team members for advice whereas any issue with non-LTS Scala 3 could be shrug off as "EOL, not supported anymore".
CC @Kordyjan - Scala 3 compiler team lead at Virtus Lab.

As far as I can tell from talking to the compiler team developers, Scala 3.3 will not introduce any feature that was absent in 3.2. Some APIs marked as experimental will become stable and that's it - so 3.3 will be a stabilization of the current API, but with a guarantee that it will not be touched for a few years.

Judging from https://github.com/apache/incubator-pekko/pull/273 where we were forced to update to Scala 3.2 due to Jackson only targetting Scala 3.2, I can confirm that most (if not all?) of the changes were put into Scala 3.2, not 3.3 (I already made a PR against Scala 3.3 at https://github.com/apache/incubator-pekko/pull/270 just to see how painful it is, and it seems really tame).

https://contributors.scala-lang.org/t/stable-presentation-compiler-api/6139
Some support only start with 3.3.0

Some thoughts after reading the thread.
Source compatibility and minor version
First, I wanted to state that source compatibility has nothing to do with adding new features or naming a release as "minor". This may be counterintuitive, but upgrading from 3.2.1 to 3.2.2 has the same probability of breaking your code as upgrading from 3.2.2 to 3.3.0. And contrary to what many believe, that probability is small.
Let me show you where source incompatibilities come from to see that better. I can think of three distinct sources of such incompatibilities:

Changes to the type inference. This is the most common reason for problems. The entire inference system is interconnected, and changing it to work as expected in one example can cause other, more complex examples to start behaving unexpectedly. This is made worse by the fact that there may be no distinction between what is expected and what is not for complex cases, as different people can expect different types to be inferred. The good thing is, usually, to fix source incompatibility there, you only need to add explicit type ascription.
Bugfixes. Sometimes the compiler has bugs causing an acceptance of incorrect code. For example, inline parameters are not stable references, but all compiler versions up to 3.2.2 were, by mistake, treating them as such. Many libraries were relying on that fact, and from their point of view, 3.3.0 will introduce a new source incompatibility, even though it will only prevent them from writing potentially unsound code. We do what we can to be clear about those changes in the release announcements and to offer possible workarounds.
Intentional changes. I can only recall a single case of that happening since 3.0.0: this PR.  It solved one infuriating source of problems, changing the meaning of a very specific, niche code. Before we do anything like that again, we will ensure that it has a negligible impact on open-source projects.
This "checking of impact" takes the form of building the entire Scala 3 open-source ecosystem with our Open Community Build. You can read more about it here and there. Thanks to that, we will know about any real-life source incompatibility in advance and, when possible, take steps to ensure that it will not be included in a stable release.

I don't believe there is a reason to fear source-breaking changes now, as they are much less frequent than in the past and communicated in advance. Also, they can happen both in a patch and in minor releases.
The language version as a dependency
The notion that upgrading the compiler version is much more dangerous and unpredictable than upgrading the version of the library your project depends on stems from the fact that Scala 2 was not following semantic versioning. This is no longer the case.
Moreover, there is an argument that bugs in the compiler may not manifest in your project but may be visible in downstream projects. While this is true, it is precisely the same as with the library you and your downstream projects depend on. It can introduce bugs that are not visible to you, but people depending on your library can start to have problems because you upgraded your dependency. The eviction mechanism kicks in, and now they unknowingly use a newer (bugged) version of a dependency. In the case of the compiler upgrade, there is no eviction, so every change needs to be made explicitly. This means that bumping the minor version of the compiler is even safer than bumping the minor version of any other library.
Older versions and stability
Sometimes, I see claims that older versions can be more dependable because they are more mature. This could be a case if they received any updates. Bugs found in 3.1.3 will never be fixed in any future 3.1 release because there will be no future 3.1 releases. Some of them were, however, fixed in 3.2.0. There was no significant redesign or rewrite between 3.1.3 and 3.2.0. This version could have been named 3.1.4 if it hadn't introduced any forward-incompatible (but still backward-compatible) changes. This means that 3.2.0 was more mature than 3.1.3, even though Scala 3.1 was there for a year, and Scala 3.2 was a new thing.
In terms of stability and correctness, the newer version of the compiler will always be better than the older one. Of course, we need to keep in mind that we may change the minimum required version of the compiler for downstream projects, but this is the same consideration as with any other dependency.
What about LTS?
The LTS line (that is 3.3) will be different than previous versions. Even though we will release 3.4 soon, 3.3 will still receive backward and forward-compatible patch releases for at least 3 years, possibly longer. Of course, we will keep our guarantee that all projects compiled with any version of 3.3 can be used as a dependency for projects compiled with any version of the compiler newer than and including 3.3.0.
This is why I think the newest 3.3 version at the moment of release would be the best compiler version for Pekko. For the time being, I encourage you to use 3.3.0-RC3 for development and, later, 3.3.0-RC4 - which comes out next week.
3.3.0-RC3 is stable. lichess.org is even using it in production. We need another RC because the linter still has some unsolved problems. Even though we have been pushing this release a few times already, we are confident that the stable 3.3.0 will be out before the end of April.

Thanks for taking the time to write this comprehensive response, @Kordyjan!
A few comments:

I don't believe there is a reason to fear source-breaking changes now, as they are much less frequent than in the past

That's probably true especially in the future. In the past, there were many (mostly minor) source compatibility issues between Scala 3 versions, so I am somewhat careful about these statements.

In the case of the compiler upgrade, there is no eviction, so every change needs to be made explicitly.

You can see that both ways. No eviction means, a user of a library compiled with a newer version will instead get compiler errors like these:
class file ... is broken, reading aborted with class dotty.tools.tasty.UnpickleException
[error] TASTy signature has wrong version.
[error]  expected: {majorVersion: 28, minorVersion: 1}
[error]  found   : {majorVersion: 28, minorVersion: 2}
[error]
[error] This TASTy file was produced by a more recent, forwards incompatible release.
[error] To read this TASTy file, please upgrade your tooling.
[error] The TASTy file was produced by Scala 3.2.1.

That's not necessarily better than eviction, especially when you have multiple dependencies built with different versions of the Scala compiler and then need to figure out where to go based on those error messages.

The LTS line (that is 3.3)

I would imagine that all these problems mostly go away when everyone adopts 3.3.x. We will see if the situation will be vastly better than with Scala 2. At least you will be able to use "old" 3.3 artifacts when you start to adopt 3.4 for any of the new features. Realistically, library authors will have to stay on 3.3 for as long as possible.
(Besides, note, how the compatibility benefits of the LTS version are not a technical feature but a feature of the promise of stability and its execution.)
Supporting 3.4 or later features in parts of your codebase will be somewhat hard. We will see how that will play out when the ecosystem has to move to the next LTS version after 3.3. With the current cross-versioning scheme (_3) that will mix partly incompatible versions into the same namespace, I can foresee some issues... Fortunately, that will be a long time away.

lichess.org is even using it in production. We need another RC because the linter still has some unsolved problems.

It's crucial to understand that an application has vastly different requirements than a library. You can always run the test suite for your application to ensure that your application runs fine on a new version. Library authors cannot completely predict how their library is used and what constraints and problems the users will have. Therefore, the hard-learned conclusion is to be very conservative about adopting new versions of any transitive dependency and not pull transitive dependencies to higher versions than necessary for minor reasons.
In summary, I agree 3.3 LTS is great and we should definitely make sure that our binaries work with it and eventually move to it ourselves.

That's not necessarily better than eviction, especially when you have multiple dependencies built with different versions of the Scala compiler and then need to figure out where to go based on those error messages.

I agree that the error message needs some improvements. It should only contain a path to the offending file, the classpath entry from which the file comes, the current version of the compiler, and the version of the compiler that was used to produce the offending file. The details about tasty signatures should be hidden behind the --explain flag. I'm, of course, open to any suggestion on how it can be further improved.

It's crucial to understand that an application has vastly different requirements than a library.

I wasn't clear, but I intended to suggest that if Pekko is to be released on 3.3.0, it is a good idea to develop it using 3.3.0-RCx. That would allow catching potential problems early when there is still some possibility they are fixed before the stable release. The lichess bit was only a curiosity, showing that 3.3.0 can be trusted and there is nothing inherently wrong with it (some may think that way seeing how much work it takes to prepare RC4).

Fortunately, that will be a long time away.

It definitely will be long time away. 3.3 LTS will receive patch updates for at least a year after the release of the next LTS. We will see how hard the maintenance would be, but I'm optimistic and think that the guaranteed year is an understatement.

@Kordyjan

This is why I think the newest 3.3 version at the moment of release would be the best compiler version for Pekko. For the time being, I encourage you to use 3.3.0-RC3 for development and, later, 3.3.0-RC4 - which comes out next week.

Incase you may have missed this, we already have a version for Pekko 3.3.0-RC3 so far our CI hasn't picked up anything (you can see it here https://github.com/apache/incubator-pekko/pull/270)

Closing this as done


Redirect PPro

Task/Issue URL: https://app.asana.com/0/0/1206957384492835/f
Description
See task
Steps to test this PR
See task

#4381  👈
develop

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @marcosholgado and the rest of your teammates on  Graphite


#68 multilang markup

markdown pages language tagging
markdown pages need some french lang attribute so that they're being read properly.
Related issues / Requêtes associées
closes #68

huzzah


fix adding nil if hostname isn't set up first

This one small block of code had caused me more headache then anything else so I wanted to send a fix.
In my chef runs the hostname may not be setup prior to the first run so I would always get a compile error if cerner_kafka was in my run_list with:
undefined method `+' for nil:NilClass

Then the entire run would die and it would never setup the host.  by moving the brokerId increment after the check and using a return I can now bail out of the cerner_kafka cookbook on my first run and allow the host to be setup and kafka can get setup on a consecutive run.  I also added a bit better message to output so I could see what each item was set to and what it was looking for.
A better way may be to do a lazy evaluation of the broker ID prior to writing the template.

I tried with errors.push but then the next line where we increment the brokerId errors with the undefined method `+' for nil:NilClass message.  Using the return logs the message but continues the chef run without processing the rest of the recipe.  This allows my other recipes to configure the hostname and default setup and kafka will work on the next chef-client run.
Like I mentioned it can probably be done better by moving all the logic to the template generation and use lazy loading for the variables.  This would allow the rest of the chef run to complete and still configure kafka in the first path.
I'll dig a little deeper and see what's required to do this.

Ugh duh should learn to read better. If we don't assign a broker id I'm not sure if kafka will start.
Your work around of not running the recipe seems reasonable but conflicts with someone mis-configuring their node and us not being able to come up with an id. It does seem to log a message about it but that assumes someone is watching and possibly watching closely.
If you really need to run chef twice in this instance would it be worth trying to use an override run list the first time?

A single pass client run is of course ideal.  I'd really just like for my host setup stuff to run before failing the chef run.  As it was it wouldn't even compile the run_list unless the servers hostname was in the brokers list.  If I could get the rest of the chef run to run first then check if the brokerId can be set and bail on error at that point I think it would work out better overall.  At least if the kafka recipe is run last the host should be in a mostly configured state and ready for a second run after fixing the settings.
I'm trying to set this up to automatically deploy with chef-provisioning in a completely hands off manner, and allow for semi-auto up scaling if needed.

re-arranged a few things.  I still have it return on any errors so the run will complete, but it pushes the errors like it did before.
I also removed some unneeded checks and cast the brokers to an array prior to checking it.  This would allow you to specify a single broker as a string.

Can we make the behavior to ignore errors and continue an attribute and by default keep the behavior as it was before?

Should have asked this earlier but if we just make the check run during converge instead of compile will that work for you? We should just be able to wrap that ruby code in a ruby_block.

I had the same idea and tried just wrapping it in a ruby_block and it didn't seem to update the node object.  I wasn't sure if i was doing it right so I backed that out and made this change.  having a continue_on_errorr attribute default to false would be fine as well.  Although if it could be moved to converge phase I think that wold be the 'chef' way to do it.

Ok, i just retried this and tested it in kitchen.  I'll update in a second if you want to take a look.  When I unset broker.id in .kitchen.yml I got:
 Recipe: cerner_kafka::default

           ================================================================================
           Error executing action `run` on resource 'ruby_block[assert broker and zookeeper lists are correct]'
           ================================================================================

           RuntimeError
           ------------
           Unable to run kafka::default :
            -node[:kafka][:brokers] or node[:kafka][:server.properties][:broker.id] must be set properly]

           Cookbook Trace:
           ---------------
           /tmp/kitchen/cookbooks/cerner_kafka/recipes/default.rb:36:in `block (2 levels) in from_file'

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cookbooks/cerner_kafka/recipes/default.rb

         9: ruby_block 'assert broker and zookeeper lists are correct' do
...

if I set kafka.brokers to [test1,test2] I get:
       Error executing action `run` on resource 'ruby_block[assert broker and zookeeper lists are correct]'
           ================================================================================

           RuntimeError
           ------------
       Unable to run kafka::default :
            -Unable to find kafka, <IP_ADDRESS> or kafka in node[:kafka][:brokers] : ["test1", "test2"]]

which should be a little more helpful of an error message because you'll see what it's actually looking for if your hostname isn't in the array.
adding kafka (the hostname set in kitchen) to the list it runs as normal with the brokerId based on the array index.
I think this should cover everyone's expectations, am I right?

working on the rspec tests now

I think chefspec is overriding the code in the ruby_block and not allowing it to execute.  Makes sense because if you had something in there to modify the host it would do it to the host running the tests.  I'm not sure how to test it other than with an assertion that the ruby_block is there...

I think I'm ok with us not verifying that behavior anymore in chefspec. If we are really worried about we can move the code to a library and test that.

ok, i was able to get all the current tests to run with chefdk 0.5.1  lets see if the work in travis under the older version of chefspec.

something's not right.  The block is running but I don't think it's updating the node object.

OK, i found it. because the assignment is happening now at converge time the empty array is being assigned to the template resource for zookeepers and brokerid.
Ok, not sure I can get to that today.  let me see how some of these other deployments go.

I'm fine with ignoring foodcritic for now about this. You can fix it by changing this line, https://github.com/cerner/cerner_kafka/blob/master/Rakefile#L29

I added the comment way to disable foodcritic warnings, that allows it to stay visible and works as a reminder that that section of code isn't as awesome as it could be.

I can't remember where we left off but this looks good to me if you are fine with it

I'm using it with chef provisioning to deploy kafka to aws and it's working well.

@noslowerdna or @mkwhitacre can you take a look at this?

+1

@TD-4242 Thanks for the pull!

@TD-4242 do you mind opening a pull request to add yourself as a contributor.


python311Packages.georss-generic-client: 0.7 -> 0.8

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python311Packages.georss-generic-client is: Python library for accessing generic GeoRSS feeds
meta.homepage for python311Packages.georss-generic-client is: https://github.com/exxamalte/python-georss-generic-client
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.8 with grep in /nix/store/2myrdl72cldb9ni307x5n3g3ljlfvqx8-python3.11-georss-generic-client-0.8
found 0.8 in filename of file in /nix/store/2myrdl72cldb9ni307x5n3g3ljlfvqx8-python3.11-georss-generic-client-0.8

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath
home-assistant-component-tests.geo_rss_events
python311Packages.georss-generic-client

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/2myrdl72cldb9ni307x5n3g3ljlfvqx8-python3.11-georss-generic-client-0.8 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python311Packages.georss-generic-client https://github.com/r-ryantm/nixpkgs/archive/af1d2c64cc202d9064a8c1116d7dbb8c3eb24260.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/af1d2c64cc202d9064a8c1116d7dbb8c3eb24260#python311Packages.georss-generic-client

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/2myrdl72cldb9ni307x5n3g3ljlfvqx8-python3.11-georss-generic-client-0.8
ls -la /nix/store/2myrdl72cldb9ni307x5n3g3ljlfvqx8-python3.11-georss-generic-client-0.8/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

python311Packages.georss-generic-client

Maintainer pings
cc @fabaff for testing.

duplicate of https://github.com/NixOS/nixpkgs/pull/286366


[Integration] Error in console, DIAL hangs if to edit user-message while response is being generated

Send any message1
While the response1 is being generated update the message1 to message2
On Save& Submit -> error in console appears and DIAL hangs. Stop generating button doesn't work -> bug

Expected result: Previously we allowed to update user-messages see also #161. Need to discuss whether it's better to forbid Edit/delete

Verified on staging successfully


R version switch does not work in new Rstudio release

I am afraid that more details are needed for this. What OS? What rig version? What RStudio version? What command does not work exactly?

HI.
Windows 10, RIG -- The R Installation Manager 0.5.2, Rstudio 2022.12.0 Build 353.
When I first installed this Rstudio version,  the Choose R Installation window poped-up, I checked "Use machine's default 64-bit version of R". Then, when I ran "rig rstudio 4.1.2" it only started the r version 4.0.2 (automatically put by default)

Also I tried running "make-links"  subcommand but I did not find where CMD created links

I can confirm this issue on Win10. Previous RStudio-2022.07.2-576 works well with rig v0.5.2, but RStudio-2022.12.0-353 always starts with the R version selected in Rstudio's own GUI, ignoring the R version invoked using rig.
The command would be for example "rig rstudio 4.2.2" versus "rig rstudio 3.6.3".

Reported in rstudio: https://github.com/rstudio/rstudio/issues/12545

It is part of the release notes, i guess: "Updated RStudio Desktop installer on Windows to detect running RStudio by process name. (#10588)" (https://posit.co/download/rstudio-desktop/). Is there any workaround to this problem?

@AGSCL that seems to be about the installer, so unrelated to this issue. To this issue, AFAIK the only workaround is to install an earlier version of RStudio.

As today, this issue persist. Hope that the next Rstudio version resolves it

Yeah, the situation is still the same, so maybe we could work around it in rig.
Apparently, from 2022.12.0 RStudio uses a config.json file to store the selected R version. So if we change that before starting RStudio, and then restore the file after, that could work.
IDK if there are any caveats. E.g. if RStudio does update this file on startup, and then we restore the original file, then the updates by RStudio will be lost, and this also creates a race condition potentially.
Alternatively, I could send a patch to RStudio to look up the R executable from an environment variable, just like on Unix.

This should work now with the dev version, release is coming soon!


Remove the v prefix

what
Remove the v prefix to keep releases consistent
why

Consistency

references

@Makeshift pointed this out earlier today

The tag name is correct - this is just the title.

This is why I labelled it as a minor inconsistency in chat - It confused me for a couple of seconds because I accidentally wrote the release name and not the tag name when doing a ?ref (because I'm used to them being the same). But it's a cosmetic issue really, it doesn't make a difference unless people don't know how ?ref works.
I'm assuming the correct place to PR this would be here and here, though I'm not sure it even merits the effort ;)

Oh woops. Good catch folks.


docs: fix failing example

Hi there,
Since the last addon update, numpy and pillow (maybe others) seemed to be broken when installed through pip. See here:

https://community.home-assistant.io/t/using-pillow-with-appdaemon4/335884
https://community.home-assistant.io/t/issue-with-numpy/332610

It seems that this latest image is giving problems with Python C extension libraries. For example, this used to work:
system_packages: []
python_packages: [Pillow]
init_commands: [python3 -c 'from PIL import Image']

Now this needs to be translated to:
system_packages: [py3-pillow]
python_packages: []
init_commands: [python3 -c 'from PIL import Image']

The same happens with Numpy.
Proposed Changes
This PR suggests changing the Pillow from python package to system package  (py3-pillow). I totally understand this is just a documentation example, but in my opinion, it makes more sense to have a working example rather than a broken one.
Regards,
Xavi M.

I don't want to adjust that.
The main reason; is that it shows an example of how to use the configuration, it is not meant to be cut and pasted. For that reason, I'm actually perfectly happy with the fact it doesn't work.
Nevertheless, thanks for willing to contribute 👍


[C++][Parquet] Fix Arrow library path detection

Reporter: Kouhei Sutou / @kou
Note: This issue was originally created as PARQUET-940. Please see the migration documentation for further details.

Uwe Korn / @xhochy:
Issue resolved by pull request 286
https://github.com/apache/parquet-cpp/pull/286


Add versions to irma-ansible

The idea is to have a working version while we modify the scripts & correct minor bugs.

a testing branch has been created


Comet 2019015

Added new UNode for comet search engine

Sorry,  I will update the branch first before making the pull request.


add GLMM.jl for EM with LOCO and more functions

@hkim89 I would like to avoid the notion of genotype (thus genodata structure) and anything related to genetics, in this package ... because it is meant to be a generic tool for variable selection under GLMM right? In susieR we managed to implement it such that you cannot tell it's for genetic data anlaysis in the code. The inputs are just matrices with columns. Can we try the same? Even with LOCO, that's essentially running SuSiE-GLMM with different kinship inputs. Those parts should be handled in downstreams analysis pipelines and not in this package. Recall that we discussed that the package will not be built to scan the genome. Rather it should focus on analyzing the inputs of one region to fine-map, as outlined in our write-up.
I'm open to discussion on this though, if you would like to convince me to handle it otherwise.


Markdown Extension to include LaTeX support

Would it be possible to introduce some mathematical expression support? I have seen unicode rendering done in the past of math expressions, but perhaps even some iterm/sexel magic could be done to embed equations.
Thank you for all your work on this project, by the way! It's quite phenomenal!

I will look into this! If there's a crate/tool that can turn latex expressions into images, then it should be fairly straightforward to add here. We'd just need a step that takes latex code blocks, runs them through this and emits an operation to render an image.

I think if you're willing to have a temporary folder instantiated, it could be very cool and worthwhile to have the ability to summon the native PDFLaTeX renderer and pass the generated document through ImageMagick to get a fully rendered LaTeX image, especially for the scientists who I feel like are pretty likely to be attracted to a utility like this (like in my case).
However, for nice and compact solution, the https://docs.rs/mathjax/latest/mathjax/ crate seems to closely approximates what is already done by GitHub-Flavored Markdown, and you could leave the responsibility of rendering more complex LaTeX expressions to people to handle on their own.
I'm personally biased toward solution 1 (or a mix of the two) but mostly because it would be really frickin awesome, but otherwise either sounds pretty chill and and awesome enhancement to have!

Great, thanks for the pointers. I'll look into this after I fix outstanding issues, this would be a great addition for sure.

Alright so I looked a bit into this. My thoughts on each of these:
Mathjax
This is as simple as integration could get. The output of the render operation for that crate gives you a DynamicImage from the image crate which is the same presenterm uses internally so it would be trivial to integrate it. However, I do have some concerns:

Dependencies: this crate alone with the default features (uses a headless chrome under the hood) pulls in 280+ dependencies. This is not necessarily terrible but that'd more than double the number of crates used by this tool so it isn't great.
Reliability: this requires an internet connection to render formulas because it points to the mathjax js library hosted on some CDN. This to me is quite bad as I expect presentations to work above anything else. If your formula is correct and you have whatever tools are needed, it should be rendered.  Granted, we could tweak the code to have a local version of that script instead but that reduces the "simplicity" selling point of this solution (e.g. we need to bundle that file, copy it somewhere, reference it, etc).
Speed: rendering a tiny formula takes around 4 seconds in release mode in my computer. This is a bunch of time that would stall the app when it opens (and yes, this could render in the background but I'd rather avoid this). We could cache it in the filesystem somewhere (e.g. ~/.presenterm/cache) though, so you'd only really pay for this once per formula. The cache also helps with the "reliability" point above: if you render it once you don't need to render it again so unless you're giving the presentation in a new computer, it would work. But then again, it goes against the simplicity selling point.

pdflatex + convert
This works quite nicely as well. I followed this idea (let me know if there's a better way to do this!) and it seems to work fine. How this could work would be we'd have a template .tex file, and then whenever we find a code block that contains latex the template file gets the user defined latex code replaced, written somewhere, then pdflatex it, convert, read it, display it. This would mean that you could not necessarily just have formulas which is... I guess okay? We could try to limit this to \formula but it's going to be quite tricky to restrict that you're not somehow escaping it so I think we might as well render whatever latex you write and there's that. This means you'd have to have a block with latex as a language and then use \formula inside.
Notes on this one:

Dependencies: you need to have pdflatex + the standalone class installed + imagemagick's convert for this to work, but at least these are external dependencies and don't bloat the app binary.
Reliability: this is self contained so as long as you have the dependencies it should work.
Speed: rendering is very fast; we wouldn't need caching or anything else.
Security: I'm maybe a bit concerned about people escaping the latex file and doing bad stuff (e.g. writing to a file, running a command, etc). I'm not very savvy in latex so I don't know what you could do or how to avoid it, but the idea here is that presenterm would replace latex code blocks with the output image as it parses through the input markdown file. Therefore I don't want to allow for people uploading a malicious presentation file that does something bad. Could anyone provide thoughts on this one? Is there any way to prevent this? It seems like passing in -no-shell-escape should do the trick? Is there any other way to escape pdflatex that I should be concerned about? Think reading/writing arbitrary files.
Portability: we should make sure this works in all platforms, or at least in mac given images don't currently work on windows anyway.

What are people's thoughts on this?

I am honestly quite a fan of this pdflatex + convert solution, but I'd love to hear thoughts from other people in the community.

What about using the tectonic engine? It's written in Rust, and can be embedded as a Rust dependency in the project.

Yeah that's worth exploring too. I was also considering typst which, while not LaTeX, will let you write formulas. I managed to make this work in like 2 lines of typst the other day + I only had to install a single tool, while I had to use a bunch of boilerplate and install a bunch of stuff to make the pdflatex one work.

Yeah, installing tex nowadays is really painful comparing to other modern tools. Tectonic aims to be easy to install and use and still produce latex output.

Could an interchangable backend, perhaps a plugin-friendly interface be suitable for future steps with this and other potential connections to external applications?

perhaps a plugin-friendly interface

I don't really want to go this route until there's enough use cases that makes this worthwhile. For now I think it makes sense to use the best tool available here, which may be tectonic given besides being a single tool, it can be installed in every platform easily.
I have some stashed code that already works to some extent here but I need to pick it back up and polish it. I'll get to this again soon.

Would it be possible to get a sneak peek at this rough code, say so I could give a presentation tomorrow with it? If not, that's very understandable.

lol no, sorry, it's a bit too early. I'm prioritizing other stuff for now but I'll get to this soon. What I had was pretty crappy, as in, the size of the rendered formula was tiny. You can generate images and embed them if you want to use presenterm for now

That's fair, thank you for the response!

Closing this. Docs are here

Thank you so much!


Use KoreanAnalyzer for Korean language (ko)

This PR enables the use of KoreanAnalyzer, an analyzer specialized for Korean.
The previous CJKAnalyzer only splits sequences into bi-grams, while KoreanAnalyzer splits a sentence into morphemes.
LUCENE-8231

Hi @sudokim thanks for the PR! Do you have any idea if effectiveness improves as a result of switching the analyzer? E.g., on MIRACL or Mr.Tydi?

Hi @lintool, here is the comparison result between CJKAnalyzer(previous) and KoreanAnalyzer(suggested) on Mr. Tydi v1.1 (ko) dataset.

Analyzer
Indexing time
Index Size
Train Recall@100
Dev Recall@100
Test Recall@100

CJK
3:57.55
1.3G
0.6178
0.6733
0.6188

Korean
11:12.56
880M
0.7162
0.7409
0.6971

It seems that KoreanAnalyzer performs better, although indexing takes much longer.

Great! Do you happen to have MRR scores? And also results on MIRACL? (Which will give us nDCG scores.)

Sure! Here are the results:
Mr.Tydi v1.1

Indexing Time
Index Size

CJK
3:57.55
1.3G

Korean
11:12.56
880M

Train Recall@100
Dev Recall@100
Test Recall@100
Train MRR
Dev MRR
Test MRR

CJK
0.6178
0.6733
0.6188
0.2596
0.2888
0.2848

Korean
0.7162
0.7409
0.6971
0.3103
0.3281
0.3025

MIRACL

Indexing Time
Index Size

CJK
3:43.85
1.3G

Korean
11:13.62
876M

Dev nDCG@10
Dev Recall@100

CJK
0.4190
0.7831

Korean
0.4528
0.8554

Awesome, that's great!
We'll get this merged in... but it triggers a long dependency chain... we need to fix the regression... we also need to fix the pre-built indexes for pyserini, etc.
Let me queue this up and figure out the cleanest way to do this.
In the meantime, would you be willing to add a test case that confirms tokenization is done "correctly"?
cc/ @crystina-z @thakur-nandan


Missing bullet points in Outlook API doc

In https://docs.microsoft.com/en-us/dotnet/api/microsoft.office.interop.outlook.application?redirectedfrom=MSDN&view=outlook-pia#methods you have:
The Outlook Application object has several purposes:
followed by three empty bullet points.  It appears that some text is missing here.

@supernova-eng,
Sorry for the delay on this. @davidchesnut and our team does not control .NET APIs. Can you please find the actual docs team owner? Thank you.

@supernova-eng,
Sorry for the delay on this. @davidchesnut and our team does not control .NET APIs. Can you please find the actual docs team owner? Thank you.

@jquinn60137 Please move this issue to https://github.com/dotnet/dotnet-api-docs/issues


GAP-2096 | conditionally don't reset section status to in progress

Description
GAP-2096
Summary of the changes and the related issue. List any dependencies that are required for this change:
Needed to add a way to not reset the section status when editing a Custom Section question from the summary screen
https://github.com/cabinetoffice/gap-find-apply-web/pull/234
Type of change
Please check the relevant options.

[ ] Bug fix (non-breaking change which fixes an issue).
[x] New feature (non-breaking change which adds functionality).
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected).
[ ] This change requires a documentation update.

How Has This Been Tested?
Please describe the tests that you ran to verify your changes:

[ ] Unit Test

[ ] Integration Test (if applicable)

[ ] End to End Test (if applicable)

Screenshots (if appropriate):
Please attach screenshots of the change if it is a UI change:
Checklist:

[ ] If I have listed dependencies above, I have ensured that they are present in the target branch.
[ ] I have performed a self-review of my code.
[ ] I have commented my code in hard-to-understand areas.
[ ] I have made corresponding changes to the documentation where applicable.
[ ] I have ran cypress tests and they all pass locally.

needs tests - we also dont have a test to check that completed sections are marked as in progress when you update a question in it so ill add both


Flying key preview is pretty awful

Using the default popups, it looks bad. Custom popups will take a lot more work, may also cause performance hits?

@gazlaws-dev But at least try to make it like the standard android keyboard popups: larger box, smaller text   and black text on white box.

Duplicate of #14


[relates #627] Restore xmlbeans target generated-classes

Refer to this link for build results (access rights to CI server needed):
https://fabric8-ci.fusesource.com/job/wildfly-camel-pullreq/449/
Test PASSed.


error: JSON error: invalid type: sequence, expected a map

When I try to check inscriptions of the wallet it returns this error.
Is the command wallet inscriptions not working yet?

did you manage to get around that ?

Did not try again as there was no update yet.

hey, half a year passed - anyone figured it out? 😄