Official 3.0 schema?

In the schemas directory, I'm not seeing v3.0.  Are there plans to release an official jsonschema document?

There are actually two Schemas that are being worked on.  One is here https://github.com/OAI/OpenAPI-Specification/pull/1270 and the other is here https://github.com/OAI/OpenAPI-Specification/pull/1236
As JSON Schema cannot fully validate an OpenAPI document we are still not sure how best to include and describe a JSON Schema.  Calling it an "official" JSON Schema has the risk of making false promises.

As JSON Schema cannot fully validate an OpenAPI document...

I'm not sure I understand what you mean.  That previous v2 schema did a pretty good job.

Calling it an "official" JSON Schema has the risk of making false promises.

Hopefully we get an official schema; otherwise, tooling and frameworks will be more difficult to author.

@jsdevel the previous v2 schema did a partial job, as will the one for v3. Relying solely on the schema for validation is not good enough.
We will publish a schema, but it is not the source of truth. JSON Schema simply cannot fully validate the spec.

I see.  As long as we have something close to what we had for v2 then I think we'll be in good shape.  I see what you mean now about not being able to fully validate the schema.  I personally was happy with how close v2 came.
I really appreciate the work you guys are doing and look forward to receiving the new 3.0 schema as soon as it's ready!

You can try using the ones in the PRs for now, one of them will surely be used. I don't expect anything 'official' to be available for at least two weeks.

Hopefully we get an official schema; otherwise, tooling and frameworks will be more difficult to author.

Yes, writing tooling that fully validates the spec is hard work (so was writing the spec :-) ).  But hopefully we will only have to do it once for each language.
My concern is that if tooling vendors take a shortcut by depending on JSON Schema to do all the validation we will end up with partially validated documents and potentially incompatibilities between tooling.

My concern is that if tooling vendors take a shortcut by depending on JSON Schema to do all the validation we will end up with partially validated documents and potentially incompatibilities between tooling.

I share your concern.  It's kind of a catch 22.  If you don't provide any schema, the chances for variance is high.  If you do provide a schema, the chance for variance is lower, but reliance on the schema will still cause variance to exist.

@darrelmiller wrote:

My concern is that if tooling vendors take a shortcut by depending on JSON Schema to do all the validation we will end up with partially validated documents and potentially incompatibilities between tooling.

This is something that I want to address in the next draft (draft-08, not draft-07 which is in the feature-frozen pre-publication feedback period).
JSON Schema currently does not support extended or restricted vocabularies well.  You can declare your meta-schema with $schema but if the tool does not recognize that meta-schema then it is useless.  However, if you limit yourself to standard vocabularies, you find this sort of partial-validation situation.
I am trying to build support for a more modular approach to JSON Schema vocabularies.  The standard validation vocabulary would be one, and you could declare that you are using that.  Tools that understand it will validate properly based only on that vocabulary (so this is the partial validation).
You would also be able to declare and use your own vocabulary, which would be recognized by any tools in your own ecosystem (including plugins to validators that offer an extensible design, as some do to varying non-standardized degrees right now).
So you could make up keywords that would enforce the missing constraints, and declare that you are using those in such a way that tools that you support/endorse/whatever can perform full validation, while standard tools support a graceful degredation to the standard validation vocabulary.
If this sounds of interest, please let me know (probably by email- my email is on the JSON Schema spec- or in an issue at the json-schema-org/json-schema-spec project rather than further comments here).  Or if it sounds like a horrible idea, that would be helpful to know as well.
You could still end up with variance this way, particularly if there is no support for your custom vocabulary in someone's chosen language.  But there would be a structured, interoperable mechanism to add support.  I assert that this would be useful, but it is a controversial proposal so I am trying to socialize the idea more broadly to gauge support.
OpenAPI is a major use case for JSON Schema  (both to validate it, and as a user of JSON Schema within its own format), and I would like to see its use cases fully supported.

@handrews I think the features you outlined would be helpful.  So long as we get some form of a JSON schema finalized, I think we can at least support more tooling.
To date, has a schema been officially published?  I'm still not seeing v3 in the schemas directory.
Also, even having the draft schemas in the repo would be helpful.  Iteration is fine.

See #1270 and #1348.

@ronkorving @webron and I discussed the state of things this week and I will be taking a pass at finalizing this in the next couple of weeks.

This is really a duplicate of https://github.com/OAI/OpenAPI-Specification/issues/1301.  Going to close this out to keep the focus there.

@jsdevel if this issue is a duplicate, then the published readme (https://www.npmjs.com/package/express-openapi) should probably not point to this page, but to the correct issue?


make en

1 error(s), 1 warning(s)
Traceback (most recent call last):
File "third-party/build/closurebuilder.py", line 293, in
main()
File "third-party/build/closurebuilder.py", line 282, in main
compiler_flags=options.compiler_flags)
File "/home/zero/Documents/blockly/blockly-games/third-party/build/jscompiler.py", line 160, in Compile
raise JsCompilerError('JavaScript compilation failed.')
jscompiler.JsCompilerError: JavaScript compilation failed.

Several dependencies (including Blockly) changed over time, breaking the build.  Blockly Games has been updated so that everything builds properly now.


[Enhancement] Discord Integration

Hey there,
Thinking it might be a good idea for a Discord integration on Keybase, just another level of verification I think would be pretty useful.
Thanks,
Pointy

This is a duplicate.
It was decided that there is no way to publicly verify the proof through a simple http request, so no was the answer.


Extracted ADT cell Barcodes do not match those found in the STAMPS

After extracting the top 3000 ADT counts, I was surprised that only about half of these barcodes was present in the cell barcode read counts file output from the dropseq-tools pipeline.
Is this due to the barcode synthesis correction step in the dropseq pipeline?
Do you have plans to integrate this script with that pipeline in order to take advantage of their barcode error detection/correction? It should not be too much of an issue since the pipeline extracts the cell and UMI barcodes from read 1 and adds them to the XC and XM tags, respectively, of the resulting BAM/SAM file.

Hello @JulianSpagnuolo
I missed your issue sorry for the late response.
You will almost always see a discrepancy between rna and tag data for droplet based protocols. This is because the lowest RNA content cells that might not be selected in the end will still be valid from a tag perspective.
As an example, High mitochondrial/dying cells will come up with tags but won't be kept for RNA because of the low amount RNA.
There is not cell barcode correction in CITE-seq-Count, so you definitely lose a few reads. But the sensitivity of CITE-seq is so high that I would not worry about this.
You can "adjust" this by using a whitelist of cell barcodes. Either take the top cells in RNA and use those for the tags or the other way around. You could also be very broad and take the union of top cells from each.
There is definitely an incentive to integrate both data in dropSeqPipe. It will be done in the future, but there is no fixed plan as for a specific release date yet.

Thanks @Hoohm , thanks for the feedback, I have found that it is better to use cell barcodes that have high antibody counts to find the "good" cell barcodes in the transcriptome data (because of the high sensitivity of the method).
As for the barcode correction - I have already gone ahead and written my own Rcpp solution (currently available on my git: julianspagnuolo/adtseq) that is compatible with the dropseq-tools pipeline written by @jamesnemesh
I also modified the antibody barcode detection to take into account the phred quality of the base call since I was concerned about mismatches at low quality base-calls (not always a problem but I had a bad sequencing library to get data from).


cmake: fix CMAKE_FIND_FRAMEWORK

Motivation for this change
#37015 introduced new macOS specific default CMake flag: CMAKE_FIND_FRAMEWORK=last. But it has no effect since the flag value is case sensitive and should be CMAKE_FIND_FRAMEWORK=LAST, see CMake documentation.
CMAKE_FIND_FRAMEWORK removed as not needed anymore for these derivations:

openorienteering-mapper
weechat
mozart

Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Yeah - this is because everything depends on the cmake setup-hook

Of course. What I was wondering is why this flag is not specific to darwin ? Like
 if stdenv.platform == "darwin" then // include the flag..
Found the answer by myself :-). Because it is a bash script, not a nix file.

should have gone to staging, merged in 1a3b07cf8e7128ab6b8c58255f98f932d3c97cc9


introducing reselect

Problem:
We have some Zustand selectors that are very expensive to run (3ms per frame). We also have a ton of 0.1ms-sized selectors that add up to a significant time!
Fix:
This PR introduces reselect and uses it in 1 place, in the fattest selector we had in NewCanvasControls.
Commit Details:

Removing the editor dependency from getPropertyControlsForTarget, created a new getPropertyControlsForTargetFromEditor that extracts properties from the editor for the callsites I didn't want to touch now
created the first reselect selector selectElementsThatRespectLayout
added optional selectorName argument to useEditorState for easier debugging

appears to be under active(?) development

the approved PRs seem to be very minor things, like update README and them moving the repo and bumping lodash


titles on sw-text-editor-toolbar-button

Description
For better user experience there should be a function related title on the buttons. sw-text-editor-toolbar-button
Example

Hello,
thank you for creating this issue.
I have opened a ticket on our Issue Tracker for you. See the ticket link: https://issues.shopware.com/issues/NEXT-7780
Please use this ticket to track the state of your issue.

Hi @tinect
I have imported this issue to our internal ticket system.
If you don't want to wait for us to fix this an pull request is very welcome :)
If this is okay for you I'd like to close this issue here.
Sunny regards
Pantrtxp

Interested :-) will work on it today again. Existing PR #686

interested, will check and extend existing PR #686


Improvements in setup of jest runner. Update GettingStarted documentation.

We should probably keep the same patterns with defining E2E test like unit in jest (talking about tests pattern). Also setup can be in one file if we conditionally load detox object. How do you feel about this? :) @DanielMSchmidt

I've applied some changes to this one @DanielMSchmidt. If anything more is bothering You add more notes :)


How can I hide the "+ New" Button for cutom user?

Question
I'm trying to provide access to certain users or employees. However, even after I set the permissions to minimum required permissions, the (+ New) button still shows in their screen. How can I hide this button for these users?

Also, how can I disable the the options on the chart to keep the chart simple? or at least hide some of these options.

Thanks in advance

+1

Hi @sweileh - I don't believe this functionality exists in Superset right now. I'm tagging this issue as an enhancement request.

Thanks Will!
This would be great.

Is this functionaly implemented?

Hi @MHammad200 and @sweileh , thanks for the enhancement request. I will be considering implementing this feature next year after some product research. assigning this issue to myself, feel free to follow up with me in Slack.

+1


Create recipe book of useful custom actions/commands

Things like join, editor.action.joinLines and other useful custom actions

We could start on the wiki, tho people prob won't find it unless it's in the main docs hub


Update also the fascdr installation scripts in thirdparty

commit bc434bd changes in installation scripts are linked to similar ones in thirdparty/fastcdr. fastcdr must reference commit cb513de.

CI not run. Not necessary for this PR.


[Impeller] Better handling of multi-contour paths with non-zero fill mode.

Currently Impeller uses a fast path for tessellation if SkPath reports that the path is convex. If not, we fall back to using libtess. Unfortunately, SkPath doesn't track the convexity per-countour. IN some cases, we may have multiple countours each made up of convex shapes. These shapes could be individually tessellated using libtess or the convex tessellator, and then drawn with overdraw prevention enabled. This would be much faster than throwing the entire thing at libtess.
Naturally this only works for the non-zero fill mode.

From Triage: @jonahwilliams says "we can't really figure out if something is a hole of not without doing a whole bunch of math" which is probably not worth it.


add support for multiple mirrors on the same port

Codecov Report

Merging #494 into master will decrease coverage by 0.04%.
The diff coverage is 80%.

@@            Coverage Diff             @@
##           master     #494      +/-   ##
==========================================
- Coverage   94.62%   94.58%   -0.05%
==========================================
  Files          33       33
  Lines        2216     2218       +2
  Branches      189      191       +2
==========================================
+ Hits         2097     2098       +1
- Misses         76       77       +1
  Partials       43       43

Impacted Files
Coverage Δ

...hBoundControllerAbstraction/proxy/faucet/parser.py
68.8% <80%> (-0.36%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 767079e...d51df3f. Read the comment docs.


[BUG] focus indicator not visible in dark mode for grey nav links

Description
Set the site to dark mode. Tab through the site on desktop to the navigation on the left-hand side. Press enter on Frontend to show the categories within then tab through Animations, colors, Design Insp... etc. The focus indicator cannot be seen because it is black on a dark purple background.
Screenshots
No response
Any additional information?
This is probably a good first issue so if someone else wants to have a go then feel free. Otherwise, I can fix it.
What browser are you seeing the problem on?
Chrome
Checklist

[X] I have checked the existing issues
[X] I have read the Contributing Guidelines
[X] I am willing to work on this issue (optional)

https://github.com/rupali-codes/LinksHub/assets/57045550/60917bc8-af93-4b47-8fd0-3d2e75a4fdd0

i fixed it can i get it assigned so i can make pr ?


Python 3 tests, in shared Docker container.

Similar to PR #553, but does everything within a single Docker container rather than trying to have Travis handle the different Python versions (which didn't work because Docker's actually the one deciding what version(s) of Python get installed).
Fixes #508.

(sigh) this worked locally. I'll close this PR until I figure it out.


examples: vue sub-components

Closing due to conflicts, but we should try to get this example in the docs at some point


Quiz Game GUI Based

Interactive Quiz Application
This Python application creates an interactive quiz using the Tkinter library for the graphical user interface. Players can answer a series of multiple-choice questions and test their knowledge on various topics.
Features:
Graphical User Interface: The application provides a user-friendly GUI, built using Tkinter and styled with the Plastik theme from ttkthemes.
Quiz Questions: The quiz consists of multiple-choice questions. You can easily add, edit, or remove questions by modifying the self.questions list in the code.
High Scores: The application allows players to enter their name, and it stores their high scores in a dictionary.
Sound Effects: It uses the pygame library to play sound effects for correct and incorrect answers.
Timer: Each question is timed, and the timer label displays the time left to answer each question.
Usage:
Clone or download this repository to your local machine. Make sure you have the required dependencies (Tkinter, ttkthemes, pygame) installed. Run the Python script, and the quiz application will start. Answer questions, and your score will be displayed at the end of the quiz. Feel free to customize the quiz questions or the appearance of the application to suit your needs. Enjoy playing and learning!
Note: Make sure to provide the required sound files, "correct.mp3" and "incorrect.mp3," in the same directory as the Python script or specify their correct paths.
Wish to get this commit merged and gain the hacktoberfest badges for this

Wish to get this commit merged and gain the hacktoberfest badges for this


SOLR-16397: Swap core v2 endpoints have been updated to be more REST-ful

https://issues.apache.org/jira/browse/SOLR-16397

Description
V2(JAX-RS) Endpoints for core admin op "SWAP".
Solution
Removed the older SwapCoresAPI.java class. Created new interface SwapCoresAPI.class and a class implementing the interface SwapCores.java.
Tests
There is already one test testing the internal Swap core functionality. What's actually missing is the test case for JAX-RS endpoints. We are working on it, hopefully in future there are test cases for that as well.
Checklist
Please review the following and check all that apply:

[x] I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
[x] I have created a Jira issue and added the issue ID to my pull request title.
[x] I have given Solr maintainers access to contribute to my PR branch. (optional but recommended)
[x] I have developed this patch against the main branch.
[x] I have run ./gradlew check.
[x] I have added documentation for the Reference Guide

Thanks again @iamsanjay ; another API down!


Update ComposeSequence.java

More compose sequences.
Those characters are supported by Linux international keyboard with AltGr+? dead key.
Because ?b causes conflict with ?ba and ?bA, ?bb was used.

@klausw Would you be interested in more Compose sequences like these?  I myself was thinking of adding a bunch for emoji, but iff such would be welcome.

I personally hardly ever use Compose sequences, so I may not be the best one to judge this, but I find them hard to remember and unintuitive. I'm all in favor of straightforward additions, especially if it follows an existing standard such as the Linux international keyboard, but I think that this wouldn't be the right place to do complex additions such as emoji.
Yes, people keep asking for emoji support, but I think a hard-to-use implementation may lead to even more complaints than not having them at all.
I've been thinking about adding a generic Unicode code-page browser that would allow entering arbitrary Unicode symbols, but that would still not be the same as emoji support since people would expect searchability and compatibility fallbacks for symbols not included in the current fonts.

@KlausW Perhaps an optional 2nd (&c) quick-click of Fn to access an emoji page (or few), w/ AltChars to expand those to a more full range?


fix: set launch icon and external text alignment to the vertical center

TNL-8588
Fix launch icon and text alignment to the vertical center
before

after

Coverage remained the same at 90.646% when pulling de87d6ae7681964d27cb40f39e55de3ff235c2a3 on aansari/TNL-8588 into 4e566039d0392caf6efd6f1d8d98f0e7e91b621b on master.

:tada: This PR is included in version 16.6.4 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Rename packages to have generic api/implementations in one place

agreed on com.palantir.ls.server. format

+1 after checks pass!


release 6.18.1

I need to fix #307  asap because it blocks customer projects. To have a clean baseline, I put all dep upgrades that happened on 6.18.0 + one community PR to the 6.18.1 release. This will be the last release build for 7.18CE.
I triggered the mvn relase pipeline based on the new github release.

AS I need to move on with 6.19.0 I could need an adult: Could you check if everything is running as expected, so I can continue to work on 7.19 compatibility on master? Or should I wait until 6.18.1 is on central? Thanks!

done: https://mavenlibs.com/maven/dependency/org.camunda.community.mockito/camunda-platform-7-mockito


548 guide verify airnode address

This page was added to the /reference secant and not /guide as it really is just some links to DNS records with an explanation of Airnode addresses.

@dcroote - I made all the fixes. Am going to deploy this today as Jiri needs it. Kaiko is working on their DNS text record.


Cmake package improvements

Paging @Flamefire. Please pay most attention to 21b5e4b6ebf1c3e1e9ea06fe3ff921b07e12b04f.

Ok from a quick test build&install the shared version seems to work correctly except of the usage of plain library names instead of full paths in the installed config which will be a problem if the lib path is not in the linker path
However the static version links to libs with absolute paths from the build tree in the installed config. (e.g. libmklml_intel.so) I don't know why though. All I've found is that the INTERFACE_LINK_LIBRARIES property contains $<LINK_ONLY:$<BUILD_INTERFACE:/full/path/to/libfoo.so>> for the static version that is not there for the shared one.

Not required as no framework libraries are used
Can you please elaborate?

Thanks for testing. I'll check the static version too. I will see what I can do with TARGET_NAME / LIB_NAME / NAMESPACE as well.

Okay, I do not see how to make this work with static library and cmake 2.8.11. Cmake 2.8.12+ has (CMP0022)[https://cmake.org/cmake/help/v3.0/policy/CMP0022.html] that seems to do the trick. This has received limited testing. I'm running more tests now.

Can you please elaborate?

This was about the PUBLIC_HEADER in the install command. From the manual:

The PRIVATE_HEADER, PUBLIC_HEADER, and RESOURCE arguments cause subsequent properties to be applied to installing a FRAMEWORK shared library target’s associated files on non-Apple platforms.

This means that it only applies if the FRAMEWORK property is set (see https://devdocs.io/cmake~3.8/prop_tgt/framework#prop_tgt:FRAMEWORK for details) which is not the case here, so it can be omitted as it is easily confused with INCLUDES keyword of newer CMakes install command.

I understand the comment about PUBLIC_HEADERS only applying to FRAMEWORK targets, but setting this property makes CMake install the headers in a proper location with a single install command. I'm reluctant to change this since it works on all the CMake versions released so far.

I understand the comment about PUBLIC_HEADERS only applying to FRAMEWORK targets, but setting this property makes CMake install the headers in a proper location with a single install command.

I have to double check with my test system tomorrow but are you 100% sure about this? The docu does not tell this as I read it as "install(...PUBLIC_HEADERS installs the files marked as "PUBLIC_HEADERS" via the target property to the given destination". No such property is set here, so how would CMake know which files to install? Previously this was done via the explicit install command: https://github.com/intel/mkl-dnn/blob/b88908909240bdab10f1cb7c256caa264fc15067/src/CMakeLists.txt#L184
I'll do a clean rebuild tomorrow to double-check that headers are actually installed.

I'm almost 100% sure. I am not sure that this is the intended behavior, but this works. I also had to re-read the docs multiple times and tried multiple different ways to specify the list of public headers, but this is the only one that works.
The separate install() has been removed intentionally. If you find a strong case against this, I'd be happy to revert the change.
Thanks so much for all the help.

Ok I've just seen that this PR also adds the line actually setting the property which makes this work: https://github.com/intel/mkl-dnn/blob/b2430ba360dd94e55f730de769267cd7c174e285/src/CMakeLists.txt#L97
So yes this is intended behavior and it works although I haven't seen it before.
One reason for this might be namespacing: It became common (and for good reasons) to put headers in the same folder structure as their namespace. Your current namespace is mkldnn so headers would be expected in <some-include-path>/mkldnn and included with #include <mkldnn/mkldnn.hpp> This avoids name-conflicts. (This can be solved by installing into the appropriate subfolder, though)
Best example I remember: SDL (Simple DirectMedia Layer). They have SDL 1.x and 2.x branches with major changes. Both have a main header called SDL.h and are (often) installed into SDL/SDL.h or SDL2/SDL.h. But this wasn't done always so the expected include is #include <SDL.h>. We have a project where both versions are used (plugins, loaded dynamically) so you may see how this causes problems over e.g. #include <SDL/SDL.h> and #include <SDL2/SDL.h>
A more convincing point to use the explicit install command: Subdirectories. Say you want a detail namespace (and folder). The PUBLIC_HEADERS install will strip off all directories (this seems not to be documented) so your headers from the subdirectory will end up in the top-level folder too.
TL&DR:

using the set_property this works
folders are stripped from header paths

subfolders may be required which don't work with this approach

probably install into include/mkldnn or keep prefixing all headers with mkldnn
Note that adding the headers to add_library is not required (but helpful for e.g. Visual Studio [not Code])


Data improvement MapRoulette challenges (Various markets)

Task Description:
Microsoft has created some MapRoulette challenges using Atlas Checks in various markets listed below.  These are focused on a variety of geometrical and naming data issues detected by algorithms requiring mappers to investigate each task and make map improvements where necessary. Descriptions and documentation on specific Atlas Checks can be found here. There is also a good description and overview published in July 2020 available online. These are available for any mappers interested to improve the map to come join!
Our editors will work on these projects periodically and we encourage the community of all levels to come join us in the challenge. We will post these challenges to various social platforms, including relevant talklists.
MapRoulette is a fun, collaborative tool that allows mappers to  improve OpenStreetMap by fixing individual data points on the map.
Mapping Guidelines:
We will be processing these according to OSM local guidelines, and adhering to the Organized editing guidelines.
Area of Focus:
These will be a country wide efforts.
Data Sources:
All available aerial and street level imagery resources in JOSM will be used to resolve task challenges.
Changeset Comments:
The team will be adding the changeset comment #AtlasChecks.
Error Detection:
We will use the validator in the JOSM tool to look for errors, following the wiki as well as the LearnOSM guidelines.
Project list:

Australia: MapRoulette
Colombia: MapRoulette
Ecuador: MapRoulette
Fiji: MapRoulette
Venezuela: MapRoulette

Additional Info:
For more information please reach out to<EMAIL_ADDRESS>Feel free to post questions here on github or to mappers through MapRoulette, and comment on changesets.

See https://github.com/microsoft/Open-Maps/issues/84


Inject the repository map and custom prefix in autocompletion prompt

Validations

[X] I believe this is a way to improve. I'll try to join the Continue Discord for questions
[X] I'm not able to find an open issue that requests the same enhancement

Problem
As also mentioned in issue #2712 , automatic model completions are “blind” because they do not know enough about the context, so they try to predict parameters or predict queries without knowing the actual structure of the database/codebase
Solution
In the configuration, add a flag to choose whether or not to inject the repository map into the auto-complete prompt and possibly configure which paths to include in the repository map to avoid adding unnecessary code.
I would also propose to introduce the possibility of adding a prefix for each completion prompt.
Currently I already do this (manually) and it works really well, my use case is to add the entire database DDL as a prefix for each file in my backend, and the suggested queries are extremely accurate! (i'm using Qwen 2.7 7B for completions and with this feature is better than Copilot!)
Note:
To inject it into the prompt simply add it as commented code. Maybe by detecting the programming language and adjusting the comment in accordance with the language and adding it to the beginning of the prompt

I would love to help implement these features, only I don't know Javascript and I don't know how to create an extension for VS Code at all, so understanding the codebase and figuring out where to get my hands on it is very complex for me, however suggestions and help are appreciated (maybe using discord)

Seems related to #2615


.net web api from vs2017 iisexpress localhost to android simulator like gennymotion

Can we get a detailed help for this scenario, as it is simply not solved. I could not observe a single help on internet in this case.

Dupe of #2087


${workspaceFolder} broken in "default" of task inputs

VSCode Version: 1.38.0
OS Version: Windows 10

Steps to Reproduce:

Add a promptString input with a default of "default": "${workspaceFolder}/foo. The resolved value will be /foo.

This did work in the past but I haven't tested it in a while.

Does this issue occur when all extensions are disabled?: Yes/No

We don't support resolving variables in input values.

Why not? That's super useful.

To avoid recursion and because we just haven't done it yet for simple variables https://github.com/microsoft/vscode/issues/64358


State changes reverts graph back to initialZoom

Describe the bug
If intialZoom is not null in the config, and there is a state change to the react application then the graph will be reverted back to the initialZoom.
To Reproduce
Steps to reproduce the behavior:

set initialZoom: 0.1
have state change (can be anything), but for this purpose, "setDoesNothing(false) when node is clicked"

Expected behavior
Expected that the initialZoom is only "initial" and that any state changes should not revert the graph to it's initialZoom
Environment:

using hooks
react-d3-graph version 2.6.0

Additional context
Add any other context about the problem here.

@aoshi321 Just a hacky way... setting the config with initialZoom back to null if I make any state changes.


HTTPS connection is rejected in my server

Summary
I have set up my server and solved the issue https://github.com/syuilo/misskey/issues/2787, now I face another puzzle. My server rejects HTTP and HTTPS connections.
Expected Behavior
https://5.distsn.org:2001 shows the top page of Misskey.
Actual Behavior
https://5.distsn.org:2001 rejects the connection.
http://5.distsn.org:2002 returns the empty reply.
Steps to Reproduce

Set up my server.
Configure systemd service, enable and start it.

Environment
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.5 LTS"

Log
sudo journalctl -u misskey log:
Sep 30 18:18:52 ik1-312-16250 systemd[1]: Started Misskey daemon.
Sep 30 18:18:52 ik1-312-16250 sudo[1766]:  misskey : TTY=unknown ; PWD=/home/misskey/misskey ; USER=root ; COMMAND=/usr/bin/npm start
Sep 30 18:18:52 ik1-312-16250 sudo[1766]: pam_unix(sudo:session): session opened for user root by (uid=0)
Sep 30 18:18:53 ik1-312-16250 misskey[1766]: ><EMAIL_ADDRESS>start /home/misskey/misskey
Sep 30 18:18:53 ik1-312-16250 misskey[1766]: > node ./built
Sep 30 18:18:54 ik1-312-16250 misskey[1766]: [18:18:54 INFO] Welcome to Misskey!
Sep 30 18:18:54 ik1-312-16250 misskey[1766]: [18:18:54 INFO] [Deps] Node.js v10.10.0
Sep 30 18:18:54 ik1-312-16250 misskey[1766]: [18:18:54 INFO] [Machine] Hostname: ik1-312-16250
Sep 30 18:18:54 ik1-312-16250 misskey[1766]: [18:18:54 INFO] [Machine] Platform: linux
Sep 30 18:18:54 ik1-312-16250 misskey[1766]: [18:18:54 INFO] [Machine] Architecture: x64
Sep 30 18:18:54 ik1-312-16250 misskey[1766]: [18:18:54 INFO] [Machine] CPU: 3 core
Sep 30 18:18:54 ik1-312-16250 misskey[1766]: [18:18:54 INFO] [Machine] MEM: 2.0GB (available: 1.6GB)
Sep 30 18:18:54 ik1-312-16250 misskey[1766]: [18:18:54 INFO] [Env] NODE_ENV is not set
Sep 30 18:18:54 ik1-312-16250 misskey[1766]: [18:18:54 WARN] [Env] The environment is not in production mode
Sep 30 18:18:54 ik1-312-16250 misskey[1766]: [18:18:54 WARN] [Env] Do not use for production purpose
Sep 30 18:18:54 ik1-312-16250 misskey[1766]: [18:18:54 INFO] [Env] You have root privileges
Sep 30 18:18:54 ik1-312-16250 misskey[1766]: [18:18:54 INFO] [Config] Loaded
Sep 30 18:18:54 ik1-312-16250 misskey[1766]: [18:18:54 INFO] [MongoDB] Connecting to mongodb://misskey:****@localhost:27017/misskey
Sep 30 18:18:54 ik1-312-16250 misskey[1766]: [18:18:54 INFO] Misskey initialized
Sep 30 18:18:54 ik1-312-16250 misskey[1766]: [119B blob data]
Sep 30 18:18:57 ik1-312-16250 misskey[1766]: [1.3K blob data]
Sep 30 18:18:57 ik1-312-16250 misskey[1766]: [18:18:57 INFO] All workers started
Sep 30 18:18:57 ik1-312-16250 misskey[1766]: [18:18:57 INFO] Now listening on port 2002 on https://5.distsn.org:2001

Log in my local PC:
vaginaplant@mute:~$ curl https://5.distsn.org:2001
curl: (7) Failed to connect to 5.distsn.org port 2001: 接続を拒否されました
vaginaplant@mute:~$ curl http://5.distsn.org:2002
curl: (52) Empty reply from server

May be configuration(.config/default.yml) mismatch.

Listen port decided by port section.
Listen protocol decided by https section. (define or undefined)
url indicates URL for client browser. (Unrelated with listen port/protocol)

url scheme are allways https.
url not contains port.

For example ...
url: https://5.distsn.org/
port: 2001
# comment out
# https:
#  :

=> Listen on http://*:2001/
url: https://5.distsn.org/
port: 2001
# define https
https:
  :

=> Listen on https://*:2001/

For recording my operations, now my default.yml is:
url: https://5.distsn.org:2001

# A port that your Misskey server should listen.
# This value is not a port to use when accessing with a browser.
port: 2002

(omission)
# TLS
https:
  # path for certification
  key: /etc/letsencrypt/live/5.distsn.org/privkey.pem
  cert: /etc/letsencrypt/live/5.distsn.org/cert.pem

And now I comment outed the https section, http://5.distsn.org:2002/ shows the top page!

Finally my defualt.yml is:
url: https://5.distsn.org/

# A port that your Misskey server should listen.
# This value is not a port to use when accessing with a browser.
port: 443

(omittion)
# TLS
https:
  # path for certification
  key: /etc/letsencrypt/live/5.distsn.org/privkey.pem
  cert: /etc/letsencrypt/live/5.distsn.org/cert.pem

And now https://5.distsn.org is available! Thank you very much.

Is this ascii art in the exsample.yml is outdated, or future specification?
# URL and Port settings overview
# e.g., If you want to realize following structure:
#
#               +--- https://example.com:123 ----------+
# +------+      |+-------------+      +---------------+|
# | User | ---> || Proxy (123) | ---> | Misskey (456) ||
# +------+      |+-------------+      +---------------+|
#               +--------------------------------------+
#
# You need to set 'https://example.com:123' to 'url' prop and
# You need to set 456 to 'port' prop.
#
# In other words, the 'url' prop should be the final accessible URL seen by a user.
# 'port' prop is a port that the Misskey server should actually listen
# on and it is not necessarily the port that a user accesses.


az ad sp - documentation improvement recommendations

[Enter feedback here]
The process for creating applications and then creating the associated service principles with the CLI is not clear. It would be good to reference https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals#relationship-between-application-objects-and-service-principals for the users to understand the background relationship.
In particular the reference articles mention of the relationship between tenant, application and service principle. The Azure portal Active Directory Applications Blade doesn't make the service principle relationship obvious to the CLI user.

An application object therefore has a 1:1 relationship with the software application, and a 1:many relationship with its corresponding service principal object(s).

Here is some sample CLI code that I was using (based on an old Azure RM powershell script) and trying to upgrade to use the Azure CLI working and discovered a bug in the CLI behaviour https://github.com/Azure/azure-cli/issues/14727.
# Change the following variables as needed
$appName = "Icecream App"
$appOwnerObjectId = "00000000-0000-0000-0000-000000000000" # az ad user list (find your name).
$appHomePage = "https://icecreamery.good"
$appIdentUri = "https://icecreamery.flavour"

az login

Write-Output "Creating AAD application..."
$azureAdAppId = az ad app create --display-name $appName --homepage $appHomePage --identifier-uris $appIdentUri --credential-description "OctopusDeploy" --query 'appId' -o tsv
az ad app owner add --id $azureAdAppId --owner-object-id $appOwnerObjectId

$azureServicePrinciple = az ad sp create --id $azureAdAppId | ConvertFrom-Json

# THIS IS THE PROBLEM LINE that triggers the error.  (Feels like AppRoles is more complex that my guess)
az ad sp update --id $azureServicePrinciple.appId --set appRoles=contributor

# And this is the scope for the contributor role, which I figure is the next thing to set after this.
#az ad sp update --id $azureServicePrinciple.appId --set scopes="/subscriptions/$subscriptionId/resourceGroups/$appResourceGroup"

There are some counter intuitive values for configuring service principles that make it difficult to understand the service principle relationship to applications and resource security roles. I've attempted to create service principles with the "az ad sp create-for-rbac" option in the following code sample, and associate it with an application, unsuccessfully. The Service Principle is created, but I couldn't determine how to associate it with the application.
Write-Output "Creating Resource Group for the application..."
$rg = az group create -l $azureRegion -n $appResourceGroup | ConvertFrom-Json

$ServicePrincipleName = "https://Icreamery-Acai-ServicePrinciple" # Must be in URI format. Why? The Azure portal UI for Applications use the label "Managed application in local directory" with a hyperlink)

Write-Output "Creating AAD service principal with security (but this doesn't associate it with app.)..."
az ad sp create-for-rbac --name $ServicePrincipleName --role contributor --scopes /subscriptions/$subscriptionId/resourceGroups/$appResourceGroup -o json

Creating an application manually only takes a small amount of time via the Azure portal AD UI, but the time and effort to automate that process seems disproportionately high with the current CLI documentation and API options.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e99d25ff-03e2-e015-37f4-a29b59ec50d4
Version Independent ID: 1a8e88ed-72c7-fbe7-b0ad-68f23b3946cd
Content: az ad sp
Content Source: src/azure-cli/azure/cli/command_modules/role/_help.py
Service: active-directory
GitHub Login: @rloutlaw
Microsoft Alias: routlaw

add to S175


Proposed feature: inject required parent/sibling directive controllers as locals

See this commit for a better idea where I am headed.
The basic idea would be to allow the following syntax:
app.directive('child', function() {
return {
  require: 'parent',
  controller: function(parentController) {
   // parentController is automatically injected
  }
}

This feels more straightforward than a separate linking function that sets the parent controller on the child.

pinging @caitp since I think this is his domain.

Her domain, but I'm not sure if we want to do this. For something like this, I'd suggest sending an intent to implement email to the angular Google group, and get some discussion happening first

Will do, and apologies.
closing in favor of Google group discussion.


User details format correction when /user command is used

Date: 22/09/24
Developer Name: @PeeyushPrashant

Issue Ticket Number

https://github.com/Real-Dev-Squad/discord-slash-commands/issues/253

Description

Corrected formatting of user details when /user command is used on discord with FF enabled
Added Account Created along with other user details when /user command is used
Added created_at field in dummy test data which was needed as i have added Account Created field in user
details
Implemented a function which converts Unix Epoch time to desired DD MMM, YYYY format
Changed Joined Discord date format to DD MMM, YYYY

Documentation Updated?

[ ] Yes
[x] No

Under Feature Flag

[x] Yes
[ ] No

Database Changes

[ ] Yes
[x] No

Breaking Changes

[ ] Yes
[x] No

Development Tested?

[x] Yes
[ ] No

Screenshots

Screenshot 1
screen-capture (6).webm

Test Coverage

Screenshot 1

Would vertical alignment change if another field is added? would your current solution affect the spaces of other fields and also have to update manually again?


Add tests for channel.coffee

It's a fairly simple class, but it'd be nice to have tests for that for coverage.

No longer needed.


Model weights

Hi,
Thanks for all the great work! I'd love to try it out. Unfortunately, I see that link in https://github.com/BCV-Uniandes/AUNets/issues/1 is expired. Could you please reupload?

Hello, thank you for your report. Please try again, it is working now.

Thank you very much!


Dropdown Menu not in right location when float-right

      Click

First link in dropdown

Active link in dropdown

Will misplace the submenu and make it 20 - 40px from the edge of the screen. Makes it impossible to sustain when data-closing-time="0" as the submenu is not below the text/button.

This is my bad. I should have put "align-right" on the ul (after "dropdown menu").

I'll close this. Keep on keepin' on!


Expose record metadata to mongo-cdc

Is your feature request related to a problem? Please describe.
It's possible to capture data from multiple mongo databases and collections with the connector, but there is no way to know which database or collection each record came from.
Describe the solution you'd like
The official flink debezium format provides a way to read debezium metadata fields when creating a table DDL (https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/connectors/table/formats/debezium/#available-metadata). Something similar could be done to expose db and collection from the mongo source connector document.
I'm happy to take a stab at this if someone could point me in the right direction.

I think you can take a look at the FLIP about the design of the metadadta column. Flink has an good example in the PR[2].
[1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-107%3A+Handling+of+metadata+in+SQL+connectors
[2] https://github.com/apache/flink/pull/13732

I'm going to support metadata columns for mysql-cdc, see #249 in the next days. I think it can be a good example for other connectors.

Implemented in 65665eb


Rename AccessOrderCheck to ModifierOrderCheck

This makes it consistent with the Java version.

@adireddy This is a breaking change I guess.

Next step would be to rename PublicPrivateCheck to RedundantModifierCheck and move them to modifier.

Just added functionality in dev to allow multiple names to checks so that old checks will not break as shown below.
@name("ModifierOrder", "AccessOrder")

Oh, neat. Should one of them be marked as deprecated somehow?

Added [DEPRECATED] suffix to the description. https://github.com/adireddy/haxe-checkstyle/commit/8973d66d1b7723da5cd72ef862577374a5b3b529
Only visible when --list-checks is used.

That could be interpreted as the check having been removed, when it's really only been renamed.

Deprecation is about discouraging the use. I don't think it means removed. That's how I interpret.

I think it should mention the new name at least.

done.
PublicPrivate:   Checks for redundant modifiers [DEPRECATED, use RedundantModifer instead]


Adds check in assignment creation to make sure course exists.

Fixes #359

Is this waiting on anything other than review? @moowiz

LGTM

LGTM.


[BUG]: AttributeError: 'LlamaModel' object has no attribute '_prepare_decoder_attention_mask' when using PP

🐛 Describe the bug
/aml2/colo) root@A200:/aml2/ColossalAI/examples/language/llama2# vim /aml2/colo/lib/python3.10/site-packages/torch/nn/modules/module.py
(/aml2/colo) root@A200:/aml2/ColossalAI/examples/language/llama2# export NCCL_IB_DISABLE=1; export NCCL_SOCKET_IFNAME=eth0;NCCL_DEBUG=INFO;TORCH_CPP_LOG_LEVEL=DEBUG; export NCCL_DEBUG_SUBSYS=ALL;export TORCH_DISTRIBUTED_DEBUG=INFO;  colossalai run --nproc_per_node 2 --hostfile hostfile pretrain.py --d "/aml/data/boyang/RedPajama-Data-1T-Sample" -p "hybrid_parallel" -c "7b" -b 2  -l 2048
Error: failed to run torchrun --nproc_per_node=2 --nnodes=2 --node_rank=0 --master_addr=A100 --master_port=29500 pretrain.py --d /aml/data/boyang/RedPajama-Data-1T-Sample -p hybrid_parallel -c 7b -b 2 -l 2048 on A100, is localhost: False, exception: No authentication methods available
WARNING:torch.distributed.run:

Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.

/aml2/colo/lib/python3.10/site-packages/datasets/table.py:1421: FutureWarning: promote has been superseded by mode='default'.
table = cls._concat_blocks(blocks, axis=0)
/aml2/colo/lib/python3.10/site-packages/datasets/table.py:1421: FutureWarning: promote has been superseded by mode='default'.
table = cls._concat_blocks(blocks, axis=0)
Epoch 0:   0%|          | 0/116314 [00:02<?, ?it/s]
Traceback (most recent call last):
File "/aml2/ColossalAI/examples/language/llama2/pretrain.py", line 331, in
main()
File "/aml2/ColossalAI/examples/language/llama2/pretrain.py", line 289, in main
outputs = booster.execute_pipeline(
File "/aml2/colo/lib/python3.10/site-packages/colossalai/booster/booster.py", line 205, in execute_pipeline
return self.plugin.execute_pipeline(data_iter, model, criterion, optimizer, return_loss, return_outputs)
File "/aml2/colo/lib/python3.10/site-packages/colossalai/booster/plugin/hybrid_parallel_plugin.py", line 832, in execute_pipeline
outputs = self.schedule.forward_backward_step(
File "/aml2/colo/lib/python3.10/site-packages/colossalai/pipeline/schedule/one_f_one_b.py", line 288, in forward_backward_step
output_obj = self.forward_step(model, input_obj, criterion, accum_loss, outputs)
File "/aml2/colo/lib/python3.10/site-packages/colossalai/pipeline/schedule/one_f_one_b.py", line 164, in forward_step
output_obj = model_forward(model, micro_batch, input_obj)
File "/aml2/colo/lib/python3.10/site-packages/colossalai/pipeline/schedule/_utils.py", line 120, in model_forward
return model(**data, **internal_inputs)
File "/aml2/colo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/aml2/colo/lib/python3.10/site-packages/colossalai/booster/plugin/hybrid_parallel_plugin.py", line 118, in forward
return super().forward(*args, **kwargs)
File "/aml2/colo/lib/python3.10/site-packages/colossalai/interface/model.py", line 25, in forward
return self.module(*args, **kwargs)
File "/aml2/colo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/aml2/colo/lib/python3.10/site-packages/colossalai/shardformer/modeling/llama.py", line 233, in llama_for_causal_lm_forward
outputs = LlamaPipelineForwards.llama_model_forward(
File "/aml2/colo/lib/python3.10/site-packages/colossalai/shardformer/modeling/llama.py", line 100, in llama_model_forward
attention_mask = self._prepare_decoder_attention_mask(
File "/aml2/colo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1269, in getattr
raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'LlamaModel' object has no attribute '_prepare_decoder_attention_mask'
Epoch 0:   0%|          | 0/116314 [00:02<?, ?it/s]
Traceback (most recent call last):
File "/aml2/ColossalAI/examples/language/llama2/pretrain.py", line 331, in
main()
File "/aml2/ColossalAI/examples/language/llama2/pretrain.py", line 289, in main
outputs = booster.execute_pipeline(
File "/aml2/colo/lib/python3.10/site-packages/colossalai/booster/booster.py", line 205, in execute_pipeline
return self.plugin.execute_pipeline(data_iter, model, criterion, optimizer, return_loss, return_outputs)
File "/aml2/colo/lib/python3.10/site-packages/colossalai/booster/plugin/hybrid_parallel_plugin.py", line 832, in execute_pipeline
outputs = self.schedule.forward_backward_step(
File "/aml2/colo/lib/python3.10/site-packages/colossalai/pipeline/schedule/one_f_one_b.py", line 288, in forward_backward_step
output_obj = self.forward_step(model, input_obj, criterion, accum_loss, outputs)
File "/aml2/colo/lib/python3.10/site-packages/colossalai/pipeline/schedule/one_f_one_b.py", line 164, in forward_step
output_obj = model_forward(model, micro_batch, input_obj)
File "/aml2/colo/lib/python3.10/site-packages/colossalai/pipeline/schedule/_utils.py", line 120, in model_forward
return model(**data, **internal_inputs)
File "/aml2/colo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/aml2/colo/lib/python3.10/site-packages/colossalai/booster/plugin/hybrid_parallel_plugin.py", line 118, in forward
return super().forward(*args, **kwargs)
File "/aml2/colo/lib/python3.10/site-packages/colossalai/interface/model.py", line 25, in forward
return self.module(*args, **kwargs)
File "/aml2/colo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/aml2/colo/lib/python3.10/site-packages/colossalai/shardformer/modeling/llama.py", line 233, in llama_for_causal_lm_forward
outputs = LlamaPipelineForwards.llama_model_forward(
File "/aml2/colo/lib/python3.10/site-packages/colossalai/shardformer/modeling/llama.py", line 100, in llama_model_forward
attention_mask = self._prepare_decoder_attention_mask(
File "/aml2/colo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1269, in getattr
raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'LlamaModel' object has no attribute '_prepare_decoder_attention_mask'
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 4959) of binary: /aml2/colo/bin/python
Traceback (most recent call last):
File "/aml2/colo/bin/torchrun", line 8, in
sys.exit(main())
File "/aml2/colo/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 346, in wrapper
return f(*args, **kwargs)
File "/aml2/colo/lib/python3.10/site-packages/torch/distributed/run.py", line 762, in main
run(args)
File "/aml2/colo/lib/python3.10/site-packages/torch/distributed/run.py", line 753, in run
elastic_launch(
File "/aml2/colo/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 132, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/aml2/colo/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 246, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
pretrain.py FAILED
Failures:
[1]:
time      : 2023-11-12_05:22:03
host      : A200
rank      : 3 (local_rank: 1)
exitcode  : 1 (pid: 4960)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
Root Cause (first observed failure):
[0]:
time      : 2023-11-12_05:22:03
host      : A200
rank      : 2 (local_rank: 0)
exitcode  : 1 (pid: 4959)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
Error: failed to run torchrun --nproc_per_node=2 --nnodes=2 --node_rank=1 --master_addr=A100 --master_port=29500 pretrain.py --d /aml/data/boyang/RedPajama-Data-1T-Sample -p hybrid_parallel -c 7b -b 2 -l 2048 on A200, is localhost: True, exception: Encountered a bad command exit code!
Environment
Ubuntu 18.04
2* 2A10080G

torch 1.13.1
cuda 11.7
colossal ai the latest
BTW: TP and gemini is good, but PP not working as design

And One node for PP is also OK, just for multiple nodes, it could not works

Thank you for your feedback! We will check it later. To train LLaMA-2, I recommend you to try Colossal-LLaMA-2 under the applications folder, which is more compatible with the latest Colossal AI environment.😃(I seems to be some incompatibility between the latest Colossal AI and the /language/llama2, we are fixing it)

Thank you for your feedback! We will check it later. To train LLaMA-2, I recommend you to try Colossal-LLaMA-2 under the applications folder, which is more compatible with the latest Colossal AI environment.😃(I seems to be some incompatibility between the latest Colossal AI and the /language/llama2, we are fixing it)

Thank you for the support, actually I really think examples/language/llama2 is a good design, hope you can fix the multiple node pretrain issue.
Only for this issue...
attention_mask = self._prepare_decoder_attention_mask(
File "/aml2/colo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1269, in getattr
raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'LlamaModel' object has no attribute '_prepare_decoder_attention_mask'
Epoch 0: 0%| | 0/116314 [00:02<?, ?it/s]
Traceback (most recent call last):

Hi, this bug is caused by the structure of change LlamaModel in the newest release of transformers.  You can downgrade transformers to 4.34 and this bug should be fixed.

Hi, which version of the 'transformers' library have you installed? I try to fix this issue.

Hi, this bug is caused by the structure change of LlamaModel in the newest release of transformers. You can downgrade transformers to 4.34 and this bug should be fixed.

OK, I'll try and give the feedback

Hi, this bug is caused by the structure change of LlamaModel in the newest release of transformers. You can downgrade transformers to 4.34 and this bug should be fixed.

After change to 4.34 , fix it

您好，这个 bug 是由于LlamaModel最新版本的transformers.您可以降级transformers到 4.34，这个错误应该会得到修复。

改成4.34后修复

ImportError: cannot import name 'insecure_hashlib' from 'huggingface_hub.utils' (/home/test/miniconda3/envs/torch122_cuda121/lib/python3.8/site-packages/huggingface_hub/utils/init.py)改成4.34后又变成这个了，这样怎么解决嗷

@flybird11111 I am using transformers 4.38.2, and in main branch I am still facing the same problem. Could you take a look please?


Clean up multimodule pom and Maven warnings

Some of the issues:

Separate descriptions for the different artifacts (looks better in the generated site).
Fix warnings around scm.url and inheritance of the values it uses.
Fix Javadoc warnings that were introduced in the 3.1.0 release.

Apparently the links between the parent site and the submodule sites are intended to be broken(?):
https://maven.apache.org/plugins/maven-site-plugin/faq.html#Why_dont_the_links_between_parent_and_child_modules_work_when_I_run_mvn_site


Update MaxItems property in ListFunctions API doc

Currently, the ListFunctions API doc says that a customer can supply an integer from 0 to 10000. While this is true, the current maximum that will be returned from the AWS Lambda ListFunctions API is 50. This commit updates the documentation so it is more clear.
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Thanks for the update!


Error

While I was using this I got these errors:
Demo for python-discord-rpc Adobe illustrator
Traceback (most recent call last):
File "example.py", line 7, in
rpc_obj = rpc.DiscordIpcClient.for_platform(client_id)  # Send the client ID to the rpc module
File "/home/runner/discord-rpc/rpc.py", line 52, in for_platform
return UnixDiscordIpcClient(client_id)
File "/home/runner/discord-rpc/rpc.py", line 44, in init
self._do_handshake()
File "/home/runner/discord-rpc/rpc.py", line 59, in _do_handshake
ret_op, ret_data = self.send_recv({'v': 1, 'client_id': self.client_id}, op=OP_HANDSHAKE)
File "/home/runner/discord-rpc/rpc.py", line 107, in send_recv
self.send(data, op)
File "/home/runner/discord-rpc/rpc.py", line 115, in send
self._write(header)
File "/home/runner/discord-rpc/rpc.py", line 200, in _write
self._sock.sendall(data)
OSError: [Errno 107] Transport endpoint is not connected
I feel like I made a stupid mistake while reading it I'll try to fix it.

I ran this on Replit btw.

Don't use online IDE

Hey, Thanks this was the same error. I used on a terminal works fine.


Bug: threaded scheduler is not interruptible

import dask, time
dask.__version__
> u'0.14.1'

# can be interrupted
fun = dask.delayed(lambda x: time.sleep(5))
dask.compute([fun(x) for x in range(100)], get = dask.threaded.get)

# can't be interrupted
fun2 = dask.delayed(lambda x: sum([1 for x in range(10000000)]))
dask.compute([fun2(x) for x in range(100)], get = dask.threaded.get)

Using the threaded scheduler, I noticed that some tasks cannot be interrupted by pressing Ctrl+C in a python console. Directly after pressing Ctrl+C the CPU usage of that process jumps up: It uses all available minus one cores. Judging by the output of top, the process seems to spend the computing time in the kernel space.

Thanks for the bug report. This is due to https://bugs.python.org/issue1360, and is fixed in python 3. A workaround is to always specify a timeout to get (can be extremely long). Fix coming soon. Minimal example:
from time import sleep
from multiprocessing.pool import ThreadPool
from dask.compatibility import Queue

p = ThreadPool()
queue = Queue()
p.apply_async(sleep, args=(10,), callback=queue.put)
queue.get()   # This will block KeyboardInterrupt in python 2.7
# queue.get(1000000)  # This won't

Fixed by #2144.

As usual I am really amazed by your speed. Thanks!


Fix errors from Sonar

Fixes these findings: https://sonarcloud.io/project/issues?id=AzureAD_azure-activedirectory-identitymodel-extensions-for-dotnet&resolved=false&types=BUG
Fixed, simplified, and added tests for Equals and GetHashCode to data transfer objects in M.IM.Xml assembly. Generated methods for Equals and GetHashCode then edited to respect previous behavior for string casing, and expected behavior for collection types for Equals. Added unchecked for GetHashCode as it was another Sonar finding
Removed null checks on value type objects
Moved null check before first access
Fixed NRE condition when calling GetType on a null object

IMO, a few tricks are used for the hashcode, and I'm assuming that the updates for where these objects are used are properly tested with the added tests. What I might be missing for the utests is some usages of the objects in lists/collections for equals/compare to ensure things works as expected there.


SWDEV-401781 - Add check for AutoCleanOnLaunch flag for hipGraph.

Change-Id: Ie9db425163a11c1d34a4424aad7293b2fd081831

Changes have been merged into develop via 8993dd87efdf805570d0039f193eaeafb560a6e5.


[Llama-Index] Investigate why spans for Query Pipelines are disconnected in the new instrumentation but are fine under the legacy callback handler

fixed 5 days ago by https://github.com/run-llama/llama_index/pull/14997


Documentation update, API is changing.

The documentation is at the moment not updated to reflect the API changes made to milsymbol, but it will be updated as soon as all API changes are implemented.
The API is still backward compatible, but you might end up with a lot of things in the console if you are using things that you shouldn't be using, so keep an eye on the console.
The two main things that you need to know are these:

Do not set properties directly any more using symbol.property = 'thingie' instead use symbol.setOptions({property:'thingie'})

You don't need to use getMarker() any more, just remove it from your code, if you have done 1 everything will just keep working, if you don't want to do 1, replace getMarker() with setOptions() and you will get rid of the console warnings.

Most documentation and standard documents are now updated, next step is to update examples.


Plugin not working

Hello your plugin don't work with me

Hi
can you provide a bit more context?
The error log for example.

I don't have error

I think the socket is not listening to the fastify port


Fade In Chunks

Mod name
Fade In Chunks
Curseforge link
https://www.curseforge.com/minecraft/mc-mods/fade-in-chunks#c24
Modrinth link
https://modrinth.com/mod/fade-in-chunks
Other link
https://github.com/Johni0702/fade-in-chunks
What it does
Adds a (Bedrock-style) fade-in animation to newly loaded chunks.
Why should it be in the modpack
Adds parity between Java Edition and Bedrock Edition
Why shouldn't it be in the modpack
Doesn't add much and requires fog to work correctly.
Categories

[ ] Performance optimization
[x] Graphics optimization
[ ] New feature
[ ] Optifine parity
[ ] Fixes a bug/dependency
[ ] Replaces an existing mod

Additional details
Maybe have this as an optional mod.

I think the chunk loading should be subtle, not flashy - this looks very distracting, especially on shorter distances.
One could also compare that with the current way, video made by one of the pack's users.

IMHO this isn't flashy, and it replaces the chunks suddenly popping into existence with a much smoother fade.

It also ave compaibility problems

with sodium


Add NumberOfWordsInQuotes primitive

Determine the number of words in single/double quotes

@gsheni @thehomebrewnerd
I think the semantics for this primitive are interesting. Would this primitive match for the following sentence:
"word word word"
And return 3?
Or, do we just want to match words immediately wrapped by quotes, like the following:
Word "word"
And return 1.
The first approach enables the user to ask more interesting questions, like "how many words of dialogue were spoken in this text?" However there are a number of edge cases to be careful of here.

I would say the first approach is what I had in mind.
Regarding the edge case, I agree with you, we do count it if its enclosed in single/double quotes.


Okta Auth support

Proposed changes
Okta Auth support

Types of changes

New feature (non-breaking change which adds functionality)

Linked Issues
https://github.com/rancher/rancher/issues/15574

Further comments

LG2M
Can you add a reference to the corresponding backend PR so I can keep track of when that is merged?

I will link to it later as the backend PR is not submitted yet

@loganhz is this ready for merge or not? if not can you remove the label until the backend is merged?

Sure.
Link to the backend PR
https://github.com/rancher/rancher/pull/17236


Front-End UI Issue

Describe your issue here
I found a front-end UI Issue, the following can not scroll the page to follow the drop-down bar.
Browser chrome, Such as the following:

Useful
Info

Versions
Rancher v1.0.0 Cattle: v0.159.2 UI: v0.100.3

Access
localauth admin

Route
splash

@Chenpipi - I am unable to reproduce this.  Can you please let me know if this happens every time you create a new environment or was it just the first time you setup v1.0?

@tfiduccia -- Not the first time there will be set up, but sometimes appear, and two computers have suffered this problem.
When I add a host will appear, when I chose on the third step of：

@Chenpipi
There have been some changes to the footer since 1.0.0.  Have you upgraded to newer version and are you still seeing this problem?


Don't write build cache to emptydir

Previously, when no cache-volume was specified the caching-data got written to an EmptyDir.
EmptyDirs are thrown away once the Pod is finished.
For some buildpacks/apps (e.g. gradle when building spring-music) generating the cache takes a considerable amount of time. If we know that we will throw it away anyway we can skip that.
Therefore we:

Don't pass the --cache-dir arg to the lifecycle steps
Don't add the cache-dir to the VolumeMounts
Don't create an EmptyDir for the cache

Tagging my pairs: @bodymindarts @Haegi

@phil9909 could you rebase real fast?

Codecov Report

Merging #658 (e515453) into master (ce535db) will increase coverage by 0.07%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #658      +/-   ##
==========================================
+ Coverage   66.93%   67.00%   +0.07%
==========================================
  Files          89       89
  Lines        3632     3640       +8
==========================================
+ Hits         2431     2439       +8
  Misses        851      851
  Partials      350      350

Impacted Files
Coverage Δ

pkg/apis/build/v1alpha1/build_pod.go
97.63% <100.00%> (+0.04%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ce535db...e515453. Read the comment docs.

@phil9909 could you rebase real fast?

Sorry, I'm in a different timezone and had already started my weekend.
Rebase is done.


Fix Fedora CI gating issues

This issue is compiling the progress on the Fedora CI gating issues:

[ ]  IPv4/IPv6 tests are failing due to missing dnsmasq.
[ ] OpenvSwitch tests are failing due to missing openvswitch service.
[ ] NetworkManager-team is not being installed for some reason.
[ ] Warning due to pytest version because unregistered custom marks.

This task needs to be split into several..


#5333: remove bower.io, use local version of webpack

fixes: https://github.com/twosigma/beakerx/issues/5333

👍
resolve conflicts pls


Improve resolver speed

What does this PR try to resolve?
This PR improves the resolver speed after investigations from https://github.com/Eh2406/pubgrub-crates-benchmark/issues/6.
How should we test and review this PR?
Commit 1 adds a test showing a slow case in the resolver, where resolving time is cubic over the number of versions of a crate. It can be used for benchmarking the resolver.
Comparison of the resolving time with various values of N=LAST_CRATE_VERSION_COUNT and C=TRANSITIVE_CRATES_COUNT:

N=100
N=200
N=400

C=1
0.25s
0.5s
1.4s

C=2
7s
44s
314s

C=3
12s
77s
537s

C=6
30s
149s
1107s

C=12
99s
447s
2393s

Commit 2 replaces the default hasher with the hasher from rustc-hash, decreasing resolving time by more than 50%.
Performance comparison with the test from commit 1 by setting LAST_CRATE_VERSION_COUNT = 100:

commit
duration

master
16s

with rustc-hash
6.7s

Firefox profiles, can be read with https://profiler.firefox.com:
perf.tar.gz
r? Eh2406

Absolutely love the switch to rustc-hash, https://github.com/Eh2406/pubgrub-crates-benchmark/issues/6#issuecomment-2403545404 this is a huge speed up with no downside.

Should we split the test and rustc-hash commits out into its own PR and get that merged while we work through nohash?

I splitted to nohash commit in #14665.

@bors r+

:pushpin: Commit 93db5bfe59c2971485b15c708b1bf1711bd18bd3 has been approved by Eh2406
It is now in the queue for this repository.

:hourglass: Testing commit 93db5bfe59c2971485b15c708b1bf1711bd18bd3 with merge 6d679d3e81ad298278e4d9d0dbc4c70844d102cb...

While it doesn't seem like this part of code is under the risk of hash collision, there is a concern around the other part in Cargo, see https://github.com/rust-lang/cargo/issues/13171#issuecomment-2181266802.
I also did some exploration of blake3 as a stable hash algorithm candidate, and blake3 has become a dependency since #14137. Have we benchmarked that as well?

All of these hashes are being used exclusively in memory. So the concerns about consistency between builds let alone consistency between architectures are not relevant here. Happy to see experimentation with other algorithms though.

:broken_heart: Test failed - checks-actions

I rebased on master.

@bors r+

:pushpin: Commit 3d4d48b0ff5f2822607581446213865af1f3b75f has been approved by Eh2406
It is now in the queue for this repository.

:hourglass: Testing commit 3d4d48b0ff5f2822607581446213865af1f3b75f with merge 7aa7fb1e2f9a213850a851151a3c8176f436020a...

:sunny: Test successful - checks-actions
Approved by: Eh2406
Pushing 7aa7fb1e2f9a213850a851151a3c8176f436020a to master...


Guild List Size User Option

Could you add the ability to change the width of the list to the options section?

Not really possible because of how guilds are structured

How about a "switch" between your version of the guild list and the standard?
if remove these parameters, then everything is restored back
.theme-dark .wrapper-1_HaEi .expandedFolderBackground-1kSAf6 {
background-color: rgba(255, 255, 255, 0.0605);
border-radius: 4px;
}
.theme-dark .platform-win .scroller-3X7KbA {
}
.theme-dark .wrapper-1_HaEi {
}
.theme-dark .wrapper-1_HaEi .listItem-3SmSlK:hover .pill-2RsI5Q:empty {
}
.theme-dark .wrapper-1_HaEi .listItem-3SmSlK {
}


A referencing issue.

This issue references another issue, namely #1

It should also be possible to reference by exact url: https://github.com/sourcecred/example-repo/issues/6

We might also reference individual comments directly.
https://github.com/sourcecred/example-repo/issues/6#issuecomment-373768538

Here's a PR by direct url: https://github.com/sourcecred/example-repo/pull/5

a PR review by url: https://github.com/sourcecred/example-repo/pull/5#pullrequestreview-100313899

a PR Review Comment by url: https://github.com/sourcecred/example-repo/pull/5#discussion_r171460198

a user by url: https://github.com/wchargin

Here are several references:
#1
#2
#3
https://github.com/sourcecred/example-repo/pull/5#discussion_r171460198
https://github.com/sourcecred/example-repo/pull/5#pullrequestreview-100313899

This comment has no references.


Update native logging docs

This should me part of docs Mx9.18.1

This is still a work in progress and needs review from the team once we're done, we'll unmark it as a draft

This was closed because the changes weren't in place before the 9.18 freeze. Thus, this doc update will happen in 9.19.

I had edited this text and it's good to merge from my evaluation. If 9.18.1 happens when I'm on vacation, please give the PR one final language check and then merge it. I'll check the merged content with Osama when I return.

Hey @omnajjar is this still a draft or is it ready for release day?

Hey @omnajjar is this still a draft or is it ready for release day?

@ConnorLand it is ready for release.


[CST-4009] Discovery result sort options not reflecting what is configured

References

Fixes https://github.com/DSpace/dspace-angular/issues/1071

Description
As described in the related issue, sort options aren't provided correctly to the SearchSettingCompontent, which is only provided with fixed default values.
It has been added a SearchConfig model and a method to fetch the search configuration given scopes and configuration names.
Then this information is propagated by the root component (MyDspacePageComponent) to the child components.
Instructions for Reviewers
Navigate to the MyDspace and check the available sort options are correct.
Checklist
This checklist provides a reminder of what we are going to look for when reviewing your PR. You need not complete this checklist prior to creating your PR (draft PRs are always welcome). If you are unsure about an item in the checklist, don't hesitate to ask. We're here to help!

[x] My PR is small in size (e.g. less than 1,000 lines of code, not including comments & specs/tests), or I have provided reasons as to why that's not possible.
[x] My PR passes TSLint validation using yarn run lint
[x] My PR doesn't introduce circular dependencies
[x] My PR includes TypeDoc comments for all new (or modified) public methods and classes. It also includes TypeDoc for large or complex private methods.
[x] My PR passes all specs/tests and includes new/updated specs or tests based on the Code Testing Guide.
[ ] If my PR includes new, third-party dependencies (in package.json), I've made sure their licenses align with the DSpace BSD License based on the Licensing of Contributions documentation.

This is not a super clean solution for addressing the ticket, but the only one affordable in terms of time to fix this in beta5. This is due to the previous way to retrieve defaults and to update search params subsequently to route changes. In particular almost all the logic to fetch and initialize the sort options has been bounded in the MyDspacePageComponent initialization, but it is clear that should belong to the SearchConfigurationService and every component should manage its initialization taking this async into account (impact  in many components and their test suites).

The defaults are computed synchronously and used by every component which is provided by its search configuration service. The logic should be asynchronous and so every components needs to change the way they initialize themselves. This should be an overall refactoring and needs a separate ticket.
This can be implemented in the context of fixing another big design problem of the SearchConfigurationService which therefore already needs some design refactoring. The problem is the moltiplication of search rest call when more than one queryparam changes after filter/facet/sort selection. A way to test is to change the sort options from score/ASC to title/DESC, two request are performed (title/ASC and then title/DESC).

Also, since the rest endpoints provide as sortOptions just the list of the fields, the generation of the available options now happens as the following (open to discussion):

Rest order give the displayed priority.
Every field has by default both direction available, ASC and DESC (first displayed ASC).
So for example:
sortOptions: [{name: "dc.title"}, {name: "dc.date.issued"}, {name: "score"}]
lead to options like this:
dc.title.ASC, dc.title.DESC, dc.date.issued.ASC, dc.date.issued.DESC, score.ASC, score.DESC

With this approach, if there was the necessity to exclude an option (for instance score ASC), it would be filtered directly angular side with a deny list.
To conclude, the Rest Contract of the discover/search is not accurated and need to be allineated to the actual rest implementation. Since we now follow the rest order, we noticed also that the default configured option (in our case score) is appended at the end instead of being inserted at the beginning of the list. This is something that should be fixed as well.

Thanks @alemarte!

This is not a super clean solution for addressing the ticket, but the only one affordable in terms of time to fix this in beta5. >This is due to the previous way to retrieve defaults and to update search params subsequently to route changes. In particular almost all the logic to fetch and initialize the sort options has been bounded in the MyDspacePageComponent initialization, but it is clear that should belong to the SearchConfigurationService and every component should manage its initialization taking this async into account (impact in many components and their test suites).

The defaults are computed synchronously and used by every component which is provided by its search configuration service. The logic should be asynchronous and so components need to change the way they initialize themselves. This should be an overall refactoring and needs a separate ticket.

With this PR the sort options only work in mydspace though, was that your point? The regular search page also has sortOptions, yet none are shown with this PR.

The problem is the moltiplication of search rest call when more than one queryparam changes after filter/facet/sort selection. A way to test is to change the sort option from score/ASC to title/DESC, two requests are performed (title/ASC and then title/DESC). See subscribeToSearchOptions and subscribeToPaginatedSearchOptions

I struggled to reproduce this at first: the thing is that Relevance (i.e. score) is already sorted DESC, so you should switch from Relevance to Title ascending to reproduce it. I can confirm that this also happens on the demo site, and is therefore not caused by this PR.

Also, since the rest endpoints provide as sortOptions just the list of the fields, the generation of the available options now happens as the following (open to discussion):

Rest order gives the displayed priority.
Every field has by default both direction available, ASC and DESC (first displayed ASC).
So for example:
sortOptions: [{name: "dc.title"}, {name: "dc.date.issued"}, {name: "score"}]
leads to options like this:
dc.title.ASC, dc.title.DESC, dc.date.issued.ASC, dc.date.issued.DESC, score.ASC, score.DESC

With this approach, if there was the necessity to exclude an option (for instance score ASC), it would be filtered directly angular side with a deny list.

That will do for now, but in my opinion rest should also provide the available sort directions eventually.

To conclude, the Rest Contract of the discover/search is not accurated and need to be allineated to the actual rest implementation. Since we now follow the rest order, we noticed also that the default configured option (in our case score) is appended at the end instead of being inserted at the beginning of the list. This is something that should be fixed as well.

Is that something that can be fixed with configuration on the rest side? A better default config might be all we need here?

With this PR the sort options only work in mydspace though, was that your point? The regular search page also has sortOptions, yet none are shown with this PR.

@artlowel we've added the code to retrieve sortoptions also for the regular search page

@tdonohue @artlowel thanks for your feedback!

@alemarte and @atarix83 : I tested this today as well. I'm able to reproduce the same issues that @artlowel mentions.
Additionally, the behavior of the default search is odd, as the "Sort By" says it's sorting by "Title Ascending" but the results themselves seem to be sorted by relevance. Here's what I see:

(I'm logged out) In the header, click the search icon. Type in "test"
You'll see the search results. The "Sort By" says "Title Ascending" (though for the results I see they do not seem ordered by title, they look sorted by relevance to me.)
Click on page 2. The "Sort By" changes to "Relevance Ascending".  But, in the address bar of my browser it now says: &sortDirection=ASC&sortField=dc.title
Click Reload in the browser, Now your page of results changes, as it reorders results by "Title Ascending"

I've committed a change that add sort field and sort direction to the active route at the initialization of SearchComponent and MyDspacePageComponent. With this change every request seems correct (even those involving a page change).
(During this tiny refactor I've also moved the initialization logic to the service, in this way we don't have so much code duplication).

It looks, to me, like the "Sort By" field is not working properly or caching values at some times. This behaviour is also similar to the issue @artlowel reported (and I can reproduce), that if you change the sort several times, the "Sort By" value is not always correct.

I noticed this but forgot since I've worked almost immediately upon https://github.com/DSpace/dspace-angular/pull/1048
and that PR seems to fix this. In fact after merging with main I wasn't able to reproduce it.

Finally, in the "Sort By" dropdown I'm seeing some missing i18n keys for sorting.dc.date.accessioned.ASC and sorting.dc.date.accessioned.DESC

Done

Based on discussion in today's meeting, we've moved this ticket to the "7.0" release board. This means this work is still under review but is unlikely to be in Beta 5...mainly because the user experience with this PR isn't the greatest until we find a way to solve the REST API bugs described in https://github.com/DSpace/DSpace/issues/3227.  Therefore, we don't want to deploy this to the Testathon servers until the REST API bug is also solved (but as soon as both are ready, they will be deployed together).

@tdonohue @artlowel correct me if I'm wrong, but the server with DSpace/DSpace#3241 only provides the default direction. So we must assume that both are always available. This is something that maybe @abollini can easily confirm.
Also we made this decision to keep the alternative directions with the pattern ASC/DESC because when you receive mixed defaults from the server the select list looks confusing and messy. So summing all up, you receive from the server something similar to this:
sortOptions: [
  {name: "score", sortOrder: "DESC" },
  {name: "dc.title", sortOrder: "ASC" },
  {name: "dc.date.issued", sortOrder: "ASC"}
]

and the UI displays the selection like this (bold is the selected option)

score ASC
score DESC
dc.title ASC
dc.title DESC
dc.date.issued ASC
dc.date.issued DESC

A default direction per sort option has very limited use in my opinion. You could say it's relevant to configure that for score the default should be descending, but then again score ascending is completely useless and we should have a way to leave it out. The whole reason I noticed it was because now we have the sort option "Least relevant" on the search page. You'd have to come up with a very creative query to make use of that 😄
But if it was indeed the goal of the rest api to add a default direction per field, rather than to add all possible field and direction combinations separately, then this is something we should discuss on the rest side, and I'd just merge this PR as it is now and address it in a later PR. So it would be good if @abollini could weigh in here to clarify the intention.

@abollini : Could you weigh in here & verify what I'm seeing? @artlowel brings up some good points...but, it also seems like (to me) it might be a misunderstanding of what the REST API changes were meant to achieve?
Here's what I see:

The REST API PR (https://github.com/DSpace/DSpace/pull/3241) ensures that sort options come back as configured. Each sortOption now has its own sortOrder (which is set to the configured defaultSortOrder for that field as defined in discovery.xml).  This was redesigned from discovery.xml having a single, global "defaultSortOrder".
In this Angular PR, it appears it appears we are doing this:

Determining the first sortOption returned and using it's sortOrder to determine the default sort of the results.
Listing all other sortOptions (ignoring their sortOrder)... so, it appears the only sortOrder that matters to the UI is the one set on the first sortOption

It appears to me that we've essentially just "redefined" how the global "defaultSortOrder" is/was configured/returned.  It used to be a separate setting in discovery.xml.  But, now that global "defaultSortOrder" is just the first sortOption combined with its sortOrder.  Is this accurate?
Overall, I agree with @artlowel that this is a bit confusing, as the defaultSortOrder settings in the discovery.xml really don't matter for most fields.  But, I don't have a better, immediate solution (without a larger overhaul of the discovery.xml config).
All in all, I think we all agree that this PR might be best to go forward as-is.  I just wanted to see if we first need to create any follow-up tickets for further cleanup.

hi,
the summary from @alemarte is accurate. The analysis of the angular behavior from @tdonohue also looks correct and reasonable to me.
On the REST side we now expose the configuration exactly as it is. Informing the client about which sort options exist and which is the preferred direction for each sort option.
How this information is used by the client, angular in our case, is mostly a decision of UX. For sure the client must use the first sort option and direction preference to inform the user about which is the current sorting when a search is performed without making explicit request about the sorting.
For the other sort options if as in our current angular application all the options / direction combinations are listed the preferred direction has limited or no value but in other UI where for instance the sort criteria is set clicking on a column it could allow to use the best direction for each criteria (still allowing the user to change direction if needed).
Think about a table layout where we show
score | title | date
when the user perform a default query we will get score desc
the user click on title, he want to get staff sorted by title asc but when he click on date usually he prefers to get the result sorted by date desc
this is what the REST configuration inform you.
We have explicitly avoid to constraint a sort field to a single direction because this was

not supported on the REST side
should require enforcement on the REST side to be well designed that in turn could limit administrative/unexpected use cases
can be hard-coded on the angular side if really needed and related to UX as it was before (for the score sorting it was hardcoded to offer only a DESC option)

So my recommendation is to merge the current PRs as is or additionally hardcode the exclusion of the score ASC option only on the angular side (in this PR). If instead we really want to enforce/comunicate such constraint from the REST side a separate ticket is more appropriate

Thanks @abollini for the summary.  I agree that the REST API implementation does allow for more rich client implementations, even if our current client implementation is more limited in nature (but still a great start!).
As we have approvals on both frontend & backend PRs, I'm going to move forward with merging both.  It doesn't sound like we've identified any required follow-up work yet, but the discussion in this PR helps clarify the reason(s) for the change & future UI implementations we could someday consider.


Pyspark example

Hi,
Can you please provide a sample code for using it in pyspark? I'm trying to write a dataframe into an Exasol table, I'm currently using JDBC but it does not allow me to create the table if not exist.
df.write
.format("jdbc")
.option("driver", "com.exasol.jdbc.EXADriver")
.option("url", URL)
.option("dbtable", "mytable")
.option("user", "myuser")
.option("password", "mypass")
.option("useSSL", "false")
.mode('overwrite')
.save()

@mabdelrazzak
You may need to include the option : .option("create_table","true")
Also you need Spark Version 2.1+.
Try this example and get back to us:
shell% git clone https://github.com/exasol/spark-exasol-connector.git
shell% cd spark-exasol-connector
shell% ./sbtx assembly
shell% pyspark --master yarn --deploy-mode client --jars ./target/scala-2.11/spark-exasol-connector-assembly-0.2.0.jar --num-executors=32
...
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /__ / .__/\_,_/_/ /_/\_\   version 2.2.1
      /_/

....
>>> from pyspark.sql import *
>>> Employee = Row("firstName", "lastName", "email", "salary")
>>> employee1 = Employee("john","doe","jdoe@github.com",100000)
>>> employee2 = Employee("mary","smith","msmith@github.com",120000)
>>> employees = [employee1 , employee2]
>>> employees_dataframe = spark.createDataFrame(employees)
>>> exasol_host = "<IP_ADDRESS>"
>>> exasol_port = "8563"
>>> exasol_password = "12345"
>>> exasol_user = "jim"
>>> employees_dataframe.write.mode("overwrite").option("host",exasol_host).option("port",exasol_port).option("username",exasol_user).option("password",exasol_password).option("table","OV_AS_ACCESS_LAB.EMPLOYEE").option("create_table","true").format("exasol").save()

>>>

and then in ExaPlus GUI I see
SELECT *
FROM OV_AS_ACCESS_LAB.EMPLOYEE
;

FIRSTNAME	LASTNAME	EMAIL	SALARY
john	doe<EMAIL_ADDRESS>100000
mary	smith<EMAIL_ADDRESS>120000

Hello @mabdelrazzak,
Yes, you can use the setup and example provided above by @jpizagno.
Similarly, if you want to skip assembling the jar, you can use these two lines with pyspark command:
--repositories https://maven.exasol.com/artifactory/exasol-releases \
--packages com.exasol:spark-connector_2.11:0.2.0

I have written short walkthrough example, it is mostly for Scala but should work with PySpark also. Please find it here.
In you code example, you are using the general JDBC connector which is built in into Spark. However, we cannot support you very much there. Give a try to this Spark Exasol connector, where we are happy to support you!
Please do not hesitate to let us know if you have any other questions or feedback!

Hello,
Thanks for your feedback(s).
I have followed the first example: @jpizagno
employees_dataframe.write
...         .format("exasol")
...         .option("host", "myserver")
...         .option("table", "Test_TBL")
...         .option("port", "8563")
...         .option("username", "myuser")
...         .option("password", "mypassword")
...         .option("useSSL", "false")
...         .option("create_table", "true")
...         .mode('append')
...         .save()
And it gave me the below error:
com.exasol.jdbc.ConnectFailed: connect timed out
And when I'm using PyCharm, I get the following:
java.lang.ClassNotFoundException: Failed to find data source: exasol.
In Pycharm, it uses the following command to run my script:
/usr/local/bin/python3.7 /Users/myuser/PycharmProjects/MyProject/Test_Spark_LoadToExasol_withExasol_connctor.py --master yarn --deploy-mode client --jars /usr/local/lib/python3.7/site-packages/spark-exasol-connector/target/scala-2.11/spark-exasol-connector-assembly-0.2.0.jar --num-executors=32
I have tried solution # 2 @morazow
I got the below exception:
Traceback (most recent call last):
File "", line 10, in
File "/usr/local/Cellar/apache-spark/2.4.0/libexec/python/pyspark/sql/readwriter.py", line 734, in save
self._jwrite.save()
File "/usr/local/Cellar/apache-spark/2.4.0/libexec/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in call
File "/usr/local/Cellar/apache-spark/2.4.0/libexec/python/pyspark/sql/utils.py", line 63, in deco
return f(*a, **kw)
File "/usr/local/Cellar/apache-spark/2.4.0/libexec/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 328, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o57.save.
: java.lang.NoClassDefFoundError: org/apache/spark/sql/sources/RelationProvider

Hey @mabdelrazzak,
Thanks for trying out different approaches!
I think we are more close to getting it working on the first error :) On other cases it seems like the dependent libraries of connector are not loaded.
Could you please check that you can connect on ports in range [20000..21000] to Exasol database?
The connector uses Exasol sub-connections which return a sub connection url and port in above range that each Spark tasks needs to connect.

@mabdelrazzak
Thanks for trying our solutions. Just to explain my suggestion, the parameter "--num-executors=32" may be important.  If I do not use the correct number, then I also get connection problems.
How many nodes are in your Exasol Cluster ?
(this is related to @morazow 's answer)
There is not a 1-1 mapping, but the number of Spark executors may need to be greater (or equal to) the number of Exasol Nodes.
Maybe @morazow can elaborate.
thanks again

Thanks so much @morazow @jpizagno This is really helping, I have check with our support team and the ports range 20000:21000 were not open and it took them sometime to open it, but now all is open (hopefully :) )
Now the exception has changed:
java.lang.NoSuchMethodError: com.exasol.jdbc.EXAConnection.EnterParallel(I)I
at com.exasol.spark.util.ExasolConnectionManager.initParallel(ExasolConnectionManager.scala:55)
at com.exasol.spark.writer.ExasolWriter.startParallel(ExasolWriter.scala:48)
at com.exasol.spark.DefaultSource.saveDFTable(DefaultSource.scala:130)
at com.exasol.spark.DefaultSource.createRelation(DefaultSource.scala:98)
at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:45)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
I have done the following:

Added: spark-exasol-connector-assembly-0.2.0.jar under: ~/spark-2.3.2-bin-hadoop2.7/jars/
Added: --master yarn --deploy-mode client --num-executors=32 to the parameters list in PyCharm Run/Debug Configuration
Using Python 3.7
Using Pyspark 2.3.2
PyCharm is executing the following to run the script:
/Test_Spark.py --master yarn --deploy-mode client --num-executors=32
I'm using the following to write data to exasol table:
employees_dataframe.write
.format("exasol")
.option("host", "exasolServer")
.option("table", "DEV_MO.PAYMENTS22")
.option("port", "8563")
.option("username", "myuser")
.option("password", "mypass")
.option("useSSL", "false")
.option("create_table", "true")
.mode('append')
.save()

@mabdelrazzak
I am not sure the project here (spark-exesol-connector) is supporting pycharm. Try using pyspark or spark-submit.
That message sounds like the jar was not found in: ~/spark-2.3.2-bin-hadoop2.7/jars/.

Check that your environment variable SPARK_HOME is set to ~/spark-2.3.2-bin-hadoop2.7.
I would suggest including the spark-exasol-connector-assembly-0.2.0.jar in the --jars parameter for a pyspark or spark-submit command (like my example above).
To give some technical details, the error ("java.lang.NoSuchMethodError: com.exasol.jdbc.EXAConnection.EnterParallel(I)I") can be caused when the code does not find the JDBC Jar, which is packed in the spark-exasol-connector-assembly-0.2.0.jar. So I think spark-exasol-connector-assembly-0.2.0.jar is not being found.

Hello @jpizagno
I can connect to the DB through my script and the table is created successfully but no data is being loaded and I'm still getting the same exception, Exasol cluster has only 2 nodes. I'm getting the same exception in both PyCharm and PySpark terminal.
The jar file "spark-exasol-connector-assembly-0.2.0.jar" is already placed in:
/Library/Java/JavaVirtualMachines/jdk1.8.0_202.jdk/Contents/Home/jre/lib/ext
And
~/spark-2.3.2-bin-hadoop2.7/jars
Am I still missing any jars?
: java.lang.NoSuchMethodError: com.exasol.jdbc.EXAConnection.EnterParallel(I)I
at com.exasol.spark.util.ExasolConnectionManager.initParallel(ExasolConnectionManager.scala:55)
at com.exasol.spark.writer.ExasolWriter.startParallel(ExasolWriter.scala:48)
at com.exasol.spark.DefaultSource.saveDFTable(DefaultSource.scala:130)
at com.exasol.spark.DefaultSource.createRelation(DefaultSource.scala:98)
at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:45)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)

Good morning @mabdelrazzak,
That is strange error. Maybe exasol-jdbc jar not included in assembled connector jar file.
Are you available today (or next week) to do a live session? Then maybe we can go over your PySpark workflow together.
You can reach me on<EMAIL_ADDRESS>email.
Best

Hello @mabdelrazzak,
Since we were able to solve or recognize the issue, I am going to close this ticket.
Please do not hesitate to open new tickets if you have any questions or issues.


Payment made and received

[x] Amount column keep as float.

[x] Decimal issue

issue fixed !


[OLS4 feedback] - resources logo

In the ontology tables, some resources have logos displayed and others not, https://www.ebi.ac.uk/ols4/ontologies
While I like the idea of logos, I find this distracting here, and as logos are not consistent in look and feel, sizing etc it is hindering the user experience and table consistency.
Options could be removing logos, or putting them in a distinct fixed width column.

Sizing is now reasonably consist