[Epic]  v2 Roadmap Epics

This Epic represents the sum of all Epics that should be included in the v2 of FuseML. Aside from Epics, issues regarding features and enhancements proposals collected from the development of the previous version are also included, some of which could be linked to the existing Epics. Only Epics and stories that will be marked in the list are agreed to be worked in the v2, everything else will be moved back to backlog after the planning meetings.

Epics (in order of decreasing priority)

[x] #161  Installer: add official support to install extensions
[x] #152 NVIDIA Triton Infernece Server
[x] #153 Intel Inference Engine
[x] #21 FuseML Central Recipe Store
[x] #20 Fuseml UI
[x] #19  Design Architecture
[x] #155 Project Promotion and Community building

Features

[x] #69 Workflow templates. Related to #151 #150 #21. (epic candidate)
[x] #74 Registry to store fuseml extensions (i.e. MLFlow URL, S3 backends) . Related to #21. (epic candidate)
[x] #68 Persistent state for fuseml components (epic candidate)
[x] #120 Advanced workflow life-cycle actions

Enhancements

[x] #121 Terminate obsolete workflow runs automatically
[x] #116 CLI: Live-watch a running workflow
[x] #162 MLFlow extension: reuse previously built environment image. Related to #152 #153.

V2 has been released


Bug Report: Lynx Gets Installed Instead of Vivaldi

Issue: When selecting Vivaldi for installation, Lynx gets installed instead.
Steps to reproduce:

Select Vivaldi for installation.
Proceed with the installation.
Notice that Lynx is installed instead of Vivaldi.

Expected behavior: Vivaldi should be installed when selected.
Actual behavior: Lynx is installed instead of Vivaldi.

Fixed in #718

/label 'duplicate'
/close


Update langs

Updated a few langs:

Elixir
Erlang
Gleam
HEEx
Lua
CSS
HTML
Javascript
Python
Ruby
SQL

Tried to update Rust as well but it failed loading so I'll leave for another day

Ugh, looks like a merge conflict that GitHub doesn't feel like working with. Let me get you a patch file so you don't have to download everything again.

That's due to the other PR. I can fix it no worries :)

Those files were deleted in the other PR and update in this one. The resolution is deleting those 2 files

Whoops, didn't catch that. These paths look too similar.

Thanks again! I really should set up a reminder to update the languages every few months or so...

v0.10.4 published!

Thank you!


Setup  testing thru Browserstack

BrowserStack is device farm which allows compatibility testing, including  fully automated.
BrowserStack is free for open-source projects.
https://www.browserstack.com/accounts/subscriptions
Thanks

@unclepaul84 Thanks for the suggestion. We are aware of Browser stack. We will keep this under consideration


error trying to use the plugin

Hi! I want to use your addon but everytime that I tried Im seeing the same error
java.lang.RuntimeException: Unable to start activity ComponentInfo{org.nativescript.RPSConferenceApp/co.fitcom.SplashScreen}: com.tns.NativeScriptException:
Calling js method onCreate failed

Error: java.lang.IllegalStateException: You need to use a Theme.AppCompat theme (or descendant) with this activity.
    android.support.v7.app.AppCompatDelegateImplV7.createSubDecor(AppCompatDelegateImplV7.java:343)
    android.support.v7.app.AppCompatDelegateImplV7.ensureSubDecor(AppCompatDelegateImplV7.java:312)
    android.support.v7.app.AppCompatDelegateImplV7.setContentView(AppCompatDelegateImplV7.java:277)
    android.support.v7.app.AppCompatActivity.setContentView(AppCompatActivity.java:140)
    com.viksaa.sssplash.lib.activity.AwesomeSplash.initUI(AwesomeSplash.java:62)
    com.viksaa.sssplash.lib.activity.AwesomeSplash.onCreate(AwesomeSplash.java:55)
    com.tns.Runtime.callJSMethodNative(Native Method)
    com.tns.Runtime.dispatchCallJSMethodNative(Runtime.java:861)
    com.tns.Runtime.callJSMethodImpl(Runtim

The error below happens when you close the app and you try to open it again. The first time that the app runs doesn't show the splash screen. I already follow all the steps to make it works but now I dont have any idea, thanks

I'm using nativescript 2.1.0
Node v6.2.2
WIndows 10

If you don't mind can you share the SplashScreen.ts file i need to see what updates you made that is causing the problem

Of course, please see it
@JavaProxy("co.fitcom.SplashScreen")
export class SplashScreen extends com.viksaa.sssplash.lib.activity.AwesomeSplash {
    protected onCreate(bundle) {
        super.onCreate(bundle);
    }

    initSplash(configSplash) {

        /* you don't have to override every property */

        //Customize Circular Reveal
        configSplash.setBackgroundColor(org.nativescript.RPSConferenceApp.R.color.md_blue_900); //any color you want form colors.xml
        configSplash.setAnimCircularRevealDuration(2000); //int ms
        configSplash.setRevealFlagX(com.viksaa.sssplash.lib.cnst.Flags.REVEAL_RIGHT);  //or Flags.REVEAL_LEFT
        configSplash.setRevealFlagY(com.viksaa.sssplash.lib.cnst.Flags.REVEAL_BOTTOM); //or Flags.REVEAL_TOP

        //Choose LOGO OR PATH; if you don't provide String value for path it's logo by default

        //Customize Logo
         configSplash.setLogoSplash(org.nativescript.RPSConferenceApp.R.drawable.splash_logo); //or any other drawable
         configSplash.setAnimLogoSplashDuration(2000); //int ms
         configSplash.setAnimLogoSplashTechnique(com.daimajia.androidanimations.library.Techniques.Bounce); //choose one form Techniques (ref: https://github.com/daimajia/AndroidViewAnimations)

        const title = com.tns.NativeScriptApplication.getInstance().getString(org.nativescript.RPSConferenceApp.R.string.splash_title)
        //Customize Title
        configSplash.setTitleSplash(title);
        configSplash.setTitleTextColor(org.nativescript.RPSConferenceApp.R.color.md_white);
        configSplash.setTitleTextSize(30);
        configSplash.setAnimTitleDuration(3000); //int ms
        configSplash.setAnimTitleTechnique(com.daimajia.androidanimations.library.Techniques.FlipInX);
        configSplash.setTitleFont("app/fonts/RobotoRegular.ttf"); //provide string to your font located in app/fonts/

    }
    animationsFinished() {
        const intent = new android.content.Intent(com.tns.NativeScriptApplication.getInstance().getApplicationContext(), com.tns.NativeScriptActivity.class)
        intent.addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK);
        com.tns.NativeScriptApplication.getInstance().startActivity(intent);
    }
};

and this is my package.json file
{
  "description": "NativeScript Application",
  "license": "SEE LICENSE IN <your-license-filename>",
  "readme": "NativeScript Application",
  "repository": "<fill-your-repository-here>",
  "nativescript": {
    "id": "org.nativescript.RPSConferenceApp"
  },
  "dependencies": {
    "faker": "^3.1.0",
    "nativescript-splashscreen": "^2.0.2",
    "nativescript-telerik-ui": "^1.1.1",
    "tns-core-modules": "2.1.0"
  },
  "devDependencies": {
    "babel-traverse": "6.10.4",
    "babel-types": "6.10.2",
    "babylon": "6.8.2",
    "filewalker": "0.1.3",
    "lazy": "1.0.11",
    "nativescript-dev-typescript": "^0.3.0",
    "typescript": "^1.8.10"
  }
}

any idea?

Sorry about late reply i will try to reproduce

Thanks! I This link contain the full project

I will check it out :)

After checking out your source it seems like the styles.xml in app/App_Resources/Android/values  and app/App_Resources/Android/values-v21 is causing the problem remove them and run again

Hi @triniwiz, thanks, now it shows the splash screen but it also showed when you hit the back button, the splash screen again as a wallpaper, I will apply this fix to see if it works

The fix will work i added it to the readme

It works but now the app is unstable, when I delete the style files from those directories the app works abnormally sometimes works and sometimes doesn't, thanks anyway I think I will pass from trying to add the splash screen for now and I will concentrate to the functional part of the app


Support customisation of logged payload

As discussed, it would be good to be able to customise the formatting of the request/response body before logging.
My particular use case is the need to obfuscate passwords sent in the request before writing to the log.
As a very first stab at this, how about allowing the LoggingInterceptor to be constructed with a LoggingFormatter, something like:
public interface LoggingFormatter {

	String formatRequest(HttpRequest request, byte[] body);

	String formatResponse(ClientHttpResponse response);
}

with a default implementation:
public class DefaultLoggingFormatter implements LoggingFormatter {

	@Override
	public String formatRequest(HttpRequest request, byte[] body) {
		// tmpl method to generate log line from URL, method, body...
	}

	@Override
	public String formatResponse(ClientHttpResponse response) {
		// tmpl method to generate log line from response status, body...
	}

	protected String formatBody(byte[] body, Charset charset) {
		return new String(body, charset);
	}

	protected Charset getCharset(HttpMessage message)
	{
		// ...
	}
}

This would support resolution of e.g. #2 as well. Then perhaps #1 via introducing something like a supports(MediaType) method to the formatter?
What do you think? Happy to prepare a PR following your feedback.

I like it a lot! It certainly provides a good place to address the other issues as you say. Thanks Ryan :+1:
P.S. A pedant would say LogFormatter as it's not actually doing the logging.

Thanks for the PR!

Do you know when this version will be published to Maven Central?

@jmart1 I've just released 2.0.0, so it should be there shortly. Thanks for the reminder!

Yup its there.  Thank you!!!


Adjust padding and button placements

Move the CTA button so the floating version label doesn’t overlap with the ad. Make the expanded ad more compact to take less vertical space. Hope I don’t make more edit to the ads. 😟

The preview screenshot: https://d.pr/i/0N0SE9

many thanks!


Private registry authorization problems

I'm getting an odd issue with Shepherd.
I have a Gitlab private repository and shepherd updates are failing with...
 Get https://registry.gitlab.com/v2/wizewerx/trebuchet/manifests/latest: denied: access forbidden
Here is my docker-compose:
version: '3.3'
services:
  shepherd:
    image: mazzolino/shepherd:latest
    environment:
      FILTER_SERVICES: label=com.wizewerx.autodeploy
      SLEEP_TIME: 1m
      WITH_INSECURE_REGISTRY: 'true'
      WITH_REGISTRY_AUTH: 'true'
    volumes:
     - /var/run/docker.sock:/var/run/docker.sock:ro
    networks:
     - default
    logging:
      driver: json-file
    deploy:
      placement:
        constraints:
         - node.role == manager

I understand that the registry credentials need to be somehow acquired/passed to Shepherd and the used to login? Is this the case?
If so - where / how do I pass the credentials and where does the docker login happen  - I can't see that it does.
Thanks for your help and for a super utility.

Ok I think I got a working solution...

I logged into the docker swarm master and did 'docker login' - this created a config.json
I copied the config.json and made a config on my swarm with it.
added a config to shepherd and mapped it to /root/.docker/config.json.

Hope this helps someone.
Note: There is a risk that the authentication token will expire at some point.

Yeah, I think this is a unresolved problem. For a permanently working solution, we would probably need to pass the login credentials to the container so it could login to the registry before each update run. That would probably be a really useful change.

@mikesutton It's been a long time, but you might be pleased to know that registry authentication is integrated in the latest version.


Fix typo in README and import error

Hello, I am well interested in offline rl problems. Recently I am implementing my own research idea, and I discovered your wonderful repo, and it really helps me a lot.
However, when I run the code, I found some typos of the running commands in the README.md. Also, in eval() function in the evaluation scripts, it seems that the local import of gym might overwrite the outer gymnasium import. This can make the python interpreter think gym  module is not assigned even if I am not using the Metadrive tasks. The error message is pasted below.
load config from OfflineCarCircle-v0-cost-10/CDT_seed20-3554/config.yaml
load model from OfflineCarCircle-v0-cost-10/CDT_seed20-3554/checkpoint/model.pt
Traceback (most recent call last):
  File "examples/eval/eval_cdt.py", line 103, in <module>
    eval()
  File "anaconda3/envs/OSRL/lib/python3.8/site-packages/pyrallis/argparsing.py", line 158, in wrapper_inner
    response = fn(cfg, *args, **kwargs)
  File "examples/eval/eval_cdt.py", line 42, in eval
    env=gym.make(cfg["task"]),
UnboundLocalError: local variable 'gym' referenced before assignment

I create the PR to fix the issue. The importing error might be my configuration error, and I wonder if it also happens in your side.
Thank you for your time for checking this issue!

Thanks for pointing this out! @Ja4822 and I confirmed that this is indeed a bug for certain env configurations, and your correction works well. We sincerely appreciate your contribution!


POST Multipart: Use charset "UTF-8" instead of "ASCII" as default charset to support internationalization

Situation

1.1 Posting mulitpart requests with form parameters containing unicode chars (outside of the ascii range) results in wrong character representations on the server

Analysis

2.1 In Namespace clj-http.multipart the function
create-multipart-entity

uses the default constructor of the class  org.apache.http.entity.mime.HttpStrictMultipart. This results in using "ASCII" as encoding
2.2 In Namespace clj-http.multipart the multimethod
(defmethod make-multipart-body String ...)
uses the thre argument constructor of the class org.apache.http.entity.mime.content.StringBody
(StringBody. content (ContentType/create "text/plain" Consts/ASCII))
This results in using "ASCII" as encoding while getting the bytes of a form parameter
2.3 Both 2.1 and 2.2 prevents non-ascii characters to be transferred correctly

Solution

3.1 To mitigate 2.1  use the three argumant constructor of HttpStrictMultipart
(MultipartEntity. HttpMultipartMode/STRICT nil (encoding-to-charset "UTF-8")
3.2 To mitigate 2.2  changes the third argument to the constructor of  StringBody:
(StringBody. content (ContentType/create "text/plain" Consts/UTF_8))

Thanks @dakrone , too.


Evaluate replacing frontpage React components with Flask/Jinja/vanilla JS components from ZenodoRDM

In Zenodo RDM we have implemented the RecordsList (recent uploads) component and CommunitiesCarousel in pure Jinja and vanilla JS, for performance reasons. The question is if this should be moved to invenio-app-rdm/invenio-communities instead.
About the ZenodoRDM components
Recent uploads component:

Identical to the React component
Requires backend fetching in the frontend view function

Communities carousel:

Currently only supports 1 item per page, otherwise identical
Requires backend fetching in the frontend view function

Indeed, In V12rc2 still the carousel animation is stuttering because the current React component has unnecessary rerenders. It should be replaced with the Carousel implementation from Zenodo.


Block YouTube related videos

What website(s) would this filter apply to?
youtube.com
What content should be blocked?
Related videos on the right side of the video.
What options should be offered?
No response
Did you write a custom uBlock filter already?
www.youtube.com###related

Additional notes
No response

Thanks for your contribution :bow:
I merged both filters in https://letsblock.it/filters/youtube-recommendations


Bot freezes and becomes unresponsive after user reply in conversation

Below is the relevant code snippet:
bot.use(session({
  initial: initialSession,
  storage: new FileAdapter({
      dirName: 'sessions',
    }),
  }))

bot.use(conversations())

const myConversation= async (conversation, ctx) => {
  const { message_id } = await ctx.reply('Please enter the address');
  const { msg } = await conversation.waitFor('message');
  console.log(msg);
}

Problem
After the bot enters this conversation, it successfully sends a prompt message to the user with ctx.reply('Please enter the address'). However, as soon as the user replies, the bot freezes completely:
The console.log(msg) line is never executed, so the message is never logged.
The entire bot becomes unresponsive to any further inputs or commands.
The only way to restore functionality is by deleting the cached session and restarting the project. Simply restarting without clearing the session doesn’t solve the problem, as it replays the last session and continues to freeze.
This might be my own issue, but I've spent considerable time trying to debug without success. Any insights or suggestions on what might be causing this issue would be greatly appreciated.
Version Information

grammy: ^1.31.0
@grammyjs/conversations: ^1.2.0
bun: ^1.1.34

Which version of the plugin do you use?
This typically only happens on v1.x when you call conversation.external and pass a callback that again performs API calls. I cannot reproduce this with the above code alone.

Thank you for looking into this issue! Here’s the version information for the plugins I’m currently using:

grammy: ^1.31.0
@grammyjs/conversations: ^1.2.0

Can you reproduce the same issue with Node.js?

Yes, I encountered the same issue with Node.js as well.

The above code works flawlessly on my machine.

Here is the full code I used, just to make sure you did not introduce a bug somewhere in the code you did not share (such as in initialSession. Copy the following code, paste your token, and run it:
import { Bot, session } from "grammy";
import { conversations, createConversation } from "@grammyjs/conversations";
import { FileAdapter } from "@grammyjs/storage-file";

const bot = new Bot("");

function initialSession() {
  return {};
}

bot.use(session({
  initial: initialSession,
  storage: new FileAdapter({
    dirName: "sessions",
  }),
}));

bot.use(conversations());

const myConversation = async (conversation, ctx) => {
  const { message_id } = await ctx.reply("Please enter the address");
  const { msg } = await conversation.waitFor("message");
  console.log(msg);
};
bot.use(createConversation(myConversation));

bot.command("start", (ctx) => ctx.conversation.enter("myConversation"));

bot.start();

Thank you for your guidance. I was able to reproduce the issue using the following code:
import { Bot, InlineKeyboard, session } from "grammy";
import { conversations, createConversation } from "@grammyjs/conversations";
import { FileAdapter } from "@grammyjs/storage-file";

const bot = new Bot("");
export const extendContext = async (ctx, next) => {
  ctx.safeAnswer = async (...args) => {
    if (!ctx.callbackQuery) {
      return false;
    }
    return await ctx.answerCallbackQuery(...args).catch(() => false);
  };

  ctx.replyAndSafeAnswer = async (...args) => {
    await ctx.safeAnswer();
    return await ctx.reply(...args);
  };
  await next();
};

const initialSession = () => {
  return {};
};

bot.use(
  session({
    initial: initialSession,
    storage: new FileAdapter({
      dirName: "sessions",
    }),
  })
);
bot.use(conversations());
bot.use(extendContext);

const myConversation = async (conversation, ctx) => {
  const { message_id } = await ctx.replyAndSafeAnswer("Please enter the address");
  const { msg } = await conversation.waitFor("message");
  console.log(msg);
  await ctx.replyAndSafeAnswer("Done");
};
bot.use(createConversation(myConversation));

bot.command("start", async (ctx) => {
  ctx.reply("hi", {
    reply_markup: new InlineKeyboard().text("setAddress", "setAddress"),
  });
});
bot.callbackQuery("setAddress", async (ctx) => {
  await ctx.conversation.enter("myConversation");
});
bot.start();

I noticed that the issue only occurs when entering the conversation via the inline keyboard button (setAddress). If I trigger the conversation directly through a command, it works as expected. Let me know if you need any further details. Thank you for your assistance!

Ah well. That explains a lot.
There is a short and a long explanation as to why this happens. There is a hacky fix for v1 and a clean way to fix this for the soon-to-be-released v2.
Let's start with the short explanation and the clean solution, and then I'll give you the long explanation and the hacky fix.
The Short Explanation
The custom function that you install on the context object is not supported by conversations v1. It can detect that there is a function, but it does not know how to handle it. The plugin then tries to come up with a fix, but ends up calling the wrong function internally. This confuses the plugin so much that it basically waits forever.
The runtime sees that no code is running anymore and garbage-collects the entire process. This is what you observe as a terminating process.
 I rewrote the entire plugin from scratch and these things are all fixed now. It will be released in the coming time after the documentation is completed and has been translated to all 5 other languages.
The proper fix is to upgrade to v2. You can install it from GitHub.
npm install github:grammyjs/conversations

A number of things have changed, and the migration guide is not done yet, but the README as well as https://github.com/grammyjs/website/pull/1147#issuecomment-2466258345 with its deployment preview should give you some insight.
Basically, the only thing you have to change is to use
bot.use(createConversation(myConversation, {
  plugins: [extendContext]
}));

I have tried your code and this fixes the issue.
Check out https://t.me/grammyjs/277882, too.
The Long Explanation
As you can read in the v2 docs, conversations are replay engines. This is true for both versions of the plugin. However, v1 tried to directly persist the context objects themselves in a file. You installed a function. Functions cannot be persisted. In an attempt to fix this, the v1 plugin delegates the function call to the new incoming context object. However, this causes the replay logs to deviate from the the actual behaviour of the conversation builder function. There is a weird bug in the v1 plugin that it cannot detect this, so it tries to wait for the result of an API call that has never been recorded. As a result, it waits for data that does not exist, which is like doing await new Promise(() => {})—it garbage-collects the current stack and hence terminates the runtime. This is very hard to fix with the implementation of v1. The underlying flaw is that context objects are reused across replays, and that should just not be done. (That's why v2 does not do it.)
The Hacky Fix With v1
I do not feel like expanding on why this works, but it does. (I tested it, too.)
// Replace
const { message_id } = await ctx.replyAndSafeAnswer("Please enter the address");
// with
if (!conversation._isReplaying) {
  const { message_id } = await ctx.replyAndSafeAnswer("Please enter the address");
}


how to get url parameters?? like if a url is <IP_ADDRESS>:8000/api/books?query=jason, how to get the query param in japronto?

GitHub issues aren't for you to ask general "how to" questions. They're for actual issues with the product.
With that being said, take a look at the examples and do some research on your own. I'll help you get started. Take a look at this line in particular.
If you have any further questions, you should consider posting the question in a more suitable place like Stack Overflow.

def getQueryString(request):
url_params = request.query_string

@ATLUSio There is currently no stack overflow question and answers about this library. There doesn't seem to be a critical mass of users there.
See search link here: https://stackoverflow.com/search?q=japronto


Make taxes read-only when connected to accounting integration

Details

When connected to accounting integration(QBO or Xero) taxes should be read-only(user cannot delete/add them). This PR covers that.
Fixed Issues

$ https://github.com/Expensify/App/issues/39911
PROPOSAL: N/A
Tests

[ ] Verify that no errors appear in the JS console

Offline tests

QA Steps

[ ] Verify that no errors appear in the JS console

PR Author Checklist

[ ] I linked the correct issue in the ### Fixed Issues section above
[ ] I wrote clear testing steps that cover the changes made in this PR

[ ] I added steps for local testing in the Tests section
[ ] I added steps for the expected offline behavior in the Offline steps section
[ ] I added steps for Staging and/or Production testing in the QA steps section
[ ] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[ ] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[ ] I included screenshots or videos for tests on all platforms
[ ] I ran the tests on all platforms & verified they passed on:

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

[ ] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[ ] I followed proper code patterns (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[ ] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[ ] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I followed the guidelines as stated in the Review Guidelines
[ ] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] I verified that if a function's arguments changed that all usages have also been updated correctly
[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[ ] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[ ] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[ ] I verified that all the inputs inside a form are aligned with each other.
[ ] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

@hayata-suenaga @lakchote @trjExpensify reopening PR for https://github.com/Expensify/App/issues/39911

Right now tags and categories are already read-only, because they were fixed in other PRs. Only one thing to fix is to make taxes read-only.

Are you sure preventing both Add and Delete actions are implemented for tags

Correct! We're also waiting for https://github.com/Expensify/App/pull/41562 to be merged

When we're ready, @hungvu193 will take the C+ review of this.

@SzymczakJ Looks like linked PR was merged. I think this should be ready for review.

@hungvu193 this is ready for review 😄

@SzymczakJ  Awesome, can you please complete your author checklist? Ty 😄

@hungvu193 done!

Reviewer Checklist

[x] I have verified the author checklist is complete (all boxes are checked off).
[x] I verified the correct issue is linked in the ### Fixed Issues section above
[x] I verified testing steps are clear and they cover the changes made in this PR

[x] I verified the steps for local testing are in the Tests section
[x] I verified the steps for Staging and/or Production testing are in the QA steps section
[x] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I checked that screenshots or videos are included for tests on all platforms
[x] I included screenshots or videos for tests on all platforms
[x] I verified tests pass on all platforms & I tested again on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[x] I verified proper code patterns were followed (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I verified that this PR follows the guidelines as stated in the Review Guidelines
[x] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

https://github.com/Expensify/App/assets/16502320/1ca603b5-e157-4689-a1b0-e983a545a2f2

Android: mWeb Chrome

iOS: Native

https://github.com/Expensify/App/assets/16502320/d242832b-4596-49a4-b1af-5eaac987d438

iOS: mWeb Safari

https://github.com/Expensify/App/assets/16502320/58121de3-91dc-4c05-b776-04c28f152694

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/16502320/043daa20-685d-4e88-912c-4cc51f76621a

MacOS: Desktop

https://github.com/Expensify/App/assets/16502320/57b6f36e-e0df-490f-af9f-7dfdb954bbb2

🚀 Deployed to staging by https://github.com/lakchote in version: 1.4.77-11 🚀

platform
result

🤖 android 🤖
failure ❌

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to staging by https://github.com/lakchote in version: 1.4.77-11 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/mountiny in version: 1.4.78-5 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Improve martialling of compiler information

The way compiler properties are managed is rather hap-hazard at the moment. For each compiler we need to know things such as the argument used to specify where module files are created. Plus any special arguments needed for that compiler.
At the moment I can't find a coherent and extensible way to manage these which makes them easy to obtain and also easy to extend with new compilers.

I immediately imagine two potential solutions.
An abstract interface which specifies a compiler with concrete implementations for each supported compiler.
A generic class representing a compiler which is multiply instantiated from a configuration file.


Error while processing some SVGs with version 2.1

Describe the bug
With the latest release (2.1) some SVGs are no longer being processed. They fail with the error:
Error: Error in parsing SVG: Non-whitespace before first tag.
Line: 0
Column: 1
Char: �
File: /tmp/test.svg
    at processSVGData (/node_modules/svgo/lib/svgo/coa.js:290:13)
    at /node_modules/svgo/lib/svgo/coa.js:267:17
    at async Promise.all (index 0)
    at async Command.action (/node_modules/svgo/lib/svgo/coa.js:182:13)
    at async Promise.all (index 0)

Note that the file is rendered correctly on Chrome.
To Reproduce
Run svgo on the attached file.
Expected behavior
The file should be optimized correctly.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

SVGO Version: 2.1.0
NodeJs Version: 14.4.0
OS: MacOS
issue.svg.txt

I see. It's utf-16. Well, v1.3.2 failed in the same way. v2.0.0 didn't failed but produced empty svg.
So it was never actually supported. Is it possible for you to get utf-8?


Tags are not being propagated to cloudformation resources

Reporting Bugs
Per boto3 documentation if Tags are passed when create_stack is called, those tags must also be propagated to the resources created in that stack.
Tags (list) -- Key-value pairs to associate with this stack. AWS CloudFormation also propagates these tags to the resources created in the stack. A maximum number of 50 tags can be specified.
I do not think moto is propagating these tags to the resources created when creating the stack.

@bblommers Is that correct? Also, if so, is there a way to get passed it?

Hi @sahilshah6196, that's partially correct. Tags are propagated to EC2 resources, but not to any other services.
See this section of the CF logic: https://github.com/spulec/moto/blob/master/moto/cloudformation/parsing.py#L603
There's no way around that currently, afaik. Will mark it as an enhancement to have a more centralized approach to storing tags.

Hi @sahilshah6196, that's partially correct. Tags are propagated to EC2 resources, but not to any other services.
See this section of the CF logic: https://github.com/spulec/moto/blob/master/moto/cloudformation/parsing.py#L603
There's no way around that currently, afaik. Will mark it as an enhancement to have a more centralized approach to storing tags.

Ah! I see. Thanks for the info

For what it is worth, in my experience CloudFormation itself does not always propagate tags to resources, and is especially poor at updating tags.

@sahilshah6196 Was there a specific service/resource that you wanted fixed?
Given the fact that AWS is not consistent in this approach either, and the work involved to completely copy that behaviour, we might be better off reducing the scope to only what is needed now.


rustsat-cadical compilation fails on macOS + workaround

FYI, I noticed that rustsat-cadical fails to compile on macOS.
It turned out to be caused by this issue in cc:

https://github.com/rust-lang/cc-rs/issues/963
https://github.com/rust-lang/cc-rs/issues/1001

It works fine after doing cargo add<EMAIL_ADDRESS>So most likely rustsat-cadical will work fine once the bug in cc gets fixed, but I'm posting this here so that others who are struggling to get rustsat-cadical work will find a workaround somewhere…

Thanks for pointing this out and providing a workaround :+1:
Interesting that the macos github CI seems to not have problems. I guess this is exclusive to macos 14.

It fails also on my Intel iMac with macOS 12.7.4, so it doesn't seem to be exclusive to macOS 14.

(But my guess is that GitHub CI has got also Homebrew-Clang, while I've got just Apple-Clang and Homebrew-GCC, and this is enough to make a difference.)

Seems to work now, thanks to this: https://github.com/rust-lang/cc-rs/pull/1007


Processing HCP-like dataset

Hii all,
I'm trying to process an HCP-like dataset and would appreciate any tips regarding any specific flags that may be appropriate.
My DWI acquisition consists of 2 PE images that are not similar (and therefore not exactly the same as HCP's):

One DWI series (dir-FWD) is the "main" one, and consists of 186 volumes (4 shells).
Another DWI series (dir-REV) consists of 7 volumes (2 shells).

I'm having some trouble understanding which flags are appropriate in this case, and specifically wondering about these:

--denoise-after-combining
--distortion-group-merge

Would love any thoughts about this.
Thanks! (:

I've not done anything but the default for denoising. For --distortion-group-merge, this must be "concat" unless you have the exact same sequence played out with opposite gradients. There must be a 1:1 match between every measurement dir-FWD and the same measurement dir-REV.

Thanks @cookpa, I'll give it a try.

It seems that the "DWI series" refers to the "dir-REV" acquisition, although, as described earlier, the main DWI image is the "dir-FWD".
Is it even important for the pipeline if --distortion-group-merge concat is passed?
And if it is - is there anyway to let the query know that the "dir-FWD" is the main DWI series?

I've tried following your suggestions and only passed --distortion-group-merge concat.
It encounters the following error (the full log is attached below):
indexed_gzip.indexed_gzip.IndexedGzipFile.__reduce__
  File "indexed_gzip/indexed_gzip.pyx", line 737, in indexed_gzip.indexed_gzip._IndexedGzipFile.export_index
AttributeError: '_io.BytesIO' object has no attribute 'mode'

@cookpa, Any thoughts?
Node: qsiprep_wf.single_subject_14_wf.anat_preproc_wf.t1_2_mni
Working directory: /work/qsiprep_wf/single_subject_14_wf/anat_preproc_wf/t1_2_mni

Node inputs:

compress_report = auto
explicit_masking = True
flavor = precise
float = True
initial_moving_transform = <undefined>
lesion_mask = <undefined>
moving = T1w
moving_image = /work/qsiprep_wf/single_subject_14_wf/anat_preproc_wf/skullstrip_wf/rigid_acpc_resample_head/highres001_N4Corrected0_trans.nii.gz
moving_mask = /work/qsiprep_wf/single_subject_14_wf/anat_preproc_wf/skullstrip_wf/rigid_acpc_resample_mask/highres001_BrainExtractionMask_trans.nii.gz
num_threads = 8
orientation = LPS
out_report = report.svg
reference = T1w
reference_image = /usr/local/miniconda/lib/python3.8/site-packages/qsiprep/data/mni_1mm_t1w_lps_brain.nii.gz
reference_mask = <undefined>
settings = <undefined>
template = MNI152NLin2009cAsym
template_resolution = 1

Traceback (most recent call last):
  File "/home/groot/.local/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/home/groot/.local/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run
    result = self._run_interface(execute=True)
  File "/home/groot/.local/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface
    return self._run_command(execute)
  File "/home/groot/.local/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/home/groot/.local/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 429, in run
    runtime = self._post_run_hook(runtime)
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/niworkflows/interfaces/registration.py", line 60, in _post_run_hook
    return super(RobustMNINormalizationRPT, self)._post_run_hook(runtime)
  File "/home/groot/.local/lib/python3.8/site-packages/nipype/interfaces/mixins/reporting.py", line 50, in _post_run_hook
    self._generate_report()
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/niworkflows/interfaces/report_base.py", line 55, in _generate_report
    mask_nii = threshold_img(fixed_image_nii, 1e-3)
  File "/home/groot/.local/lib/python3.8/site-packages/nilearn/image/image.py", line 824, in threshold_img
    img_data = _safe_get_data(img, ensure_finite=True, copy_data=copy)
  File "/home/groot/.local/lib/python3.8/site-packages/nilearn/_utils/niimg.py", line 51, in _safe_get_data
    img = copy.deepcopy(img)
  File "/usr/local/miniconda/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/miniconda/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/local/miniconda/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/miniconda/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/miniconda/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/miniconda/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/local/miniconda/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/miniconda/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/miniconda/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/miniconda/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/local/miniconda/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/miniconda/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/miniconda/lib/python3.8/copy.py", line 161, in deepcopy
    rv = reductor(4)
  File "indexed_gzip/indexed_gzip.pyx", line 956, in indexed_gzip.indexed_gzip.IndexedGzipFile.__reduce__
  File "indexed_gzip/indexed_gzip.pyx", line 737, in indexed_gzip.indexed_gzip._IndexedGzipFile.export_index
AttributeError: '_io.BytesIO' object has no attribute 'mode'

I'm not sure what's happening here exactly, but it may help to use a new working directory with -w just in case there are some stale or missing files from your previous run


Refactor robustness methods

Pull Request Checklist:

[ ] This PR addresses an already opened issue (for bug fixes / features)

This PR fixes #xyz

[ ] Tests for the changes have been added (for bug fixes / features)

[x] (If applicable) Documentation has been added / updated (for bug fixes / features)

[x] CHANGES.rst has been updated (with summary of main changes)

[ ] Link to issue (:issue:number) and pull request (:pull:number) has been added

What kind of change does this PR introduce?

Add ipcc-advanced as a test to ensembles.change_significance. This corresponds to the simpler alternative of approach C as described in the Cross-Chapter Box 1 of the IPCC Atlas (AR6, WG1).
Add two examples in the "Ensembles" notebook to show how xclim can be used to get the IPCC-recommended hatching masks.
Add a very simple detrend function.

Does this PR introduce a breaking change?
No, but it might, see below.
Other information:
As you can see in the examples, the IPCC decouples the "change significance" from the "sign agreement", while xclim does not. This make the second example more complex than expected.
I thought of :

Changing the output of change_significance so that pos_frac is independent of change significance
Adding a third output so we have: significant_change_frac, significant_positive_change_frac, positive_change_frac.
Adding a keyword argument to switch between the two modes.
First two are breaking changes while the thirds feels overly heavy for something this simple...

@huard After our discussions and some thinking, I rewrote the function completely.
I was not satisfied with the output of change_significance so I rewrote it as robustness_fractions. The former still works but it will warn the user that it is deprecated. Also added robustness_categories to help with making the actual map.
Finally, I simplified the notebook a lot. May be too much...
I'll add tests tomorrow.

@RondeauG , would you have time to look at the new functions ? And may be @mccrayc, if you have time.
I'm simply looking for a review of the general idea and usage, as shown in the example : https://xclim--1514.org.readthedocs.build/en/1514/notebooks/ensembles.html#Change-significance-and-model-agreement

coverage: 89.812%. first build
when pulling bc3eef56e72fff70d8cf7d3bc986585e16da47cb on ipcc-robustness
into ca3caff8168bf148dc6fe0a4b5fcce9e53a09011 on master.

This looks pretty good! robustness_categories will be very useful.


Create plugin that checks code coverage before publishing

Would be good to fail before version/publish if coverage falls below a certain percentage (default to 80?)

Created PoC of this plugin in the coverage branch. Not sure if it belongs in this monorepo long-term, or should be published independently.


Fixing cli.bats tests

I broke the integration tests in #1065 :disappointed:
This PR fixes them :smile:
Before:
$ bats test/integration/cli.bats
 ✓ cli: show info
 ✗ cli: show active help
   (in test file test/integration/cli.bats, line 14)
     `[ "$status" -eq 0  ]' failed
 ✗ cli: show config help
   (in test file test/integration/cli.bats, line 20)
     `[ "$status" -eq 0  ]' failed
 ✗ cli: show inspect help
   (in test file test/integration/cli.bats, line 26)
     `[ "$status" -eq 0  ]' failed
 ✗ cli: show ip help
   (in test file test/integration/cli.bats, line 32)
     `[ "$status" -eq 0  ]' failed
 ✗ cli: show kill help
   (in test file test/integration/cli.bats, line 38)
     `[ "$status" -eq 0  ]' failed
 ✗ cli: show ls help
   (in test file test/integration/cli.bats, line 44)
     `[ "$status" -eq 0  ]' failed
 ✗ cli: show restart help
   (in test file test/integration/cli.bats, line 50)
     `[ "$status" -eq 0  ]' failed
 ✗ cli: show rm help
   (in test file test/integration/cli.bats, line 56)
     `[ "$status" -eq 0  ]' failed
 ✗ cli: show env help
   (in test file test/integration/cli.bats, line 62)
     `[ "$status" -eq 0  ]' failed
 ✗ cli: show ssh help
   (in test file test/integration/cli.bats, line 68)
     `[ "$status" -eq 0  ]' failed
 ✗ cli: show start help
   (in test file test/integration/cli.bats, line 74)
     `[ "$status" -eq 0  ]' failed
 ✗ cli: show stop help
   (in test file test/integration/cli.bats, line 80)
     `[ "$status" -eq 0  ]' failed
 ✗ cli: show upgrade help
   (in test file test/integration/cli.bats, line 86)
     `[ "$status" -eq 0  ]' failed
 ✗ cli: show url help
   (in test file test/integration/cli.bats, line 92)
     `[ "$status" -eq 0  ]' failed
 ✓ flag: show version
 ✗ flag: show help
   (in test file test/integration/cli.bats, line 104)
     `[ "$status" -eq 0  ]' failed

17 tests, 15 failures

And with this PR:
$ bats test/integration/cli.bats
 ✓ cli: show info
 ✓ cli: show active help
 ✓ cli: show config help
 ✓ cli: show inspect help
 ✓ cli: show ip help
 ✓ cli: show kill help
 ✓ cli: show ls help
 ✓ cli: show restart help
 ✓ cli: show rm help
 ✓ cli: show env help
 ✓ cli: show ssh help
 ✓ cli: show start help
 ✓ cli: show stop help
 ✓ cli: show upgrade help
 ✓ cli: show url help
 ✓ flag: show version
 ✓ flag: show help

17 tests, 0 failures

Signed-off-by: Dave Henderson<EMAIL_ADDRESS>

/cc @ehazlett @nathanleclaire

I still get a fail on the first test.
 ✗ cli: show info
   (in test file test/integration/cli.bats, line 8)
     `[[ ${lines[0]} =~ "NAME:"  ]]' failed

@ehazlett - weird... I don't know why it was passing for me :confused:. Will update soon.

@ehazlett - fixed. PTAL

LGTM


Feature request: deleteRemote option

When using watch with a task runner like gulp or grunt I only wan't to to sync/upload files when the process has finished compiling the build files. Right now when the task runner has run the files in the build folder are instantly deleted causing remote-sync to also delete the remote files. Would it be possible to have a deleteRemote option, which I can set to false, and only sync/upload when a new file is ready after the build process has completed.

Absolutely needed working with a Task Runner!


sunshine: Missing amd and intel gpu encoding deps

Describe the bug
sunshine is built with an override for cuda but not for amd or intel
Steps To Reproduce
Steps to reproduce the behavior:

Try to encode on amd, fails

Expected behavior
For at least rocm/amd support to be added
Additional context
arch example of needed deps
Notify maintainers
@devusb

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
output here

Add a :+1: reaction to issues you find important.

Sunshine seems to use CUDA exclusively for NVFBC capture. For regular GPU accelerated video encoding, the standard VAAPI driver should be sufficient for both AMD and Intel which are the drivers that the Archlinux docs mention  (not ROCM).
For the VAAPI drivers, Nix uses the module system to enable these. See this page on enabling the necessary drivers.

The AMD VAAPI driver is just part of Mesa so should be present if you have working graphics already.
I think we just need to always enable the autoAddDriverRunpath hook since we need to use the system OpenGL drivers regardless if we are using CUDA or not.
If you use the CUDA version of the package, then does the video encoding (using AMD) work? Set:
services.sunshine.package = pkgs.sunshine.overrideAttrs( p: {
    nativeBuildInputs = p.nativeBuildInputs ++ [ pkgs.autoAddDriverRunpath ];
  }
);

@Redhawk18 what issues are you running into? I exclusively use Sunshine with AMD + VAAPI -- works out of the box with the module. These are the log lines I see at startup.
[2024:09:20:07:16:19]: Info:
[2024:09:20:07:16:19]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024:09:20:07:16:19]: Info:
[2024:09:20:07:16:19]: Info: Found H.264 encoder: h264_vaapi [vaapi]
[2024:09:20:07:16:19]: Info: Found HEVC encoder: hevc_vaapi [vaapi]
[2024:09:20:07:16:19]: Info: Found AV1 encoder: av1_vaapi [vaapi]
[2024:09:20:07:16:19]: Info: Configuration UI available at [https://localhost:47990]
[2024:09:20:07:16:19]: Info: Adding avahi service Sunshine
[2024:09:20:07:16:20]: Info: Avahi service Sunshine successfully established.

For what it's worth, the module should not be required -- if I remember correctly CAP_SYS_ADMIN may be required for Wayland, but I think it worked using VAAPI on xorg without it.
Certainly possible that there could be some idiosyncrasy of my config that makes this work, but would be surprised -- running Plasma 6 on Wayland, nothing too weird.

This is my log, the sunshine devs replied saying its because we use stock ffmpeg so take that as you will.
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Screencasting with KMS
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: /dev/dri/card1 -> amdgpu
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Found monitor for DRM screencasting
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Found connector ID [107]
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Found cursor plane [82]
Sep 20 11:23:00 Mythra sunshine[2717]: libva info: VA-API version 1.21.0
Sep 20 11:23:00 Mythra sunshine[2717]: libva info: Trying to open /run/opengl-driver/lib/dri/radeonsi_drv_video.so
Sep 20 11:23:00 Mythra sunshine[2717]: libva info: Found init function __vaDriverInit_1_21
Sep 20 11:23:00 Mythra sunshine[2717]: libva info: va_openDriver() returns 0
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: SDR color coding [Rec. 709]
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Color depth: 10-bit
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Color range: [JPEG]
Sep 20 11:23:00 Mythra sunshine[2717]: libva info: VA-API version 1.21.0
Sep 20 11:23:00 Mythra sunshine[2717]: libva info: Trying to open /run/opengl-driver/lib/dri/radeonsi_drv_video.so
Sep 20 11:23:00 Mythra sunshine[2717]: libva info: Found init function __vaDriverInit_1_21
Sep 20 11:23:00 Mythra sunshine[2717]: libva info: va_openDriver() returns 0
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Error: [hevc_vaapi @ 0x406cb480] No usable encoding entrypoint found for profile VAProfileHEVCMain10 (18).
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Retrying with fallback configuration options for [hevc_vaapi] after error: Function not implemented
Sep 20 11:23:00 Mythra sunshine[2717]: libva info: VA-API version 1.21.0
Sep 20 11:23:00 Mythra sunshine[2717]: libva info: Trying to open /run/opengl-driver/lib/dri/radeonsi_drv_video.so
Sep 20 11:23:00 Mythra sunshine[2717]: libva info: Found init function __vaDriverInit_1_21
Sep 20 11:23:00 Mythra sunshine[2717]: libva info: va_openDriver() returns 0
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Warning: [hevc_vaapi @ 0x4070d540] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info:
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: // Ignore any errors mentioned above, they are not relevant. //
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info:
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Found H.264 encoder: h264_vaapi [vaapi]
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Found HEVC encoder: hevc_vaapi [vaapi]
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Adding avahi service Sunshine
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Configuration UI available at [https://localhost:47990]
Sep 20 11:23:01 Mythra sunshine[2717]: [2024:09:20:11:23:01]: Info: Avahi service Sunshine successfully established.

This might work, but I don't know this program well enough to really understand.

The AMD VAAPI driver is just part of Mesa so should be present if you have working graphics already.
I think we just need to always enable the autoAddDriverRunpath hook since we need to use the system OpenGL drivers regardless if we are using CUDA or not.
If you use the CUDA version of the package, then does the video encoding (using AMD) work? Set:
services.sunshine.package = pkgs.sunshine.overrideAttrs(p: {
    nativeBuildInputs = p.nativeBuildInputs ++ [ pkgs.autoAddDriverRunpath ];
  }
);

I'll have to try this again with my friend, since together our connection is slow enough to require encoding.

@Redhawk18 your log looks like vaapi was found, so hardware encoding should be working? the bits below
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: // Ignore any errors mentioned above, they are not relevant. //
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info:
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Found H.264 encoder: h264_vaapi [vaapi]
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Found HEVC encoder: hevc_vaapi [vaapi]

also FWIW we actually don't use stock ffmpeg anymore -- we use sunshine's vendored binaries and patch them with this bit.

Hmm, how can I test this with that commit? I don't believe rebuilding sunshine with a local version of nixpkgs will update the service and I don't want to rebuild everything to unstable.
log from my phone
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Sunshine version: 0.23.1
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Found display [wayland-0]
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Found interface: zxdg_output_manager_v1(31) version 3
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Found interface: wl_output(61) version 4
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Found interface: wl_output(62) version 4
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Warning: Missing Wayland wire for wlr-export-dmabuf
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: /dev/dri/card1 -> amdgpu
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: /dev/dri/card0 -> amdgpu
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Found display [wayland-0]
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Found display [wayland-0]
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Found interface: zxdg_output_manager_v1(31) version 3
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Found interface: wl_output(61) version 4
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Found interface: wl_output(62) version 4
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Resolution: 1920x1080
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Resolution: 1920x1080
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Offset: 0x0
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Logical size: 1920x1080
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Name: DP-1
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Found monitor: AUS VG278/16843009
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Offset: 1920x0
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Logical size: 1920x1080
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Name: HDMI-A-1
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Found monitor: ACR GN246HL/1668346451
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: -------- Start of KMS monitor list --------
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Monitor 1 is DP-1: AUS VG278/16843009
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Monitor 0 is HDMI-A-1: ACR GN246HL/1668346451
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: --------- End of KMS monitor list ---------
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Error: Couldn't find any working encoder matching [amdvce]
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Trying encoder [nvenc]
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Screencasting with KMS
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: /dev/dri/card1 -> amdgpu
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: /dev/dri/card0 -> amdgpu
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Error: Couldn't find monitor [1]
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: /dev/dri/card1 -> amdgpu
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: /dev/dri/card0 -> amdgpu
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Error: Couldn't find monitor [1]
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: System tray created
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: Screencasting with KMS
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: /dev/dri/card1 -> amdgpu
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: /dev/dri/card0 -> amdgpu
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Error: Couldn't find monitor [1]
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: /dev/dri/card1 -> amdgpu
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Info: /dev/dri/card0 -> amdgpu
Sep 20 11:22:59 Mythra sunshine[2717]: [2024:09:20:11:22:59]: Error: Couldn't find monitor [1]
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Encoder [nvenc] failed
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Trying encoder [vaapi]
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: Screencasting with KMS
Sep 20 11:23:00 Mythra sunshine[2717]: [2024:09:20:11:23:00]: Info: /dev/dri/card1 -> amdgpu

sunshine from 24.05 should also use vendored ffmpeg, but again from the logs you sent it looks like yours is using vaapi correctly.

Then do you have any other optimization tips? I applied the basics from the sunshine docs I saw.

Dealing with this issue on unstable as well. Has a fix been pushed yet?

Dealing with this issue on unstable as well. Has a fix been pushed yet?

If you get a fix from an update post it here, because I would like to request a backport to.

@Redhawk18 it's not completely clear to me what issue you are experiencing -- from the logs you sent, VAAPI is being detected and used, which is using for hardware accelerated encoding on AMD and Intel -- essentially, the logs indicate that Sunshine is working as expected. are you having another issue outside of that?

@Redhawk18 it's not completely clear to me what issue you are experiencing -- from the logs you sent, VAAPI is being detected and used, which is what is used for hardware accelerated encoding on AMD and Intel -- essentially, the logs indicate that Sunshine is working as expected. are you having another issue outside of that?

Just a slow connection, I tried everything I could to aggressively encode everything but it still not be a fast enough connection.

Sounds like it might be more of a connectivity issue -- assuming you've tried wired connection on same subnet to see if as direct as possible works?

Sounds like it might be more of a connectivity issue -- assuming you've tried wired connection on same subnet to see if as direct as possible works?

Not possible, different Lan's but I'll close for now


[Feature Request] Endless scrolling

Is your feature request related to a problem? Please describe.
It’s annoying to click the next entry every time.
Describe the solution you'd like
The first thought is to add a shortcut key, but why not just cancel the next entry button, let the scrolling be infinite, such as loading next chapter on the last page.
Describe a small use-case for this feature request
save some time, keep your hands on the keyboard, focus on reading.
Additional context
n/a

Good idea, but I am worried that if the user keeps scrolling, we will have a huge DOM, and we might have some memory issues on low-end devices. Ideally, we should remove the pages that are not in view from the DOM, but that would be quite complicated as most pages are server-rendered, and we are not using a JS framework like Vue/React. Let me think about it and see what we can do!

Good idea, but I am worried that if the user keeps scrolling, we will have a huge DOM, and we might have some memory issues on low-end devices. Ideally, we should remove the pages that are not in view from the DOM, but that would be quite complicated as most pages are server-rendered, and we are not using a JS framework like Vue/React. Let me think about it and see what we can do!


UK collection

                     To update or to get new version

@kiranchhetri1 thank you for your contribution, please sign CLA.

What is this supposed to solve?

@kiranchhetri1 Can you please check @mihaimaruseac's comments and keep us posted. Thanks!

It has been 14 days with no activity and the awaiting response label was assigned. Is this PR still valid? Assigning the stalled label. Please comment to reassure me that this is still being worked on.


Added object destructure snippet for javascript

This time I tried to make the snippet more understandable by the average reader.

@Chalarangelo is there something wrong with the styling used in this snippet?
I think it follows the contribution guidelines, is there something I'm missing?
Thanks


Allow components to either use context provider or use props

Quite a few components rely on a context provider and associated hook, but in many cases it may also make sense to allow devs to pass in a prop instead and not rely on setting up a context provider.
For example, <Metafield/> uses useShop(), but only need the locale from it. We could also allow a locale prop and not rely on the useShop() hook in that case.

Hm, while it makes sense for easy ones like locale to be either a prop or from context, I'm less sure on complex ones, like those that come from the CartProvider.

prop should override context
if no prop value is provided - we default to context

We should also update CartProvider to optionally pull from ShopifyConfig the contryCode.

Decision: leave the useCart() things alone for now, as there is more complexity there, but allow places that use useShop() to optionally take in a prop.


WA DAMN EFFICIENT. CANNOT TAKE IT

HAHAHA

Eh... cant seem to fix this issue. Hahaha!

hahaha. Anw if you need help let me know? Just open issue then assign to me. but will only do after Monday. got orbital submission hahaha


MVP for Create a Jira issue button

This PR adds simple Jira button to each failed test case.
Description
This is a bit of MVP since the button doesn't use any Jira authentication. It redirects the user to Cognifide's Jira. In the future button could open a modal dynamically populated by Jira fields.
Motivation and Context
When an AET test case fails that usually means new Jira bug is raised. The button makes this a bit easier.
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

I hereby agree to the terms of the AET Contributor License Agreement.

Hi @zofoklecja do you plan on finishing this feature in the near future?

Closing due to lack of activity.


Have optoins for finetuning rbenv init

A rbenv init does for 4 different things. Can it be possible to do all 4 seperately. For example some people like me want set up path only once, so .profile should be used. But rbenv shell functions have to be in .*rc file.

You don't have to use rbenv init at all; it's entirely optional. You only need to manually prepend ~/.rbenv/shims to the PATH if you're going to skip the rbenv init step.


[Bug]: Duplicate waystones that can't be forgotten

Is there an existing issue for this?

[X] I have searched the existing issues

Description
When anyone opens a waystone, some names appear twice. When trying to forget either of the two duplicated names, they don't get removed, the list just appears to move up by one. Both names can be used to teleport to the waystone. You can still forget other waystones that are not duplicated. This happened after updating a few mods from the server to newer versions, but the Waystones mod was not changed.
To Reproduce
The list of mods:
modlistforserver.txt
(Waystones version 2.5.5)
Unsure of how to reproduce in singleplayer and don't have another server to reproduce on.
Expected behavior
Should be able to forget all waystones, and not have duplicated waystones appearing in the list.
Mod Version
v2.5.5
Minecraft Version
1.18.2
Java Version
Java 17
Fabric Version

Fabric Loader: 0.14.8-1.18.2
Fabric API: 0.58.0+1.18.2

Anything else?
Here's an image of some of the duplicates. I can attach a video example if needed.

Which mods were updated?

here's a ss of the mods that were updated to new versions

same here, happened after moving the files to another server or editing the config file?
searched for a file to delete the duplicates myself, but i can't find it? is there a "txt/json"? file where it gets saved to?

@bloodyeyesblackblood you can check the wiki page for manual Waystone editing:
https://github.com/LordDeatHunter/FabricWaystones/wiki/Manual_Editing
As for the issue, I am aware of it, but haven't had any luck in reproducing it so far.

@bloodyeyesblackblood you can check the wiki page for manual Waystone editing. As for the issue, I am aware of it, but haven't had any luck in reproducing it so far.

thanks, i'll try to reproduce the things i did.
the nbt editor is a bit tricky, also the duplicates have a different hash.
5
[color: 13743503](https://irath96.github.io/webNBT/#)
[dimension: "minecraft:overworld"](https://irath96.github.io/webNBT/#)
[hash: "[17, 105, 99, -73, 112, -114, 26, -30, 96, 60, -73, -54, -59, -51, -102, 107, 61, 42, -78, -29, -44, -126, 62, -39, -121, 121, 114, 76, -76, -72, -110, -124]"](https://irath96.github.io/webNBT/#)
[name: "BD -3480 ~ 86"](https://irath96.github.io/webNBT/#)
[position 3 ints](https://irath96.github.io/webNBT/#)

6
[color: 13743503](https://irath96.github.io/webNBT/#)
[dimension: "minecraft:overworld"](https://irath96.github.io/webNBT/#)
[hash: "[-83, 98, -23, 22, 99, -93, -120, -27, -36, 94, 15, 120, -124, 116, -106, 111, 98, 15, -109, -8, -68, 108, 91, 122, -43, -96, 49, 112, -126, -109, -22, 32]"](https://irath96.github.io/webNBT/#)
[name: "BD -3480 ~ 86"](https://irath96.github.io/webNBT/#)
[position 3 ints](https://irath96.github.io/webNBT/#)

idk if it helps :)

Well, kinda 🤔 The hashes aren't reversible, so I can't really check what's going on for sure. Given the same position (x, y, z coords) & dimension, it should always return the same hash.
The only exception I can think of is if somehow a single Waystone generates 2 hashes, 1 for the bottom block & 1 for the top block....

Yikes, I see what's up with the hashes...
#225

I encountered the same issue with the latest update:

wraith-waystones-3.1.3+mc1.20 -> wraith-waystones-3.3.2+mc1.20.1


Fixed twitter issue

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

advocates/index.html.yml
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Flameshot clipboard doesn't work in sway.

Describe the bug
Copying to the clipboard doesn't do anything, nothing gets copied.
I am using copyq and I see that nothing shows up in copyq.
Steps To Reproduce
Steps to reproduce the behavior:

open flamework with "flameshot gui"
select area
press "Copy selection to clipboard"

Additional context
I am using sway.
Notify maintainers
 @scode @oxalica
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.53, NixOS, 22.11 (Raccoon), 22.11pre390948.5f43d8b088d`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.9.1`
 - channels(root): `"home-manager, nixos"`
 - channels(quantmint): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

I've also had this issue on X11 with bspwm with flameshot 11.0.0, pinning to version 12.1.0 from unstable nixpkgs resolved it for me.
It's possible this is a flameshot bug: https://github.com/flameshot-org/flameshot/issues/2626
There are also a few other clipboard related issues in the flameshot repo but they seem to be for non-linux OSes.

Upstream report: https://github.com/flameshot-org/flameshot/issues/2848
I have the same behavior on Plasma + Wayland.

Enabling wayland clipboard compilation flag helps, at least on KDE: https://github.com/flameshot-org/flameshot/issues/2848#issuecomment-1199643659
What's the best way to incorporate it upstream? I'm not sure we want to incorporate kguiaddons verbatim for everyone including GNOME users?


BinaryExpr evaluate lacks optimization for Or and And scenarios

Describe the bug
As shown in the code link, BinaryExpr directly calculates the left and right operands without considering the possibility that left might already be false and op is And, or that left might be true and op is Or.
For example, in this complex filtering statement, unnecessary calculations will be performed:
SELECT
    customer_id,
    first_name,
    last_name,
    email,
    total_purchases
FROM
    customers
WHERE
    1 = 0  -- This condition is always false
    AND (
        (total_purchases > 1000 AND EXTRACT(YEAR FROM registration_date) > 2018)
        OR (LOWER(email) LIKE<EMAIL_ADDRESS>AND loyalty_points > 500)
        OR (
            LENGTH(first_name) + LENGTH(last_name) > 15
            AND SUBSTR(last_name, 1, 1) IN ('M', 'N', 'O', 'P')
            AND DATE_PART('month', last_purchase_date) BETWEEN 6 AND 8
        )
    )
    AND (
        customer_id % 3 = 0
        OR EXISTS (
            SELECT 1
            FROM orders
            WHERE orders.customer_id = customers.customer_id
            AND orders.order_status = 'Completed'
            AND orders.order_total > 250
        )
    );

To Reproduce
When evaluating AND or OR expressions
Expected behavior
When an And expression is executed, a false occurs to stop the computation of the result.
When an Or expression is executed, a true occurs and the result is stopped.
Additional context
No

Could you give an example where this would make a difference? We have an optimization pass SimplifyExpressions which already takes care of true and false constants in AND and OR during planning.

possibility that left might already be false and op is And, or that left might be true and op is Or.

In general I think there is a tradeoff between doing short circuiting (what I think this ticket is describing) and having to check if each row should be evaluted
So for a predicate like (a = 5) AND (b = 10) it is very likely faster to evaluate (a = b) and (b = 10) with very tight loops / SIMD instructions and then && the resutlting boolean together that it would be to evalute (a = b) and then have a loop that checked the result for each row when evaluting b = 10.
However, for an example like (a = 5) AND (very_expensive_udf(b) = 10) it may well be faster to do short circuting execution
Note DataFusion already does short circuting for evaluating CASE
https://github.com/apache/datafusion/blob/4f4cd81de72a858896ac37a51b0e354cb379307c/datafusion/physical-expr/src/expressions/case.rs#L125-L187
SO TLDR I think this would be an interesting optimization to explore and as @Dandandan  notes finding some benchmarks where it matters is likely a good first step

Could you give an example where this would make a difference? We have an optimization pass SimplifyExpressions which already takes care of true and false constants in AND and OR during planning.

My apologies, my example might not have been the best illustration because cases with constants are indeed optimized by the rule. What I'm trying to get at is that there's currently unnecessary computation happening within BinaryExpr (perhaps in many cases, just calculating the left side would be sufficient to determine the result).

What I'm trying to get at is that there's currently unnecessary computation happening within BinaryExpr (perhaps in many cases, just calculating the left side would be sufficient to determine the result).

I would be surprise if there is any, could you provide the example that shows optimizer failed to do what you expect?
Here is the optimize rule that skip the right hand side if left is false
https://github.com/apache/datafusion/blob/3421b52605b00cd2e5a6498ea210cce196a19496/datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs#L848-L853

What I'm trying to get at is that there's currently unnecessary computation happening within BinaryExpr (perhaps in many cases, just calculating the left side would be sufficient to determine the result).

I would be surprise if there is any, could you provide the example that shows optimizer failed to do what you expect?
Here is the optimize rule that skip the right hand side if left is false
https://github.com/apache/datafusion/blob/3421b52605b00cd2e5a6498ea210cce196a19496/datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs#L848-L853

What I'm trying to describe is probably something like this.
SELECT
    customer_id,
    first_name,
    last_name,
    email,
    total_purchases
FROM
    customers
WHERE
    some_udf(first_name)  -- if this condition is mostly false at runtime.
    AND (
        (total_purchases > 1000 AND EXTRACT(YEAR FROM registration_date) > 2018)
        OR (LOWER(email) LIKE<EMAIL_ADDRESS>AND loyalty_points > 500)
        OR (
            LENGTH(first_name) + LENGTH(last_name) > 15
            AND SUBSTR(last_name, 1, 1) IN ('M', 'N', 'O', 'P')
            AND DATE_PART('month', last_purchase_date) BETWEEN 6 AND 8
        )
    )
    AND (
        customer_id % 3 = 0
        OR EXISTS (
            SELECT 1
            FROM orders
            WHERE orders.customer_id = customers.customer_id
            AND orders.order_status = 'Completed'
            AND orders.order_total > 250
        )
    );

possibility that left might already be false and op is And, or that left might be true and op is Or.

In general I think there is a tradeoff between doing short circuiting (what I think this ticket is describing) and having to check if each row should be evaluted
So for a predicate like (a = 5) AND (b = 10) it is very likely faster to evaluate (a = b) and (b = 10) with very tight loops / SIMD instructions and then && the resutlting boolean together that it would be to evalute (a = b) and then have a loop that checked the result for each row when evaluting b = 10.
However, for an example like (a = 5) AND (very_expensive_udf(b) = 10) it may well be faster to do short circuting execution
Note DataFusion already does short circuting for evaluating CASE
https://github.com/apache/datafusion/blob/4f4cd81de72a858896ac37a51b0e354cb379307c/datafusion/physical-expr/src/expressions/case.rs#L125-L187
SO TLDR I think this would be an interesting optimization to explore and as @Dandandan notes finding some benchmarks where it matters is likely a good first step

Adding short-circuiting after the left calculation alone resulted in performance improvements in my local TPCH tests.
commit link
Here are the test results for my local TPCH run (on my M3Pro chip with 36GB of RAM):
Comparing main and add_short_circuit
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃     main ┃ add_short_circuit ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │  89.54ms │           90.81ms │     no change │
│ QQuery 2     │  24.09ms │           20.56ms │ +1.17x faster │
│ QQuery 3     │  37.53ms │           38.00ms │     no change │
│ QQuery 4     │  23.99ms │           23.73ms │     no change │
│ QQuery 5     │  61.97ms │           54.52ms │ +1.14x faster │
│ QQuery 6     │  21.62ms │           20.98ms │     no change │
│ QQuery 7     │  75.46ms │           75.51ms │     no change │
│ QQuery 8     │  49.76ms │           50.09ms │     no change │
│ QQuery 9     │  68.53ms │           68.13ms │     no change │
│ QQuery 10    │  61.78ms │           60.36ms │     no change │
│ QQuery 11    │  13.16ms │           13.55ms │     no change │
│ QQuery 12    │  34.62ms │           34.35ms │     no change │
│ QQuery 13    │  41.50ms │           42.54ms │     no change │
│ QQuery 14    │  32.07ms │           30.29ms │ +1.06x faster │
│ QQuery 15    │  43.47ms │           43.82ms │     no change │
│ QQuery 16    │  18.59ms │           16.59ms │ +1.12x faster │
│ QQuery 17    │  97.63ms │           95.67ms │     no change │
│ QQuery 18    │ 115.07ms │          111.64ms │     no change │
│ QQuery 19    │  53.83ms │           57.37ms │  1.07x slower │
│ QQuery 20    │  45.42ms │           44.09ms │     no change │
│ QQuery 21    │  82.97ms │           82.69ms │     no change │
│ QQuery 22    │  16.46ms │           13.59ms │ +1.21x faster │
└──────────────┴──────────┴───────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (main)                │ 1109.06ms │
│ Total Time (add_short_circuit)   │ 1088.88ms │
│ Average Time (main)              │   50.41ms │
│ Average Time (add_short_circuit) │   49.49ms │
│ Queries Faster                   │         5 │
│ Queries Slower                   │         1 │
│ Queries with No Change           │        16 │
└──────────────────────────────────┴───────────┘

Thanks for providing some benchmarks / example @acking-you !
I think the missing optimizations seem to be:

Checking for array.true_count() to be all zero / all true. I think this is an optimization that might be better to implement in the arrow-rs and/or (and_kleene, or_kleene) kernels as a special case instead of in DataFusion.
UDF -> Immutable/non-volatile are executed during optimization in the constant evaluator, so the extra optimization will only apply for non-volatile UDFs.

Checking for array.true_count() to be all zero / all true. I think this is an optimization that might be better to implement in the arrow-rs and/or (and_kleene, or_kleene) kernels as a special case instead of in DataFusion.

I think the root cause is that left and right are calculated first, and then further calculations are performed based on op. The specific code is in the link below:
https://github.com/apache/datafusion/blob/main/datafusion/physical-expr/src/expressions/binary.rs#L260-L261
To make changes here, you'll need to modify the DataFusion code.
Possible Solutions: commit link

Thanks for providing some benchmarks / example @acking-you !
I think the missing optimizations seem to be:

Checking for array.true_count() to be all zero / all true. I think this is an optimization that might be better to implement in the arrow-rs and/or (and_kleene, or_kleene) kernels as a special case instead of in DataFusion.
UDF -> Immutable/non-volatile are executed during optimization in the constant evaluator, so the extra optimization will only apply for non-volatile UDFs.

I'm not sure if modifying and / or kleene logic help, since it needs two boolean array, but in this case we expect to short circuit if the left hand side has boolean array with all false for AND clause.
I think we might need to extend expression simplify logic for array case in Simplifier 🤔

I'm not sure if modifying and / or kleene logic help, since it needs two boolean array, but in this case we expect to short circuit if the left hand side has boolean array with all false for AND clause.

In the kernel we can add a fast case as well, just as @acking-you has done in his code . The benefit of this is that it will apply for any place that uses and/ and_kleene/or/or_kleene rather than in BinaryExpr alone.

I'm not sure if modifying and / or kleene logic help, since it needs two boolean array, but in this case we expect to short circuit if the left hand side has boolean array with all false for AND clause.

In the kernel we can add a fast case as well, just as @acking-you has done in his code . The benefit of this is that it will apply for any place that uses and/ and_kleene/or/or_kleene rather than in BinaryExpr alone.

I agree that we can also optimize the kleene kernel, so we can early return the result.
But besides that we also need to find a way to avoid computation in datafusion.
Current code is like
let left: BooleanArray = expr.evalute()
let right: BooleanArray = expr.evalute()
let result = and_kleene(left, right) // we can optimize this too!

But ideally we could avoid computation of right
let left: BooleanArray = expr.evalute()
// left is always false
skip right evaluation // right evaluation is possible to be expensive
skip and_kleene

#11247 indeed did a good job, but it seems like a specialize hacking code, not sure if it is the best solution 🤔

Hm I get where you're going at, there is some more optimization possible if we add it to BinaryExpr as well

Hm I get where you're going at, there is some more optimization possible if we add it to BinaryExpr as well

I believe @Dandandan  filed https://github.com/apache/datafusion/issues/11262 to discuss this idea further

I did some more looking into the idea of using true_count in boolean kernels in arrow to speed things up (e..g special casing when BooleanArray::true_count and BooleanArray::row_count were the same to avoid actually computing the binary result
However, I concluded that since BooleanArray::true_count has to check all the values of the array anyways it might be nearly as fast to just do the binary operation (which is already quite fast) than to count the true/faslse values
https://docs.rs/arrow-array/52.0.0/src/arrow_array/array/boolean_array.rs.html#142


Question about change of type relations rules in regards to index signatures

From 1.6 spec, rules about subtypes/supertypes:

M is a string index signature of type U, and U is the Any type or S has an apparent string
index signature of a type that is a subtype of U.
M is a numeric index signature of type U, and U is the Any type or S has an apparent
string or numeric index signature of a type that is a subtype of U.

"U is the Any type or " was added here. But similar rules for assignability weren't changed. That is strange: until 1.6 assignability rules were more relaxed than subtyping rules. What is the rationale for that? Thanks!

The original change was performed in PR #4074. Why this isn't part of assignability, I'm not sure, but maybe @ahejlsberg can answer.

Well, an example at #4074 deals with assignability, not subtyping. So I guess it's just an error in spec, and changes should be moved or copied to assignability.

It's an error in the spec. The new wording should also be present in the section on assignability. Thanks for pointing it out.


Joined nodes not setting correct permissions on default-storage

So today I set up 3x Ubuntu 20.04.2 LTS servers (eksd01, eksd02, eksd03) and ran snap install eks --classic --edge on each. I joined 02 and 03 to 01 in turn by running eks add-node on 01, then running the produced eks join. All fine at this point.
Later, I'm playing around with the consul-helm chart, which creates a pod and associated pv/pvc. On eksd01, the consul server pod is able to start, but on the others it's failing with a message about not being able to write to a location. I go digging around and I find this issue suggesting it's permissions: https://github.com/hashicorp/consul/issues/3795.
# eksd01
ls /var/snap/eks/common/default-storage/ -alh
total 28K
drwxr-xr-x 7 root root 4.0K Apr  2 12:56 .
drwxr-xr-x 5 root root 4.0K Apr  2 07:53 ..
drwxrwxrwx 2 root root 4.0K Apr  2 12:56 consul-data-consul-eksd-server-0-pvc-a8abe292-c2d1-4890-a690-ecba8eff64b5
...

# eksd02
$ ls -alh /var/snap/eks/common/default-storage/
total 24K
drwxr-xr-x 6 root root 4.0K Apr  2 12:57 .
drwxr-xr-x 5 root root 4.0K Apr  2 08:23 ..
drwxr-xr-x 2 root root 4.0K Apr  2 12:49 default-data-default-eksd-server-1-pvc-bc7e3e99-49c8-41eb-ac50-87fe472537c5
...

Huh, weird, 0777 vs 0755.
I run chmod -R 0777 /var/snap/eks/common/default-storage/ and restart the failing pods. They come back happy, healthy and writing.
So yeah, I'm guessing it's something to do with joining nodes given the host running eks add-node hasn't had any issues.

It's just occurred to me this might be more appropriate as a microk8s issue; let me know and I'll close.

I would not recommend the storage class that comes preinstalled for a cluster setup as it uses hostpath and does not replicate data across nodes. You may want to run eks.init and enable one of the amazon's storage solutions or you may want to setup your own node shared storage.
On the issue you are reporting, yes it seems to be a bug.


Fill holes in data not working in 0.9.7

Hi,
I'm trying to use the functionality to fill holes in data in my line chart but it won't work. This is my config:
lineChartOptions:any = {     animation: false,     responsive: true,     fullWidth: true,     chartPadding: {       right: 40     },     lineSmooth: Chartist.Interpolation.cardinal({       fillHoles: true,     }),     low: 0   };
which looks like this:

How to I make this work? I get no errors and everything is working except for this.
On version 0.9.7

Tested with 0.9.8. does not seem to be an issue. http://jsbin.com/linegah/edit?js,output

Please re-open if issue persists

@gionkunz Thank you, will check later.


Add KrunalDarekar

Related issue

What changes does this PR do?
Screenshots (If any)
Questions

So I corrected the mistake in my machine locally and pushe the code to the branch, should I raise a pull request again or will it automatically update the change here? @MusfiqDehan

So I corrected the mistake in my machine locally and pushed the code to the branch, should I raise a pull request again or will it automatically update the change here? @MusfiqDehan

No need for another PR :) the current PR will be updated.


Cast value column to float in HHS before downstream operations

Description
The value column was getting dropped by the geomapper during a groupby, which seems to be due to a new pandas release (downgrading fixes the issue). Casting to a float prevents this from happening.
Changelog
Itemize code/test/documentation changes and files added/removed.

Cast value column in dataframe to a float right after it's generated

Fixes

Fixes #(issue)

@nmdefries the R tests (well, building the dependencies) ran for 81 minutes before github cut it off; I've restarted the job but if it happens again we may have to back out #1081


Fix yellow background on active elements in code review

This PR ensures that yellow background on .active element is applied only to actual code lines.
Fixes this issue:

ping LG-TM