Initial fallback for request with dynamic urls

Bug report
Description / Observed Behavior
I have functionality in the app which includes list with loading during scrolling + list filters.I am using useSWRInfinite with fallbackData option. The problem is that setting fallbackData makes data property from useSWRInfinite always set and in result between filter changes I see old data from fallback and no loading. It's replacing the list for a moment with original fallback list.
The problem is also that fallback option suggested in docs is object literal, where swr allows keys others than string, these keys are stringified by internal functions of swr (serialize). So in result where I have complicated server query with many params I am not able to replicate the key in form of string because swr doesn't do simple JSON.stringify by uses internal serialize function.
Both those problems create no possibility to use prefetched data. The hack needs to be applied which checks if we have first call of useSWRInifinite. But this also do not solve all isssues as the page in the end is not in cache.
Here is example with standard useSWR
https://codesandbox.io/s/useswr-fallback-behaviour-d9c8i
So yes I understand why it works like that - fallback is related with given key, so new key means fallback is applied. Problem is that I have one fallback for initial request, initial filters and this fallback does not apply to any other next filter. I understand why we cannot say it is fallback for initial request as useSWR doesn't see difference between one call and another outside of key.
I wanted to use fallback property but my key is serialised internally by swr and function for this is internal part of the lib. So I am stuck with that also. What I need to do is hack like this one:
https://codesandbox.io/s/useswr-fallback-behaviour-hacked-wrongly-6vbcf?file=/src/App.js
But this does not work as useEffect is called right after and we really have state change and "loading" instead of prefetched data. To deal with it we need some kind of counter like:
https://codesandbox.io/s/useswr-fallback-behaviour-hacked-counter-clve0?file=/src/App.js
Take a look now it does work as expected (almost) as there is one rendering with fallback when we firstly change the key (its a small second but even though kind of not wanted behaviour).
Finally the problem does not allow me to easily use useSWR with prefetched data. I cannot:

use fallback because it accepts only strings as keys, where I have array
I cannot define that fallbackData is only for first request in some group/category of keys

Why do you use array, use string as a key
I could do that, but the current fetcher works with array of query params and makes a call from it. Also this fetcher is not only used in one place, in result it means that such a change has a big impact:

we need to make proper url as a key in nextjs level
we need to change fetcher to accept url string not a list of arguments which makes different responsibilities of this function

This is problematic as most fetcher utilities allow on using them with separated url and separated query params. Current fallback forces manually query params concatenation.
And the last is - why useSWR accepts as a key any other thing than string if fallback doesn't support this. It looks like something is missed in the design.
Expected Behaviour
Possibility to use Map instead of object literals in fallback in order to not needing to serialize manually the key. This would allow to pass arrays in the same way we can do that as key in useSWR.
Maybe some way to define fallback for category or requests. A way to tag/categorize some group of keys, For example I would like to say useSWR(key, {category: 'users_list', fallbackForCategory: fallbackData} then fallback would be applied only for first request of this category.
Repro Steps / Code Example
The example issue ( fallback shows for every key change):
CodeSandbox - issue example
Example hack:
CodeSandbox - hack example
Additional Context
SWR 1.0.1

Hi, thanks for writing the description carefully! This is definitely something we need to refine in the future. However, currently you can use this to serialize array/object keys and then use it as the key of fallback:
import { unstable_serialize, SWRConfig } from 'swr'

<SWRConfig value={{ fallback: {
  [unstable_serialize(['array', 'key'])]: 'fallback_data'
}}}

It's not documented, but we will once it gets finalized.

Also could you elaborate more on the "category" idea? Is it similar to scopped/namespaced cache? Not sure if I'm following.

Hey @shuding many tnx for fast answer. Will check, but probably this saves the day for me. In terms of the idea I am not familiar with scopped/namespaced cache but it looks like similar idea. What I meant is I can define which category/namespace I am using and for this namespace set fallback. So not for some specific key, but for the whole namespace. In other words - namespace is empty - fallback is applied for the first row, key does not matters. In that way I could be not bother in checking what exact key had ssr, I would just define this category and say - this is fallback for first request.

Hey, unfortunately it does not solve the problem. There are two issues with it:

Browser request key has prefix like - $req$$inf$arg$ which I would need to append manually
Sending undefined is not possible as this is not JSON serializeble - what means we need to add $undefined manually

In result this two problems forces to concatenate unstable_serialize result with $req$$inf$arg$ and with eventual $undefined what makes the solution cumbersome and hacky. The only way I see now is to use string urls always for keys.

Hi! For useSWRInfinite, you need to use
import { unstable_serialize } from 'swr/infinite'

And then
fallback: {
  [unstable_serialize(getKey)]: [initialPageData]
}

Sending undefined is not possible as this is not JSON serializeble

If undefined is in the key array, it should be possible (please let me know if it doesn’t) because we’re not using standard JSON serialization.

Thank you @shuding . That works. One question more as it is my concern now. Fallback works fine, but it is never used as real data, in other words it makes better UX, but when we look at it from technical stand point I have one request on the server side, and right after landing in the browser I have another request. Yes fallback is shown but still it is not considered as real data, only as placeholder.
If we do that instead:
  useSWRConfig().cache.set(unstable_serialize(getKey)], [initialPageData]);

We have a cake and can eat it too. So we fully use the data fetched from server as legit fetched one. And this exactly we want I suppose in most cases. If the server call is done right before rendering there is no sense in having another call right after, as it totally makes the initial fetching unnecessary.
Ok so question is why in nextjs docs we have info about using fallback, and why there is no info that we can set the cache directly. Maybe this is wrong approach, if so why?

@macsikora I have exactly the same use case and exactly the same question.  Where are you putting useSWRConfig call ?

Sorry that I missed your previous comment @macsikora.
The reason that SWR doesn’t encourage writing to the cache is because that can be dangerous. Writing to the cache is a mutation, which can cause side effect or undefined behaviors (race conditions, unpredictable order and result, etc). However fallback won’t cause these side effects because there is no write behavior.
But you can still prefill the cache with these initial data, by doing it at initialization rather than explicitly write to it:
function App({ data }) {
  return (
    <SWRConfig value={{ provider: () => new Map(Object.entries(data)) }}>
      <Page/>
    </SWRConfig>
  )
}

https://swr.vercel.app/docs/advanced/cache#create-cache-provider

@shuding How does that relate to the serialised key though?

@Antonio-Laguna Not related, I was answering the question regarding writing to the cache verses serving as fallback.

We now have docs for cases like this now: https://swr.vercel.app/docs/with-nextjs#complex-keys
It’s marked as unstable because the API (wrapping with [] and then call a serialize function) is still not ideal, and we can probably find a better way to do this in the future. Any ideas are welcome!

@shuding the issue is that as @macsikora hints, I don't want to repeat a request for which I already have data provided as fallback and unless I'm missing something, it's unavoidable. The data is already rendered and already visible but a request is made nevertheless

That works. One question more as it is my concern now. Fallback works fine, but it is never used as real data, in other words it makes better UX, but when we look at it from technical stand point I have one request on the server side, and right after landing in the browser I have another request. Yes fallback is shown but still it is not considered as real data, only as placeholder.

@Antonio-Laguna With this approach, the extra request can be avoided if you have revalidateIfStale: false:
function App({ data }) {
  return (
    <SWRConfig value={{ provider: () => new Map(Object.entries(data)) }}>
      <Page/>
    </SWRConfig>
  )
}

By disabling that option, if the data exists (stale), and SWR will not revalidate when mounting.
However the revalidation on mount is still helpful for highly dynamic data, especially for SSG cases (getStaticProps), the data was fetched at built time and it might change when the webpage is actual visited.

@shuding thanks for getting back!
I don't think that works though. I think (could be wrong) you're proposing something for, let's imagine a "movie" page. In that case, the page is considered "static" as one URL matches one movie.
In my case, the page however is dynamic. It's a search page and has filters so only the initial combination should actually be avoided whereas if I change a filter value, I want SWR to kick in and fetch the data with the new value. I'm using useSWRImmutable and It's not SSG but SSR so it's all happening on getServerSideProps.
This is already happening except there's an extra request on mount which is what I want to avoid, it's not especially harmful but I don't need it since I know the data has just been fetched and it'll be extremely rare that I'd need it again.
Here's the rough code:
function MyComponent({ fallback }) {
  return (
    <SWRConfig value={{ fallback }}>
      <Page/>
    </SWRConfig>
  )
}

MyComponent.getServerSideProps = async function({ query, params }) {
  const searchArgs = [
    searchTerm,
    subtype,
    page,
    query.start_date,
    query.end_date,
  ];
  const dataKey = serializeSearchParameters( ...searchArgs );
  const fallback = {};
  fallback[ dataKey ] = await fetchData( ...searchArgs );

  return {
    fallback
  }
}

Hey @Antonio-Laguna so what I am doing is that I generate the key by unstable_serialize and fetch the initial data in getServerSideProps and I pass it down, then I have a wrapper component, kind like:
const config = useSWRConfig();
  if (!isServer) {
    // set only client cache
    config.cache.set(props.swrMyKey, [props.myData]);
  }

Very important is to not do it in server side, as without the condition you will end in situation that you will have a cache data in ssr, that also means this thing will grow in memory of the server and can lead to bugs.
And isServer can be simple typeof window === 'undefined'.

Hi! For useSWRInfinite, you need to use
import { unstable_serialize } from 'swr/infinite'

Thanks for the help. Its working fine, awesome library!!!!I
Andreas
P.S: I only wish the docs where up to date ;)


Invalid reference to [List]

Not sure if the issue is in docgen or in the dart-doc viewer.
Document references to parameterized types show incorrectly. For example, this  doc:
https://github.com/dart-lang/html5lib/blob/eaca2abbf149ab4d280ae94cb6e67480db421c0e/lib/parser.dart#L48
which says:
/// Parse the [input] html5 document fragment into a tree. The [input] can be
/// a [String], [List<int>] of bytes or an [HtmlTokenizer].
Comes up as:
Parse the input html5 document into a tree. The input can be a String, dart:core<dart:core> of bytes or an html5lib/tokenizer.HtmlTokenizer.
(see http://www.dartdocs.org/documentation/html5lib/0.12.0/index.html#html5lib/parser)
I would expect the dart:core<dart:core> to actually say List<int>

Thanks for the report. However, we don't plan to address this issue with dartdocgen. New dartdoc tool coming online: https://github.com/dart-lang/dartdoc


debian package that applies the patch any time slack-desktop is changed

Hi @widget-, what do you think of a debian package to keep slack-desktop patched?

Hi @widget- , any feedback on this pull request? I've been using it to keep the patch applied across debian updates. Thanks!

I think it exceeds the scope of this project by quite a lot. If this were a full framework, etc. (which I'd eventually like it to be) then maybe it would fit in better, but for now all that this package is is a css file. You could easily make a separate project that fetches this one's .css URL (or any css URL really) since they should have the same setup.
Also keep in mind that Slack 3.0 is not compatible with the current setup.


Add osx binary

Create new nuget jsreport.Binary.OSX with osx jsreport binary. This will make the sdk working also on mac.

Do you have an expected date for this?

I've just released the jsreport.Binary.OSX in pre-release. Would you be able to try it out?
Note that the phantom-pdf recipe won't work there if you use OSX Sierra, because the default jsreport uses phantomjs 1.9 which is not supported there. I will need to release also version with the phantomjs 2 for it.

The binary for osx is available.


config-linux: RFC 2119 wording for linux.resources.devices

Since #780, the spec has been more aggressively punting to the kernel APIs (vs. carrying local versions of kernel limitations).  For the properties touched by this commit, the JSON Schema changes in #690 were rejected as part of this punting approach.  However, before this commit, it wasn't clear exactly what kernel interface was being punted to.
With this commit, we replace the old inline docs with an explicit punt to the device whitelist controller, listing the exact actions that the runtime MUST take for given config values.  This allows for compliance-testing runtimes (#746, ensuring config portability between compliant runtimes) and makes it possible to validate a given config against a given kernel (e.g. Linux 4.11.1 only accepts a, b, and c as type characters).

Rebased around #846 with f43520d → 5a7ee3a.

Looking at this change, its clear we don't want the defaults that default to something insecure.  We should just make these things explicit by the user and provide things like -1 for wildcard.


CUDA error: unknown error

When I finetuned the G-LLava on 8 A100s, I met such a problem several times.
The full trace is here
https://drive.google.com/file/d/195PO96uWKnx4LE3BWjxm0DsrQxWbj3QP/view?usp=sharing
The script is here
https://github.com/pipilurj/G-LLaVA/blob/main/scripts
It worked well for finetuning the first stage using run_alignment.sh. But when I finetuned the second stage using run_qa.sh, I met the aboved problem.  Now when I input "nvidia-smi" in the terminal, it shows "Unable to determine the device handle for GPU 0000:4F:00.0: Unknown Error". Can anyone please help me solve my problem? Thank you!

@liuhui0401 - this seems like a cuda error, or a bad state that the GPUs are in. If you power cycle the machine, does nvidia-smi work?

@liuhui0401 - this seems like a cuda error, or a bad state that the GPUs are in. If you power cycle the machine, does nvidia-smi work?

Yes. But if I finetune again, I will meet the same problem again. I don't know the reason.

I see, what cuda version are you using currently and can you try with a newer version as well?

Closing as stale, if you are still hitting this, please comment and we can re-open.


Support for converting YAML to JSON Package Manifests

Support for converting YAML to JSON Package Manifests

/azp run

/azp run


luci-proto-ipip: add nohostroute configurable

Add nohostroute configurable to luci-proto-ipip.
Also explicitly set this option, and the previous df configurable, to optional.
linked pr (merged) openwrt #4970

Thanks


Allow authenticated SOCKS5 connections without a password

Hi.
The SOCKS5 protocol does not require the password to be non-empty. Some SOCKS5 server implementations authenticate the client by a username only, so it makes no sense to require a user of the library to provide any non-empty password just to make authentication work.

The socks5 protocol doesn't allow empty password for USERNAME/PASSWORD authentication method. See  https://www.rfc-editor.org/rfc/rfc1929
In any case this PR doesn't solve any issues


CTHUB - Save/Update Users on Admin Page

Describe the task
the frontend looks ready but currently is not saving users or permissions. We need to add create/update functionality on the backend and frontend
Purpose
so that the admin user can add and update users
Acceptance Criteria

[x] as an admin user
[x] I can save new users and update existing user permissions
[x] so that users can upload or make permission changes themselves

Additional context
we need to figure out the rules for removing admin permissions from users. Maybe a user cannot remove their own admin permission?

It looks great. The only thing is the red "x" is not able to delete a user. I'm not sure if this was requested. Would you like a new card created?  @emi-hi

Card 249 has been created for the "x" function.


Problem: Train on my own dataset.

Thanks for your excellent work！
I want to train my own dataset using tridentnet. I convert my dataset to coco and run create_coco_roidb.py , change gpus and num_class in config. Do I need to change another parameters?
And I meet the problem when run
python detection_train.py --config config/tridentnet_r101v1c4_c5_1x.py
Traceback (most recent call last):
File "detection_train.py", line 290, in
train_net(parse_args())
File "detection_train.py", line 21, in train_net
transform, data_name, label_name, metric_list = config.get_config(is_train=True)
File "/media/fangjt/3dc196fc-1311-49bb-ad24-5f77d8735211/simpledet/config/tridentnet_r101v1c4_c5_1x.py", line 132, in get_config
num_branch=Trident.num_branch, scaleaware=Trident.train_scaleaware)
File "/media/fangjt/3dc196fc-1311-49bb-ad24-5f77d8735211/simpledet/models/tridentnet/builder.py", line 84, in get_train_symbol
rpn_loss = rpn_head.get_loss(rpn_feat, rpn_cls_label, rpn_reg_target, rpn_reg_weight)
File "/media/fangjt/3dc196fc-1311-49bb-ad24-5f77d8735211/simpledet/models/tridentnet/builder.py", line 310, in get_loss
image_anchor = p.anchor_assign.image_anchor
AttributeError: 'NoneType' object has no attribute 'image_anchor'

Sorry, this is a regression introduced by #224. Please change anchor_assign to anchor_generate

Close by #228


temperature offset conversion

Hello Tiago,
I’am using your library in my wifi energy meter project. Great work, saved me a lot of time dealing with cs5490. But I have an issue with the temperature offset. Reading the cs5490 datasheet I found “… with the binary point to the right of bit 16.”, but in the library you have “toBinary(23,MSBsigned,value);”. Can you check this.

Hello!
Yeah you are right, I'll fix it. Thanks!! : )


activity: only render logs if we need them

Classic case of rendering nouns that we won't print unless a flag is set. Noun renders can be quite slow for bigger nouns, and turns out we're printing a good handful of those. This should resolve the worst of the performance problems people were seeing.
(Activity still isn't fast by any means, summarizing unreads may still take over a second, but the impact of noun rendering was just egregious.)

Any objections to merging? Suffering on wannec atm


Grid inside ViewCell

I used SimpleGalleryPage.xaml as my start-point.
I wanted to make template looks like :
<Grid Padding="3">   <Grid.ColumnDefinitions>     <ColumnDefinition Width="1*" />     <ColumnDefinition Width="2*" />   </Grid.ColumnDefinitions>    <ffimageloading:CachedImage HeightRequest="100" Aspect="AspectFill" Grid.Column="0"      DownsampleHeight="100" DownsampleUseDipUnits="false"        LoadingPlaceholder="image_loading.png" ErrorPlaceholder="image_error.png"      Source="{Binding ImageUrl}"/>     <Label x:Name="Label" HorizontalOptions="Fill" HorizontalTextAlignment="Center"     Text="{Binding FileName}"  Grid.Column="1"/> </Grid>
First its looking ok, then after like 0.5s it gets messy.

It may be some Xamarin.Forms layout system issue. We'll need some more code to help. I'll reopen it after that.


Angular 2 docker deployment throw prerendering error

I got issue when try deploy to docker.
I run  dotnet publish which
"scripts": {
    "prepublish": [ "npm install", "npm run build:vendor", "npm run build:prod" ],
    "postpublish": [ "dotnet publish-iis --publish-folder %publish:OutputPath% --framework %publish:FullTargetFramework%" ]
  }

In my image, I copy publish folder to container. (FROM microsoft/dotnet)
When I run container which is CMD dotnet myapp.dll, it's throw error prerendering.
I had no idea with this issue.
any suggest? any advise?
Of course, I have try add node_modules and webpack config to container, but that is not best practice delivery with node_module since we have bundle webpack. Anyway not working, throw another error with need more add source typescript. nonsense.

Hi. I'm afraid I don't know what's wrong in this specific case, since I don't know exactly what you've included in your docker image, or what the underlying exception is. You could find out what the underlying exception is by running your application with UseDeveloperExceptionPage: https://docs.microsoft.com/en-us/aspnet/core/fundamentals/error-handling#configuring-a-custom-exception-handling-page
A futher thought, though: If you're trying to use docker for production deployment, please note that the templates already contain a working Dockerfile. If you build your docker image from this, it already works. I'm not sure what's different about the way you are building your docker image.
I'll close this because it doesn't sound like a bug (it sounds like an issue with your custom docker image). However, if you track this down futher and discover something you think is an issue in any of the code here, please let us know!

@hendrathings Are you using an old build of the template? I've seen this in the past. Does your webpack.vendor contain a client & server bundle? The server bundle should have vendor: ['aspnet-prerendering'] in there, if not then you may be using an old version of the template.

@MarkPieszak Ah, right. I found that server.ts need to compile as well then use asp-prerendering with compiled. I realize that old template. Thanks.


Files with --pre-archive are only copied if there are Volumes/Bind Mounts

First of all thank you for this great project. It works amazingly well.
Summary

Current behaviour: Using the --pre-archive flag only copies files if there are volumes configured.
Expected behaviour: Using the --pre-archive flag always copies files, if there are files available.

Context
The withFileSystemBind got deprecated and replaced with withCopyToContainer. As a result, most testcontainer module providers will therefore switch to using withCopyToContainer somewhere deep down in their file config helper methods, e.g.:
yourContainer = new ContainerFromModule("someImage")
    .withRamPercentage(75, 75) // some nice helper config function
    .withRealmImportFile("/your-realm.json") // some file helper function which will use `withCopyToContainer` in the end
    ...

These config files must be copied to the container before it is started. It makes therefore sense to use the --pre-archive flag to get these files copied so that we can continue profiting from these useful upstream helper functions because otherwise the alternatives are:

Use the --pre-archive flag and replace the helper functions with your own manual withFileSystemBind calls: This is not nice because you will have to do all the work of the upstream helper functions manually. See also https://github.com/joyrex2001/kubedock/issues/53.
Do not use the --pre-archive flag and therefore kubedock will copy the files after the container has already started and therefore you need to wait for the files to be copied using a custom command/entrypoint (See also https://github.com/joyrex2001/kubedock/issues/53).
Use the --pre-archive flag and a manual existing bind mount as shown below as a current work around

Current Manual Workaround
Just add an existing Bind mount and the --pre-archive flag works as expected and you can continue using the upstream testcontainer module's helper functions, e.g.:
yourContainer = new ContainerFromModule("someImage")
    .withCreateContainerCmdModifier(
         cmd ->
            cmd.getHostConfig()
            .withBinds
            (
                Bind.parse
                (
                    System.getProperty("user.dir") + "/pom.xml:/tmp/pom.xml" // any manual existing bind will do
                )
            )
    )
    ...

Thanks for the great work 👍


Deprecating untyped indexes

Untyped indexes shown to be unstable, deprecating.

Cool, congratulation, your first bug fix !


request

https://github.com/Rimmuru/Gta5-Mods-Lua-Loader/assets/155384494/350cac3f-4d97-49da-b958-ea2621d1c7bb
https://www.gta5-mods.com/scripts/mount-mod
is it possible for this to work?

looking into it as soon my gta decides to work

should be fixed :)

Alr i will check it as soon as i get home and let u know

so it doesnt give error when i run this but when i try to mount any ped it give me this:
https://github.com/Rimmuru/Gta5-Mods-Lua-Loader/assets/155384494/02604ed7-bb1d-438a-8565-25a671741bff

i swear.. i hate pointers in natives

download from main branch

now it's loading but nothing happens when im trying to open menu or just ride pet

issue appears to be with the mod itself it is from 2015 after all. i rebuilt the function still doesnt let you mount it. others in the comments of the post have said the same too

Well thats sad, thanks for trying anyway


Multiple select

I think it would be great idea to add selectMultiple(String...) to SelenideElement interface.
It should select all options listed in arguments.
example of usage:
$(String).selectOption("Option 1", "Option 2);

@kenshir007 Sure, why not. Do you mean "select by text" or "select by value"?
Probably it would be better to just add varargs to existing methods selectOption and selectOptionByValue? It would mean that you can write simply
  $("select").selectOptionByValue("aa", "bb", "cc");
  $("select").selectOption("Aaa", "Bbb", "Ccc);

Cool, I meant both, text and options, cause both of them may be required

I have add new method for retrieving all selected options:
select.getSelectedOptions().shouldHave(
        texts("Маргарита", "Theodor Woland"));

And these 3 methods now accept multiple arguments:
select.selectOption("Маргарита", "Theodor Woland");
select.selectOption(0, 2, 3);
select.selectOptionByValue("cat", "woland");

See https://github.com/codeborne/selenide/blob/master/src/test/java/integration/MultipleSelectTest.java


Unable to insert a link property during a nested insert

EdgeDB Version: 3.0-beta.1+2396772
EdgeDB CLI Version: 3.0.0-beta.1+a4e4d33
OS Version:

Steps to Reproduce:

When I do this query:

  name := "name",
  url := "url",
  path := "testpath",
  type := "monolingual",
  confidentiality := "",
  hash := "",
  paragraphs :=
    {
      (INSERT Paragraph {
        lang := (
            select Lang
            filter .LC_code = "en"
            ),
        hash := "",
        @paragraph_number := <in16>0,
        raw_text := "Text.",
      }
      ),
   }
};

I get the error QueryError: invalid reference to link property in top level shape. I imagine it is for @paragraph_number.
Schema:
    type Document {
        required property name -> str;
        property url -> str;
        required property path -> str{
            constraint exclusive;
        };
        property pdf_path -> str;
        required property hash -> str{
            constraint exclusive;
        };
        required property type -> str{
            constraint one_of('monolingual', 'bilingual');
        };
        required property confidentiality -> confidentiality;
        required property inserted_on -> datetime {
            default := (datetime_current());
        };
        multi link paragraphs -> Paragraph {
            property paragraph_number -> int16;
            property place -> str{
                constraint one_of('monolingual','left', 'right');
            };
        };
    }

    type Paragraph {
        required link lang -> Lang;
        multi link documents := .<paragraphs[IS Document];
        required property hash -> str{
            constraint exclusive;
        };
        required property inserted_on -> datetime {
            default := (datetime_current());
        };
        property raw_text -> str;
    }
};
    type Lang {
        required property LC_code -> str {
            constraint exclusive;
            constraint max_len_value(10);
        };
    }

This is a bug. In this case, you can fix by removing the curly braces as @angela-aura suggested since you're linking only a single object.
In the case that you need to insert multiple objects and link them, you can workaround this by using a for loop to loop through tuples containing your variables. So, in this case, that tuple might contain the language code filter value, paragraph number, and raw text of the paragraph.
I'm not sure of the priority, but I think this is bundled up in #4436 and will be addressed alongside a number of other link property issues when those are reworked. I believe a big part of the reason it hasn't been addressed yet is that reworking link properties is a relatively significant undertaking, and it doesn't make sense to try to squash all the individual bugs outside that larger rework.


CI workflow: execute e2e tests

@jsenko this is ready to go, altough there are some changes coming to olm bundle I think it's still ok to merge this, in worst case we have the temporary workaround to just run simple bundle (using /docs/resources/install.yaml) tests


status endpoint for attributes does not include features

Describe the bug
per spec https://github.com/many-protocol/specification/blob/main/attributes/network/9_account.adoc#features, the list of features should also be returned in addition to the attributes.

The list of features is available through the attribute’s arguments returned by the server’s status endpoint.

To Reproduce
Steps to reproduce the behavior:

./target/debug/many message --server  https://alberto.app/api status '{}'
response is:

{
    0: 1,
    1: "AbciModule(many-ledger)",
    2: h'a5010103270481022006215820e42d455843e8a18f2ee34ad9b807afc45652613630758740230080a4a5068f3c',
    3: 10000_1(h'01449aff697241dd16ce1d4a0e1719e0f78decee699b5db6158a333c91'),
    4: [0, 1, 2, 4, 6, 8, 9, 1002_1],
    5: "0.1.0",
    7: 300_1,
}

Expected behavior
attribute 9(account) in the response should also have features, for example.

This issue is more complex than previously thought if we want to fix it properly.
Features are not currently linked to modules, but they probably should since the modules drag them in.


Rails4 Support

This PR is used to track the process of adding Rails 4 support to DbCharmer. It is not ready yet, but we are really close.

Hi, any updates here?


Consider renaming to .js from .jsx

Cheers for the great work on this component.
The react native packager is unable to locate the .jsx files. I get the following error.

The module ./divider/divider-short could not be found from ./node_modules/react-native-settings-components/src/index.js. Indeed, none of these files exist:

I know React Native doesn't support .jsx naturally and I have tried the below configuration but it didn't work either.

// ./rn-cli.config.js

module.exports = {
  getSourceExts: () => [ 'jsx', 'mjs', 'js' ],
}

I will appreciate any guide on this.

@Kipko What version of React Native are you using? I have 0.57.2 and it works with this config

./rn-cli.config.js
module.exports = {
resolver: {
sourceExts: ['js', 'json', 'ts', 'tsx', 'jsx'],
},
};

The config changed in RN 0.57 because of a newer version of the metro packager.
See here: https://facebook.github.io/metro/docs/en/configuration#resolver-options
I don't consider renaming all .jsx files to .js because it clearly shows what file contains jsx and what doesn't as the AirBnB React/JSX Style Guide suggests.


[Idea] Crop face/entropy

Hi guys, this is another great Spatie's package. Thank you for all your work.
Using it, I had the idea to integrate two new manipulations:

[ ] Manipulations::CROP_ENTROPY
[ ]  Manipulations::CROP_FACE

Manipulations::CROP_FACE
Detect the first face and crop the image centered on it.
Manipulations::CROP_ENTROPY
Usually, the image part with the major pixel disorder is the most relevant. It crops, centering the disorder.
If you like it, I will work on PR.
Thanks

This is an excellent idea. Feel free to investigate this further.
Image you have a picture of two people sitting side by side. On the right we have Obama, on the left we have Trump. I'm thinking there should be some way of choosing on which face to focus on, otherwise some might think there is fraud going on in the election.
It would also be awesome if there would be a method that return the coordinates which were used for the crop. Those coordinates could be stored in a package like medialibrary and be used when generation conversions. But lets start with getting it right in this Image package first.
Thanks for your time on this, I'm excited to see what you come up with!

Image you have a picture of two people sitting side by side. On the right we have Obama, on the left we have Trump. I'm thinking there should be some way of choosing on which face to focus on, otherwise some might think there is fraud going on in the election.

Hahaha, yes, it's true. It could be an improvement after the first release.

It would also be awesome if there would be a method that return the coordinates which were used for the crop. Those coordinates could be stored in a package like medialibrary and be used when generation conversions. But lets start with getting it right in this Image package first.

Good idea 💅

@freekmurze, I lost my days thinking about these new features. Finally, I came to the conclusions I'm sharing with you.
intervention/image
It doesn't support features like "edge enhancement", "black threshold" and others: are available only with Imagemagick, no GD. I'm currently working on a PR to support all these features, but I don't know when and if they will accept it.
Package manipulations
This package has a lot of manipulations inside the class Manipulations.php. My idea is to create a manipulation interface implemented by any "real" manipulation (entropy, face, crop top right, etc), detaching all manipulations from a single class: creating a code easier to maintain. It means that the package could break to work and should be necessary to flag a new version, like 2.x.
I'm happy to share with you all these thoughts, to understand your philosophy about it.

Appreciate the update!

creating a code easier to maintain

I'm all for that! 👍


Add Logistic Regression algorithm

Signed-off-by: Xun Zhang<EMAIL_ADDRESS>Description
Add Tribuo Logistic Regression into ML-Common.
UTs and ITs of logistic regression are included in the PR. Addressed comments from the previous PR.
Issues Resolved
https://github.com/opensearch-project/ml-commons/issues/318
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #383 (d35cf05) into main (3a86a78) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##               main     #383   +/-   ##
=========================================
  Coverage     92.09%   92.09%
  Complexity      519      519
=========================================
  Files            58       58
  Lines          1481     1481
  Branches        116      116
=========================================
  Hits           1364     1364
  Misses           78       78
  Partials         39       39

Flag
Coverage Δ

ml-commons
92.09% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us.


Move documentation from preset-react

Documentation should be moved in the package README, see babel/babel#5051

👍


Fix build on latest nightly

global_asm and asm are now locked behind features on recent nightly.

Just updated to the most recent nightly version, and this isn't true...?
Projects using nx build fine for me

Actually, building with these flags shows warnings about both being stable since 1.59.0


BeautifulSoup complains about parser

To be completely honest, I have no idea why BeautifulSoup is yelling at my on my normal usage of your library, but I get it the first time I try to iterate through the generator that search() returns.
It looks to me like you're calling it correctly, or at least using it how  the error message indictates, so something must be wrong... ?
Any thoughts?
Ideally I'd like to squash this error message.
No parser was explicitly specified, so I'm using the best available HTML parser for this system ("html.parser"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.

To get rid of this warning, change this:

 BeautifulSoup([your markup])

to this:

 BeautifulSoup([your markup], "html.parser")

Oooh, it looks like your recent commit (e211d1d5570b0844375f79ec3ac2a29e1c858337) fixes this.  Can you push an updated version to PyPI?

@MarioVilas A friendly ping! :-)

Oops, sorry :) I'll update PyPy.


Dropdown still working even when button is disabled

The dropdown content still comes even when the button has a disabled attribute.
CODE:

download this app

download this app

download this app

download this app

download this app

             more

Saw that it was fixed already.


KeysBackup remaining issues and other fixes

This PR contains several things:

KeysBackup remaining issues (Fixes #2920)
Improve Login/Register Activity layout by using TextInputLayout and display errors inlined in the form
Update logo in LoginActivity to the blue one
Fixes #2917
Fixes #2916
Fixes #2893

Ok, fixed


Timesheet Create Example

Hi Team, this isn't an issue with the code - more hoping you can assist with the below.
Where can I see an example of a timesheet create for the demo apps you've created here?  I've attempted to modify other "create" routes in the examples here but keep getting errors (see below).  FYI I'm running this with the demo company.
Any direction/help you can provide would be massively appreciated.
Many thanks in advance.
My Timesheet Create route:
@app.route("/payroll_au_timesheet_create")
@xero_token_required
def payroll_au_timesheet_create():
code = get_code_snippet("TIMESHEET","READ_ALL")
xero_tenant_id = get_xero_tenant_id()
payrollau_api = PayrollAuApi(api_client)

timesheetline = TimesheetLine(
    earnings_rate_id = "0a03847b-e537-4341-956b-2350a8c34a75",
    #tracking_item_id = "str",
    number_of_units = [8.0,8.0,8.0,5.0,4.0,3.0,8.0],
    updated_date_utc = dateutil.parser.parse("2020-08-09T00:00:00Z"),
    )

timesheet = Timesheet(
    employee_id = "7fb4f0aa-7a6f-45a1-acba-bf1c6e7deebd" ,
    end_date = dateutil.parser.parse("2020-08-09T00:00:00Z") ,
    hours = 24.0 ,
    start_date = dateutil.parser.parse("2020-08-03T00:00:00Z") ,
    status = TimesheetStatus.DRAFT ,
    #timesheet_id = "a112f73b-a6aa-48c6-a793-f344563b6a1c" ,
    timesheet_lines = timesheetline ,
    updated_date_utc = dateutil.parser.parse("2020-08-09T00:00:00Z") ,
    )

try:
    created_timesheet = payrollau_api.create_timesheet(
        xero_tenant_id, timesheet=timesheet
    )
except AccountingBadRequestException as exception:
    output = "Error: " + exception.reason
    json = jsonify(exception.error_data)
else:
    output = "Employee created with first name {}".format(
        getvalue(created_timesheet, "employees.0.first_name", "")
    )
    json = serialize_model(created_timesheet)

return render_template(
    "output.html", title="Timesheet", code=code, output=output, json=json, len = 0, set="payroll_au", endpoint="timesheet", action="create"
)

The error I'm getting:
TypeError: 'TimesheetLine' object is not iterable

worked it out


Add Daystar University

Add Daystar University
http://daystar.ac.ke/index.html

@nathanmkaya If your university emails end with @daystar.ac.ke, please rename lib/domains/ke/ac/daystar/daystar.txt to lib/domains/ke/ac/daystar.txt (extra "daystar" is not necessary). Thank you!

@nathanmkaya Pull request merged. Thank you!


Issuelist icon too high

Gitea version (or commit ref): 1.15.0+dev-108-g84f5a0bc6
Can you reproduce the bug at https://try.gitea.io:

[X] Yes (provide example URL): https://try.gitea.io/issues
[ ] No

Description
The issue list icon is now too high
Screenshots

Could fix this by sticking an ac in the class here:
https://github.com/go-gitea/gitea/blob/84f5a0bc62568b6d0a67969e460f4d41339a07d6/templates/shared/issuelist.tmpl#L5
which would make this:

The top edge of the icon should align with the top edge of the first text line.

I can not reproduce the original screenshot on latest Chrome. Can you try updating?
https://github.com/go-gitea/gitea/pull/15391#issuecomment-817647436

it persists.

the issue may be related to segoe ui on linux

Yeah must be something like that. It's kind of hard to align this properly. I guess I'd need to boot up a Linux VM to verify.

I think it's just Segoe UI - its baseline sits 2.5px below the svg height.
https://core.trac.wordpress.org/ticket/48423
I'm not certain how we can handle it except by abandoning segoe-ui. I might just remove it from my fonts as I only occasionally need it.

Do we know if Windows has the same problem?

The NotoSans looks like the baseline is perhaps 1px below the svg.


Recordedit using incorrect validation for float4 column type

I have float4 columns and recordedit is complaining about values like 123.5, showing a red box around the input field and tooltip like Please select a valid value. The two nearest valid values are 123 and 124.

I can only reproduce this "validation" state when typing a float into an integer input field. I looked through each HTML document to verify that the integer validator is only on integer (int2, int4, and int8) inputs. Anything marked as float4, float8, and numeric are treated as decimal inputs with a float validator.
Could you point to the recordedit page that you were having this issue on?

On synapse-dev, go to anything in Zebrafish:Image%20Region table where
the status column says "processed" so there are fractional values in a
bunch of columns near the bottom with "Min" or "Max" in their column
names.
When I open up recordedit on any of these, I see many red-outlined
fields.

I can reproduce this bug on FF on MacOS. It looks like FF is showing an HTML validation error by highlighting the field and popping up the "different style tooltip" with the validation error message from above (Please select a valid value. The two nearest valid values are 123 and 124.)
I can actually see this same validation error on chrome, but there is no "highlight" around the box that notifies the user there is a validation error (according to HTML not our app).
A simple solution for this incorrect validation error reporting, is to add step to the HTML tag. Step works like the following:
step="any" will allow any decimal.
step="1" will allow no decimal.
step="0.5" will allow 0.5; 1; 1.5; ...
step="0.1" will allow 0.1; 0.2; 0.3; 0.4; ...

This did turn up another issue that I hadn't noticed before which is that chrome does not show any UI change for invalid float input values. Furthermore, regardless of the browser, the float validation I wrote in angular is not working at all. If you type 123.5.6 into a float input, no validation error message is shown and the page can't be submitted. FF does highlight the input for you though (which I mentioned above).

The HTML validation part of this issue has been fixed. Float inputs have step="any" attached to them. Fixed in this commit.

Further changes have been merged into master to improve the validation functionality of float inputs and integer inputs. We were relying on a combination of HTML validators and angularJS validators. This caused some bugs when the HTML validators invalidated the values which were then not passed to the angular validators to trigger the error message conditionals.
Changes were made to both float and integer input validators to remove the HTML validators and rely solely on angularJS ones. Changes were merged in PR #1874 .
Float and integer inputs will now properly show an error when the user types a value including alphabetic characters as well as a numeric string that doesn't represent either numeric type (1.1.1 for example).


[DISCOVERY JS-CPG] Not blocking exception while loading CPG

While running the checkdiscoveryrules on pattern JS 1, an error is produced
The reason is that while loading the generated CPG (stored in ./out/test.bin) an exception is raised by Joern.
However, this exception does not seem to be blocking for Joern, at least in this case.
By running the joern query the pattern obstacle can be found.
Our framework, on the other hand, is raising an error everything Joern raises an exception

@pr0me should we change this and being more relaxed wrt these exceptions?
below you have the example run in Joern console vs the example run with framework commands
the framework would stop everytime the Joern output comprises the string Error in CPG generation (see OUT2)

OUT1: Example in Joern console
joern> importCpg("./out/test.bin")
Creating project `test.bin` for CPG at `./out/test.bin`
Creating working copy of CPG to be safe
Loading base CPG from: /tp-framework/workspace/test.bin/cpg.bin.tmp
Adding default overlays to base CPG
The graph has been modified. You may want to use the `save` command to persist changes to disk.  All changes will also be saved collectively on exit
The graph has been modified. You may want to use the `save` command to persist changes to disk.  All changes will also be saved collectively on exit

java.lang.NoSuchMethodError: 'scala.Option io.joern.jssrc2cpg.JsSrc2Cpg$.postProcessingPasses$default$2()'
  io.joern.console.cpgcreation.JsSrcCpgGenerator.applyPostProcessingPasses(JsSrcCpgGenerator.scala:24)
  io.joern.console.Console.applyPostProcessingPasses(Console.scala:394)
  io.joern.console.Console.$anonfun$importCpg$6(Console.scala:365)
  scala.Option.foreach(Option.scala:437)
  io.joern.console.Console.importCpg(Console.scala:363)
  ammonite.$sess.cmd0$.<clinit>(cmd0.sc:1)

joern> cpg.call("<operator>.fieldAccess").where(n => n.code(".*splice.*")).location.toJson
res1: String = "[{\"node\":{\"name\":\"<operator>.fieldAccess\",\"signature\":\"\",\"code\":\"array.splice\",\"typeFullName\":\"ANY\",\"columnNumber\":8,\"order\":0,\"methodFullName\":\"<operator>.fieldAccess\",\"_label\":\"CALL\",\"argumentIndex\":-1,\"dynamicTypeHintFullName\":[],\"dispatchType\":\"STATIC_DISPATCH\",\"lineNumber\":30,\"id\":132},\"symbol\":\"array.splice\",\"packageName\":\"\",\"nodeLabel\":\"CALL\",\"methodShortName\":\"handleServer\",\"methodFullName\":\"pattern_src_code.js::program:handleServer\",\"lineNumber\":30,\"filename\":\"pattern_src_code.js\",\"classShortName\":\"\",\"className\":\"\"}]"
joern>

OUT2: Example as framework command output
# joern --script tp_framework/core/cpgTest.sc --params name="./out/test.bin"
joern --script tp_framework/core/cpgTest.sc --params name="./out/test.bin"
executing /tp-framework/tp_framework/core/cpgTest.sc with params=Map(name -> ./out/test.bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/joern/joern-cli/lib/joernext-querydb-io.joern.ghidra-10.2.3_PUBLIC_20230208.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/joern/joern-cli/lib/org.apache.logging.log4j.log4j-slf4j-impl-2.19.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
Compiling (synthetic)/ammonite/predef/CodePredef.sc
Compiling /tp-framework/tp_framework/core/cpgTest.sc
Creating project `test.bin1` for CPG at `./out/test.bin`
Creating working copy of CPG to be safe
Loading base CPG from: /tp-framework/workspace/test.bin1/cpg.bin.tmp
Adding default overlays to base CPG
The graph has been modified. You may want to use the `save` command to persist changes to disk.  All changes will also be saved collectively on exit
The graph has been modified. You may want to use the `save` command to persist changes to disk.  All changes will also be saved collectively on exit
Error in CPG generation
script finished successfully
()

I think raising an error in the framework is still the right approach.
It's possible that we would be fine in some cases, like here, where the error occurs in the post-processing passes.
But if it appears in earlier passes, information on which our queries depend might get lost and it's hard to check that.

Spotting when the exception was precisely raised can be challenging, especially if we just used the joern cli.  Two questions:

can the exception be visible from the cli?
if yes, would you have a set of exceptions that you would consider not so problematic?


Add data() method to CDataStream (and use it)

https://github.com/bitcoin/bitcoin/pull/9353

utACK


test: Ignore the auths tests on windows

Since the auths tests fail sporadically on the windows CI but we can't reproduce these failures with a real windows machine we are going to ignore these ones.
But we still ensure they compile.

bors merge

FYI @irevoire @ManyTheFish, this PR is not for v0.25.0 Milestones since you merged it into main :)


Updated CONTRIBUTING.md with labels [touch:22]

@MoOx What do you think? I think it's best not to have too many labels for starters. I combined some of yours, great suggestions btw.

@mxstbr feel free to correct any grammar/spelling misstakes or if you would like to add something :wink:

Left a bunch of comments!
Some of these labels are really self-explanatory, it seems like a waste of time to describe them with a sentence saying the exact same thing as the label, right?

Agreed, thanks.

All good for first set of labels?

I don't think labels need documentation. Labels are for collaborators only (so you guys) and you probably already understand them.

^ Good point @MoOx, agreed.

I agree but it's good to have something to refer with. It's already written, why waste it :smile:
Thanks though for check-in

You will have to maintain it now...

Fair point.


Is this an expected behavior: Logged in on hitting refresh

Hi,
Been doing a lot of PROOFing... just wanted to make sure that this is an expected behavior:
When I refresh my PROOF page I am still logged in...
I am just thinking from a security perspective thats all!
Please ignore if this is an expected behavior.

yes, that's expected behavior. we changed to this behavior maybe a few months ago to solve a number of problems

sometimes we'd have to reload the app to make sure it knows the server is up  after starting it - and before you'd get logged out if you did that
you can close the tab with the app and reopen and not have to log back in

from a security perspective you can't get to the app unless you're on campus or on a vpn - but i guess you could leave your computer in a public place someone could get it
any thoughts sita?

Totally makes sense why those changes were made. But it would perhaps be worthwhile to check with Jenny and Ty to see if this actually works from a Info-Sec perspective!

yeah I can ask them

ok, pinged them in slack

just adding a comment here that the log-in persists even now...

Jennifer and Ty got back to me about this - they asked InfoSec and they suggested 12 hrs. That seems too short, but maybe 24 hrs would be good - that's what the Cisco VPN does I'm pretty sure, which seems reasonable


frontend-14 homework on useref

Topics Covered:
-useRef
Output:

Didn't add the drift video
The commit message's format must be proper
Work on directory structure, modularity


Increase test coverage of WsServerContainer

Did you add "execute.validate=true" to your build.properties file ? Normally we don't use wildcard imports (please avoid that part).
Other than that, more test coverage = good.

Thanks for pointing that out. I've applied the fixes required.

okey, then all good

I'll merge this one.
However, it would be good to avoid adding pure spec tests in the future, as they are now supposed to be handled by the now-more-useable TCK (the TCK is to test all of the spec's assertions, so hopefully it would test most of these). OTOH, it is good to show reasonable coverage, so unless it is possible to add a TCK run into the coverage report [to see where we actually really stand], adding more tests to the testsuite still makes sense. I would say adding some basic spec tests to our testsuite is going to remain a good idea for a little while.
More stuff to think about basically ...


Communication package improvements

Improvements to the dmod.communication package, needed in preparation for other service additions and changes.

Note that some tests may not pass until the resolution of #134.


Update webvr-polyfill for WebVR 1.0 API changes

Per https://github.com/borismus/webvr-polyfill/tree/webvr-1.0

And also three.js' https://github.com/mrdoob/three.js/blob/dev/examples/js/effects/VREffect.js and https://github.com/mrdoob/three.js/blob/dev/examples/js/controls/VRControls.js have changed too

Yes. I have a few things on the backlog before merging:

Polyfill samples: build a WebVR and a THREE.js sample into the repo.
Boilerplate: inject UI into the polyfill element (if possible).
Add viewer param event on window broadcasting DeviceInfo and ViewerInfo. (we're thinking vrdisplaydeviceparamschange)

Just to be clear, this merge to webvr-1.0 will also greatly simplify webvr-boilerplate: https://github.com/borismus/webvr-boilerplate/tree/webvr-1.0

Woot.


Major slowdown with recent VS Code Update

This might be a shot in the dark, but after a recent (Feb 1st 2018) VS Code update (I'm now running Version 1.19.3 (1.19.3)), enabling the Better HAML extension causes a terrible slowdown during editing. I'm seeing crazy latency when typing (> 1 second per character). It's made this extension impossible to use.
Strangely, I experience the slowdown only when editing non-haml files (e.g. .rb). I don't experience the slowdown when editing HAML files.
Enabling/disabling the extension also fixes/causes the slowdown I'm experiences. So maybe it's a dependency of Better HAML that's to blame? I figured I'd start here though.

I ended up switching to "Ruby HAML" (vscode-haml-ruby) which is working just fine.

I don't get the slowdown, I've been using vscode to 1.19.3 since it come out and never got any problem.

I wonder what's different on my system. If there's anything I can do to debug this I'm more than willing.


Insert SPAR roles in FAM

Describe the task
Please add the following SPAR roles to FAM
SPAR Roles

Acceptance Criteria

[ ] first
[ ] second
[ ] third

Additional context

Add any other context about the task here.
Or here

Do as a hotfix target March 24th.

Need to confirm if all the roles require a forest client (concrete roles /abstract roles)

the ALT-supervisor one requires a forest client as well.


Release v2

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: 4679aa4aa3a2ba300e9dfb8babc7649571ee430a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


Better examples in the README here on github

I'm a reasonably profficient user of the shell, and the man page with listing of the flags are good. However, some concrete examples of the following would be nice, at least to me, and I'm probably not the only one I would think:

How to get butterfish to use previous instructions and answers? I  would think the answer would be using the shell mode.
the yaml idea is neat, but how do you use it? For some, this might be obvious. It isn't for me, and I'm not a newbie.
examples of how to use the index to generate responses based on local text. This is really exciting, and although it does seems possible to infer how to it  from the readme, I think an example could spell it out in a very nice way that would highlight this extremely useful feature.

Hope you find the input useful, let me know how I can assist you.

Hey @emilBeBri - this is great feedback, I will add details to the help here in the next few days. I think it's a good goal that you can understand the Butterfish from only the binary, rather than requiring referencing the readme.

Also to answer your first question:

How to get butterfish to use previous instructions and answers? I would think the answer would be using the shell mode.

Yes, shell mode is the right way to do this. Part of the basic functionality is to manage a history for you, so previous command output AND previous prompt input/output is visible.

Added detail in b26e0aa8239f29f7e2c399e415f6a71a84a013b1
@emilBeBri Please take a look and let me know if that's helpful. Very curious if that clarifies the use-cases or if you have more questions, let me know!


make git optional, make revision and remote mandatory

Motivation & context
Fixes #36.
Type of change

Refactoring/debt (improvement to code design or tooling without changing behaviour)
Bug fix (non-breaking change which fixes an issue)

Checklist:

[x] I have read the CONTRIBUTING document.
[x] My code follows the code style of this project.
[x] I have added tests to cover my changes.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have added an entry to the "Unreleased" section of the CHANGELOG, linking to this pull request.

Implementations

[x] TypeScript
[x] Java
[x] Ruby

TypeScript and Java done - would anyone mind doing Ruby?

TypeScript and Java done - would anyone mind doing Ruby?

Will do

I'll do the changes for ruby and js - or maybe David you would like to do it for js?
I'll take a look at the java impl too

Great work! I've release 8.0.0

I like the async team effort here 🤩


Bundle

This is a work-in-progress to expand the example application to be a replicated
mutable register. The goal is to have as simple an example as possible while
still demonstrating the power of distributed consensus. When done, the example
will also be useful for integration tests.

Looks great so far!

I've expanded this with a bunch of recent work.

@danburkert Looks good to me. Let me know when it's ready for an r+?

It's pretty much ready.  I will have more commits coming eventually, but they aren't necessarily related to these.

Ok, I'll review and merge.

@homu r+

:pushpin: Commit 7280656 has been approved by Hoverbear

:hourglass: Testing commit 7280656 with merge 32084d0...

:sunny: Test successful - status


[Question] run longhorn workloads on dedicated nodes only (taints, tolerations, affinity, nodeSelector)

hi!
we run longhorn installed via Rancher catalog on our cluster. We would like to run longhorn workloads (instance manager, engines etc) only on dedicated nodes, not on every node. The reason for this would be that we might be adding more nodes to the cluster, then remove that nodes or run experiments over that nodes that would result into longhorn failures (engine would stuck in deploying phase if we shutdown the node that used to have longhorn running). By far I can't find a way to lock longhorn within certain nodes.
Please advise,
Anton

Longhorn needs to be running on the node to provide storage on the node, so normally it's not recommended to limit the deployment for a subset of the node.
However, can you elaborate your issue with engine would be stuck in deploying phase? Longhorn is designed to scale well with the cluster, so if there is a problem with that part, we want to understand why it happened and potentially fix it.

so the scenario was (not exact, but i will be running more soon, and likely will have better steps to reproduce):

have longhorn installed via Rancher. everything is stable
add node(s) to the cluster - longhorn is getting installed
taint nodes in order to isolate them for heavy jobs (do not require longhorn, but it's already installed)
nodes are getting randomly rebooted/killed due to resources or kernel bugs
longhorn engine image becomes stuck in 'deploying' status, we are having intermittent issues on other nodes
longhorn is stabilised by reinstalling deployment

so our idea is to be able to tell longhorn what nodes it should run, so when we spin up nodes that don't need it - we don't have to deal with reinstalling it in order to respect taints that are set after longhorn is already installed

Using taints with effect NoExecute will evict the existing Longhorn workloads on the node immediately. Then rebooting/shutting down the tainted nodes won't affect Longhorn.
Rebooting/Killing the nodes on which the Longhorn workloads are running does lead to the longhorn engine image becoming 'deploying' status. In this case, waiting for the longhorn workloads back is enough and you don' need to reinstall Longhorn. If the Longhorn workload recovery somehow gets stuck, it may be a bug. You can file an issue for that then.

Another use case for supporting taints and tolerations is a cluster with windows and linux nodes where you want to use longhorn only on the linux nodes.

It doesn't work with Rancher with windows node enable. I bet people on longhorn don't talk to people on Rancher team lol

I just discovered there is a setting called "taintToleration".
if you set it like   taintToleration: "cattle.io/os=linux:NoSchedule"
It will work if you deploy from helm chart directly. But it doesn't work from Rancher App catalog.

@maxisam The taint toleration should work with the Rancher App too, once you set the default setting option?

@yasker I did. But somehow it doesn't work.

There is a defect with Longhorn taint: The toleration setting will be applied only after Longhorn workloads up in the cluster. In other words, if the toleration setting doesn't work and the Longhorn workloads can not be deployed if all nodes are tainted before Longhorn launching

@shuo-wu I don't think will work. I believe the way you said was the first time I did. I didn't taint windows nodes and I tried to fix it by setting toleration and affiliation after deployment. But toleration setting didn't stick. For a moment, it seemed working but it didn't work in the end. I remembered UI part did work. I really think it should be an easy fix from Rancher's side, just don't taint it.

@maxisam

Longhorn doesn't support Windows now.
The default setting doesn't work for the Longhorn upgrade. That's a default setting for the upcoming Longhorn system. Hence modifying the Longhorn toleration setting via Rancher App won't stick when there is already an (old) Longhorn system in the cluster.

@shuo-wu I totally aware Longhorn doesn't support Windows. I just need it run on Linux nods in a hybrid (Windows/Linux) cluster.
About 2. I think I didn't make it clear enough. I changed the settings on the deployment/daemonsets and scale the it down to 0 and scale back.  There were 2 changes I made, affinity and toleration. Affinity part works but not toleration part.

Related to https://github.com/rancher/rancher/issues/29220

Got it. Sorry for misunderstanding your comment. The 2nd issue will be tracked here: https://github.com/longhorn/longhorn/issues/1833

Longhorn needs to be running on the node to provide storage on the node, so normally it's not recommended to limit the deployment for a subset of the node.
However, can you elaborate your issue with engine would be stuck in deploying phase? Longhorn is designed to scale well with the cluster, so if there is a problem with that part, we want to understand why it happened and potentially fix it.

Hi folks! I need more info about @yasker answer above.
I was thinking on put longhorn in 3 dedicated nodes (they will exclusively will handle volumes and replicas, dashboard etc) and all other worker nodes could have its pods using PVs hosted at these 3 longhorn nodes. By @yasker answer, it isn't possible?
I have a cluster with Longhorn on all nodes (3) but I also have a bad application that create massive CPU spikes - when this happens every process suffers from CPU starvation and Longhorn in particular 'get lost'; it stops scheduling, then starts rebuilding replicas (lots of disk pressure) and the worst: many pods get their PVCs in read-only mode. And Longhorn recovers itself but I have to restart all affected workloads.
That why I was thinking about isolating longhorn on dedicated nodes.
Sorry about this lenghty question. Any thoughts are appreciated.
Regards,
Fabio Carvalho

@FCarvMobil  You need to set resource limitation.

Hi @maxisam. Those pods already have k8s resources set. The issue here is there are some paralelism with them and I can't set resources too low as they will take too long to complete.
Thanks for replying.

@FCarvMobil
You can have three dedicated nodes to provide storage, which means running replicas and stuff. But Longhorn needs to be run on every node to provide the connectivity to the volume. So if any workload on the node need access to the Longhorn volume, Longhorn needs to be run there.
Based on what you described, you need to shield Longhorn from this situation. You can try to set higher GuaranteedEngineCPU to see if it helps, which will translate into CPU Request for the key Longhorn pods. Notice that reset the value will restart all the volumes, so scale down the workload and detach the volume first.

Hi @yasker, thanks for replying.
Yes, the bottom line is that, to protect Longhorn from this situation. I'm already using the GuaranteedEngineCPU setting at the default suggested value (by the docs it should fit) but I'll try to increase it (aware to bring down workloads prior to this).
Sorry if I didn't get something about the architecture at the docs. I should run Longhorn in all nodes to provide connectivity to volumes, this implies on having replicas scheduled in each node? If not, how can I control on which nodes to allow/forbid replica scheduling?
Again, thanks!

The default value is a bit conservative since if we put too high a value then the instance manager may fail to start in user's environment.
We're also working on #1691 which should help us to get some guideline about how should we set the GuaranteedEngineCPU according to how many volumes will be used on the node.
Also, Longhorn manager needs to run on every node, but you can choose which nodes provide the storage (a.k.a has replica created). Any node without a disk set in Longhorn won't be used for replica scheduling. You can do that in the UI node page. Or you can set annotation on the Kubernetes node object to customize the default disk for the node. See https://longhorn.io/docs/1.0.2/advanced-resources/default-disk-and-node-config/
Taint toleration is indeed for dedicate nodes for Longhorn storage, but it's not required. See https://longhorn.io/docs/1.0.2/advanced-resources/deploy/taint-toleration/

I was thinking on put longhorn in 3 dedicated nodes (they will exclusively will handle volumes and replicas, dashboard etc) and all other worker nodes could have its pods using PVs hosted at these 3 longhorn nodes.

This would be great. But taints and tolerations seem not to be the right way for me. IMHO node-selector or affinity would be the right way for that.

Wait, it seems that https://github.com/longhorn/longhorn/issues/583 has fixed this by using labels.

Hi guys! Thank you all for replying.
I've seen that I can disable replica scheduling on Dashboard but the approach with nodeSelector is preferable so I haven't to manually configure anything on Dashboard - just add a new node with the appropriate label and it's done. #583 is a good reference!
I'll return here with my findings, still waiting an internal approval to build up a test scenario within our cloud provider account.
Again, thanks for replying.

cc @PhanLe1010
https://longhorn.io/docs/1.1.1/advanced-resources/deploy/taint-toleration/
https://longhorn.io/docs/1.1.1/advanced-resources/deploy/node-selector/

@PhanLe1010 This seems similar to #2199 which is implemented in v1.1.1. Is there anything more we should do here?

@yasker yes, this issue is covered by #2199 . We can close this one


Various warnings related to events

On hyprland 0.41.2+ds-1.1
An unknown event was passed into Hyprland-rs
            PLEASE MAKE AN ISSUE!!
            The event was: workspacev2
An unknown event was passed into Hyprland-rs
            PLEASE MAKE AN ISSUE!!
            The event was: movewindowv2
An unknown event was passed into Hyprland-rs
            PLEASE MAKE AN ISSUE!!
            The event was: configreloaded

Those are warnings from the hyprland-rs wrapper that I am using. Hyprland created new versions of basically all events and they are slowly migrating to them. You can ignore the warnings.

Yeah, I had some issues related to some types changes when serde de-serializing. Cool, thank you!


Dev Container Integration.

TODO:

[x] Multi-stage (prod and dev) Dockerfile (#263)

https://github.com/orgs/devcontainers/discussions/4#discussioncomment-4152158
https://github.com/microsoft/vscode-dev-containers/issues/304#issuecomment-1647101695
https://pdm-project.org/latest/usage/advanced/#use-pdm-in-a-multi-stage-dockerfile

[x] Pre-build images with GitHub Actions (#263)

https://containers.dev/guide/prebuild

[x] Pre-build images with GitLab CI/CD (#263)

https://containers.dev/guide/gitlab-ci

[x] Dev Container configuration with pre-build image (#281)

https://containers.dev/guide/dockerfile

[x] Dev Container badge (#282)

https://code.visualstudio.com/docs/devcontainers/create-dev-container#_add-an-open-in-dev-container-badge
DevContainer Workflow badge.

[x] Improve performance for .venv folder (#281)

https://code.visualstudio.com/remote/advancedcontainers/improve-performance

[x] make dev as Post start command (#281)

https://code.visualstudio.com/remote/advancedcontainers/start-processes

[ ] #280

https://code.visualstudio.com/remote/advancedcontainers/environment-variables

[x] Config VSCode Extensions. (#281)

https://containers.dev/supporting#visual-studio-code

[x] Trigger image built workflow only when changes happened in the prebuild devcontainer folder. (#263, #281)

Done for GitHub, depends on #264 for GitLab.

[x] #277
[x] Container Registry host adapted to GitHub and GitLab. (#290)
[x] Support manually trigger of devcontainer prebuild. (#286)
[ ] Documentation. (#305)

Dependencies:

The official DevContainer GitHub Actions lacks of build-args: https://github.com/devcontainers/ci/issues/270

NOTES:

No use for features and templates, try to be as general as possible, considering the scenarios that dev containers is not used but a dev/prod container image is still needed.

References:

https://code.visualstudio.com/docs/devcontainers/containers
https://containers.dev/

Closed as most of the tasks are done and those remaining (with external dependency) are tracked with separate issues.


add flag to allow skipping upload to transparency log

Summary

add flag to allow skipping upload to transparency log
we use cosign extensively with a private registry, and would like to
streamline the signing process to bypass the prompt to skip uploading to
a the transparency log with a private registry.
Signed-off-by: Kenny Leung<EMAIL_ADDRESS>Release Note

add flag to allow skipping upload to transparency log
usage cosign sign --key cosign.key --tlog_upload=false <IMAGE> to skip uploading to transparency log
Documentation

Yes, users can explicit decline upload to transparency log by passing the flag --tlog_upload=false flag. the --force flag takes priority if both are specified.

Codecov Report

Merging #2089 (98cc188) into main (00b1c2c) will decrease coverage by 0.04%.
The diff coverage is 4.76%.

:exclamation: Current head 98cc188 differs from pull request most recent head a5fdb31. Consider uploading reports for the commit a5fdb31 to get more accurate results

@@            Coverage Diff             @@
##             main    #2089      +/-   ##
==========================================
- Coverage   26.33%   26.29%   -0.05%
==========================================
  Files         129      129
  Lines        7564     7577      +13
==========================================
  Hits         1992     1992
- Misses       5317     5330      +13
  Partials      255      255

Impacted Files
Coverage Δ

cmd/cosign/cli/attest.go
0.00% <0.00%> (ø)

cmd/cosign/cli/options/attest.go
0.00% <0.00%> (ø)

cmd/cosign/cli/options/sign.go
0.00% <0.00%> (ø)

cmd/cosign/cli/sign.go
0.00% <0.00%> (ø)

cmd/cosign/cli/sign/sign.go
14.57% <12.50%> (-0.13%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us.


Remove output guard handler

Note
As we are planning to scope down the PaymentExitGame contract, we are removing some DEX supporting feature. This PR removes the output guard related logic, so now the PaymentExitGame only reads the outputGuard field as owner directly.

This also removed the extendibility e2e test on extend to DEX as that is not supported by current feature set of PaymentExitGame anymore.
Next step is to swap tx finalization with a simplified version and remove all confirm sig related optional args.
Reading with commit order is recommended. Will squash and merge for this PR.
The reducing scope plan: https://omisego.atlassian.net/wiki/spaces/RES/pages/129105932/Reduce+the+scope+of+Payment+ExitGame+V1

Warning
Would recommend clients to not integration with the result of this PR and wait for the simplification of txVerifier and then do the change at once to reduce some effort. However, it is still okay to migration to this one, just remove outputGuard related args.

checked, it is not missing any new branches. Both master and this PR is missing 3 branches. Just the total amount of branches decrease as a result lowering the coverage.

ref #501


esp-idf make defconfig uses env/esp-idf/components/mdns/Kconfig when /components/mdns/KConfig is available (IDFGH-12274)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.
v4.4.6-1-gb550086d0e
Operating System used.
Linux
How did you build your project?
Command line with Make
If you are using Windows, please specify command line type.
None
What is the expected behavior?
The KConfig options and limits specified in /components/mdns/KConfig should be used when present instead of env/esp-idf/components/mdns/Kconfig that came with esp-idf
What is the actual behavior?
During the build, the limit is set back to the default value of the env/esp-idf/components/mdns/Kconfig and the code is built with an unexpected MAX_MDNS_SERVICE
Steps to reproduce.

Copy the mdns component from env/esp-idf/components/mdns to /components/mdns
Change the CONFIG_MDNS_MAX_SERVICES  in /components/mdns/KConfig to have a larger range like this

         int "Max number of services"
-        range 1 64
+        range 1 255
         default 10

Create an sdkconfig.myproject that sets this value to 227
run these steps. cp sdkconfig.myproject sdkconfig.defaults && make defconfig

Build or installation Logs.
warning: user value 227 on the int symbol MDNS_MAX_SERVICES (defined at env/esp-idf/components/mdns/Kconfig:3) ignored due to being outside the active range ([1, 64]) -- falling back on defaults

More Information.
No response

Hi @rretanubun, unfortunately I couldn't reproduce this issue. After following these steps, make defconfig has succeeded and I got CONFIG_MDNS_MAX_SERVICES=227 in the resulting sdkconfig file.
Could you please run make list-components and verify that the mdns component is taken from the project components directory? Can you reproduce this issue also with a hello_world example, or only with your custom project?

Hi @igrr : Thanks for the reply. Here is my make list components output:
make list-components
Python requirements from /home/rretanubun/work/mmb/bayview/env/esp-idf/requirements.txt are satisfied.
GENCONFIG
Loading defaults file /home/rretanubun/work/mmb/bayview/sdkconfig.defaults...
warning: user value 227 on the int symbol MDNS_MAX_SERVICES (defined at env/esp-idf/components/mdns/Kconfig:3) ignored due to being outside the active range ([1, 64]) -- falling back on defaults
App "bayview" version: 0.10.8
*******************************************************************************
COMPONENT_DIRS (components searched for here)
/home/rretanubun/work/mmb/bayview/components
/home/rretanubun/work/mmb/bayview/env/esp-idf/examples/common_components/protocol_examples_common
/home/rretanubun/work/mmb/bayview/env/esp-idf/components
/home/rretanubun/work/mmb/bayview/main
*******************************************************************************
TEST_COMPONENTS (list of test component names)

*******************************************************************************
TEST_EXCLUDE_COMPONENTS (list of test excluded names)

*******************************************************************************
COMPONENT_PATHS (paths to all components):
<snip>
/home/rretanubun/work/mmb/bayview/env/esp-idf/components/mdns

So yes, the project appears to be misconfigured, I was expecting /home/rretanubun/work/mmb/bayview/components/mdns
Any hints on where I can check next? (I have not tried with hello_world example).

The list of component paths is built over here: https://github.com/espressif/esp-idf/blob/7a00499cc9ae66a81ab1720a1ca1c50f2b1a04b7/make/project.mk#L205
Looking at this code, I would check that 1) there is a component.mk file in mdns directory, 2) MULTI_COMPONENT_DIRS includes the components subdirectory of your project — you can edit the project.mk file to print this variable.
You can also add a debug print of the sub-expression over there which gets the list of candidate paths:
$(foreach cd,$(COMPONENT_DIRS),\
                            $(if $(findstring $(cd),$(MULTI_COMPONENT_DIRS)),\
                                 $(abspath $(dir $(wildcard $(cd)/mdns/component.mk))),)\
                            $(if $(findstring $(cd),$(SINGLE_COMPONENT_DIRS)),\
                                 $(if $(filter mdns,$(notdir $(cd))),$(cd),),)

(I have substituted $(comp) for mdns)


fix: scroll restoration

Scroll position was not resetting while navigating between pages. Fixed it by setting scrollBehavior. Then I realized it doesn't work nevertheless. I found its reason on StackOverflow.
Context:
https://stackoverflow.com/a/18573599
https://next.router.vuejs.org/guide/advanced/scroll-behavior.html

Wow, thank you! I didn't even notice this. Guess the same problem could apply to Vitesse. It would be awesome if you are interested in taking it a look as well. 🙌

Wow, thank you! I didn't even notice this. Guess the same problem could apply to Vitesse. It would be awesome if you are interested in taking it a look as well. 🙌


feat: add nvmrc and usage doc for contributing

Description of changes
Adds new dotfile, .nvmrc, to hold the suggested Node.js version for development. Also adds supporting documentation in CONTRIBUTING.md

Issue #, if available
ref #8407

Description of how you validated changes
Checklist

[x] PR description included
[ ] yarn test passes
[ ] Tests are changed or added
[ ] Relevant documentation is changed or added (and PR referenced)
[ ] New AWS SDK calls or CloudFormation actions have been added to relevant test and service IAM policies

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Hey y'all my apologies, totally missed these responses! Those are very valid points! Now that Node 16 is LTS would it be beneficial to change the hardcoded version to 14? @danielleadams I didn't see this note in the CONTRIBUTING doc

Now that Node 16 is LTS would it be beneficial to change the hardcoded version to 14?

Why would we hard code 14 instead of 16? I'm still a little skeptical about hard coding any version numbers. I foresee at least one Sev2 for us forgetting to update at some point.

Why would we hard code 14 instead of 16?

I'm not able to build the CLI with node 16, but maybe this is due to some local configuration. We could add lts instead of a strict version number

I'm not able to build the CLI with node 16

Ah, I see. I don't think it's an issue on your end. It looks like the version of node-pty-prebuilt-multiarch we use does not support Node 16 yet. It looks like they are working on it.
I was just able to build on Node 16 with<EMAIL_ADDRESS>but didn't do any testing to verify if it actually works as expected. It's also not ideal to depends on a pre-release if possible.

Hey @danielleadams I don't have strong opinions here, we can close


Update rook to latest master build.

For 4.8 preparation, we need a newer rook image.

rook update pulls in an update of the controller-runtime dependency from 0.6.3 to 0.8.0.
Seems we need to adapt our code to that.

Weeding out the build errors one by one...
I am left with two:
 make
Running deps-update
go mod tidy && go mod vendor
go: creating new go.mod: module tmp
go get: added sigs.k8s.io/controller-tools v0.4.1
Updating generated code
/Users/madam/go/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
Building the ocs-operator binary
hack/go-build.sh
# github.com/openshift/ocs-operator/controllers/storagecluster
controllers/storagecluster/external_resources.go:182:4: cannot use "60s" (type string) as type *"k8s.io/apimachinery/pkg/apis/meta/v1".Duration in field value
controllers/storagecluster/kms_resources.go:79:38: cannot use runtimeObj (type "k8s.io/apimachinery/pkg/runtime".Object) as type client.Object in argument to r.Client.Delete:
	"k8s.io/apimachinery/pkg/runtime".Object does not implement client.Object (missing GetAnnotations method)
make: *** [build-go] Error 2

Remaining error:
# github.com/openshift/ocs-operator/controllers/storagecluster
controllers/storagecluster/kms_resources.go:79:38: cannot use runtimeObj (type "k8s.io/apimachinery/pkg/runtime".Object) as type client.Object in argument to r.Client.Delete:
	"k8s.io/apimachinery/pkg/runtime".Object does not implement client.Object (missing GetAnnotations method)

Locally, I am getting this error now:
"error loading manifests from directory: error checking provided apis in bundle ocs-operator.v4.8.0: couldn't find ceph.rook.io/v1/CephNFS (cephnfses) in bundle. found: map[noobaa.io/v1alpha1/BackingStore (backingstores):{} noobaa.io/v1alpha1/BucketClass (bucketclasses):{} noobaa.io/v1alpha1/NamespaceStore (namespacestores):{} noobaa.io/v1alpha1/NooBaa (noobaas):{} objectbucket.io/v1alpha1/ObjectBucket (objectbuckets):{} objectbucket.io/v1alpha1/ObjectBucketClaim (objectbucketclaims):{} ocs.openshift.io/v1/OCSInitialization (ocsinitializations):{} ocs.openshift.io/v1/StorageCluster (storageclusters):{}]"

Why did all the CRDs get deleted in c428b8b?

Right that is weird... Investigating.

updated with the current state
still some lint/test compile errors left
and then to clarify why the crds got removed after updating the rook version

We are making progress. Now rook's image is fixed upstream via https://github.com/rook/rook/pull/7512 and the image target starts to pass. Still some things to fix from the controller-runtime update...

@jarrpa - I incorporated your patches on top of my PR and rebased. Looks a lot better! 😄 Only waiting for the e2e tests to complete now - everything else passed!

From the test output:
...
time="2021-04-01T09:38:40Z" level=info msg="Pod ocs-operator-bundle-e2e-aws-ipi-install-install succeeded after 34m10s"
time="2021-04-01T09:38:40Z" level=info msg="Executing pod \"ocs-operator-bundle-e2e-aws-optional-operators-ci-subscribe\" running image \"stable:cli\""
time="2021-04-01T09:38:50Z" level=info msg="Container place-entrypoint in pod ocs-operator-bundle-e2e-aws-optional-operators-ci-subscribe completed successfully"
time="2021-04-01T09:38:50Z" level=info msg="Container cp-entrypoint-wrapper in pod ocs-operator-bundle-e2e-aws-optional-operators-ci-subscribe completed successfully"
time="2021-04-01T09:49:40Z" level=info msg="Container sidecar in pod ocs-operator-bundle-e2e-aws-optional-operators-ci-subscribe completed successfully"
Error from server (NotFound): namespaces "openshift-storage" not found
OO_INSTALL_NAMESPACE is 'openshift-storage' which does not exist: creating
Installing "ocs-operator" in namespace "openshift-storage"
OO_TARGET_NAMESPACES is '!install': targeting operator installation namespace (openshift-storage)
OperatorGroup does not exist: creating it
OperatorGroup name is "oo-z6rqx"
Creating CatalogSource
CatalogSource name is "oo-dzq4l"
Creating Subscription
Subscription name is "oo-qjvv5"
Waiting for installPlan to be created
installplan.operators.coreos.com/install-vz8f9 patched
Install Plan approved
Waiting for ClusterServiceVersion to become ready...
Timed out waiting for csv to become ready
Dumping Namespace openshift-storage as /logs/artifacts/ns-openshift-storage.yaml
Dumping OperatorGroup oo-z6rqx as /logs/artifacts/og-oo-z6rqx.yaml
Dumping CatalogSource oo-dzq4l as /logs/artifacts/cs-oo-dzq4l.yaml
Dumping Subscription oo-qjvv5 as /logs/artifacts/sub-oo-qjvv5.yaml
  Subscription oo-qjvv5 status state: UpgradePending
ClusterServiceVersion was never created
Dumping all ClusterServiceVersions in namespace openshift-storage to /logs/artifacts/openshift-storage-all-csvs.yaml
Dumping all installPlans in namespace openshift-storage as /logs/artifacts/installPlans.yaml
{"component":"entrypoint","error":"wrapped process failed: exit status 1","file":"prow/entrypoint/run.go:80","func":"k8s.io/test-infra/prow/entrypoint.Options.Run","level":"error","msg":"Error executing test process","severity":"error","time":"2021-04-01T09:49:38Z"}
error: failed to execute wrapped command: exit status 1
time="2021-04-01T09:49:40Z" level=info msg="Container test in pod ocs-operator-bundle-e2e-aws-optional-operators-ci-subscribe failed, exit code 1, reason Error"
time="2021-04-01T09:49:40Z" level=info msg="Executing pod \"ocs-operator-bundle-e2e-aws-gather-aws-console\" running image \"pipeline:ocp-4.5-upi-installer\""
time="2021-04-01T09:49:51Z" level=info msg="Container place-entrypoint in pod ocs-operator-bundle-e2e-aws-gather-aws-console completed successfully"
time="2021-04-01T09:49:51Z" level=info msg="Container cp-entrypoint-wrapper in pod ocs-operator-bundle-e2e-aws-gather-aws-console completed successfully"
...

Not sure if this is a flake or real.
FWIW, I see pipeline:ocp-4.5-upi-installer - I think we shouldn't be running OCP 4.5 for testing OCS 4.8 / master, right?

From the test output:
...
time="2021-04-01T09:38:40Z" level=info msg="Pod ocs-operator-bundle-e2e-aws-ipi-install-install succeeded after 34m10s"
time="2021-04-01T09:38:40Z" level=info msg="Executing pod \"ocs-operator-bundle-e2e-aws-optional-operators-ci-subscribe\" running image \"stable:cli\""
time="2021-04-01T09:38:50Z" level=info msg="Container place-entrypoint in pod ocs-operator-bundle-e2e-aws-optional-operators-ci-subscribe completed successfully"
time="2021-04-01T09:38:50Z" level=info msg="Container cp-entrypoint-wrapper in pod ocs-operator-bundle-e2e-aws-optional-operators-ci-subscribe completed successfully"
time="2021-04-01T09:49:40Z" level=info msg="Container sidecar in pod ocs-operator-bundle-e2e-aws-optional-operators-ci-subscribe completed successfully"
Error from server (NotFound): namespaces "openshift-storage" not found
OO_INSTALL_NAMESPACE is 'openshift-storage' which does not exist: creating
Installing "ocs-operator" in namespace "openshift-storage"
OO_TARGET_NAMESPACES is '!install': targeting operator installation namespace (openshift-storage)
OperatorGroup does not exist: creating it
OperatorGroup name is "oo-z6rqx"
Creating CatalogSource
CatalogSource name is "oo-dzq4l"
Creating Subscription
Subscription name is "oo-qjvv5"
Waiting for installPlan to be created
installplan.operators.coreos.com/install-vz8f9 patched
Install Plan approved
Waiting for ClusterServiceVersion to become ready...
Timed out waiting for csv to become ready
Dumping Namespace openshift-storage as /logs/artifacts/ns-openshift-storage.yaml
Dumping OperatorGroup oo-z6rqx as /logs/artifacts/og-oo-z6rqx.yaml
Dumping CatalogSource oo-dzq4l as /logs/artifacts/cs-oo-dzq4l.yaml
Dumping Subscription oo-qjvv5 as /logs/artifacts/sub-oo-qjvv5.yaml
  Subscription oo-qjvv5 status state: UpgradePending
ClusterServiceVersion was never created
Dumping all ClusterServiceVersions in namespace openshift-storage to /logs/artifacts/openshift-storage-all-csvs.yaml
Dumping all installPlans in namespace openshift-storage as /logs/artifacts/installPlans.yaml
{"component":"entrypoint","error":"wrapped process failed: exit status 1","file":"prow/entrypoint/run.go:80","func":"k8s.io/test-infra/prow/entrypoint.Options.Run","level":"error","msg":"Error executing test process","severity":"error","time":"2021-04-01T09:49:38Z"}
error: failed to execute wrapped command: exit status 1
time="2021-04-01T09:49:40Z" level=info msg="Container test in pod ocs-operator-bundle-e2e-aws-optional-operators-ci-subscribe failed, exit code 1, reason Error"
time="2021-04-01T09:49:40Z" level=info msg="Executing pod \"ocs-operator-bundle-e2e-aws-gather-aws-console\" running image \"pipeline:ocp-4.5-upi-installer\""
time="2021-04-01T09:49:51Z" level=info msg="Container place-entrypoint in pod ocs-operator-bundle-e2e-aws-gather-aws-console completed successfully"
time="2021-04-01T09:49:51Z" level=info msg="Container cp-entrypoint-wrapper in pod ocs-operator-bundle-e2e-aws-gather-aws-console completed successfully"
...

Not sure if this is a flake or real.
FWIW, I see pipeline:ocp-4.5-upi-installer - I think we shouldn't be running OCP 4.5 for testing OCS 4.8 / master, right?

We're using OCP 4.7.
The CSV was never created, so it doesn't look like a flake to me.

/retest

Okay, the e2e failures may indeed be a flake. It seems it failed to even install the OCS CSV. Looking around, I found that the CatalogSource they generate to hold the CSV was in TRANSIENT_FAILURE:
time="2021-04-01T12:41:49Z" level=info msg="state.Key.Namespace=openshift-storage state.Key.Name=oo-bfs6t state.State=TRANSIENT_FAILURE"
time="2021-04-01T12:41:49Z" level=error msg="failed to list bundles: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp <IP_ADDRESS>:50051: connect: no route to host\"" catalog="{oo-bfs6t openshift-storage}"

So, as is often the case, we are probably dealing with network issues and/or API limits.
/test ocs-operator-bundle-e2e-aws

/retest

Created PR #1144 for only the CSV part of the rook upgrade.
Interestingly, it also fails e2e tests, similarly to this one.

rebased and updated to latest rook update corresponding to #1150

added DNM label. if tests pass, the patches need some polishing

/test ocs-operator-ci

seems the cluster (OCS/ceph) didn't come up in the test

Not sure if this is the problem:
  - lastTransitionTime: "2021-04-09T08:55:02Z"
    lastUpdateTime: "2021-04-09T08:55:02Z"
    message: csv created in namespace with multiple operatorgroups, can't pick one automatically
    phase: Failed
    reason: TooManyOperatorGroups
  lastTransitionTime: "2021-04-09T08:55:02Z"
  lastUpdateTime: "2021-04-09T08:55:02Z"
  message: csv created in namespace with multiple operatorgroups, can't pick one automatically
  phase: Failed
  reason: TooManyOperatorGroups

found in https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_ocs-operator/1141/pull-ci-openshift-ocs-operator-master-ocs-operator-bundle-e2e-aws/1380431051205120000/artifacts/ocs-operator-bundle-e2e-aws/ocs-must-gather/artifacts/ocs-must-gather/registry-build02-ci-openshift-org-ci-op-rig6vt5p-stable-sha256-ad1489e3d7d201d7a9cc908a56ef6e58ec02ec871ae18aa83c263b092d49d49d/namespaces/openshift-storage/operators.coreos.com/clusterserviceversions/ocs-operator.v4.8.0.yaml

/test ocs-operator-bundle-e2e-aws

same problem as before

The latest test failures seem to be because of insufficient permissions for the ocs-operator. This could be due to the update in the controller runtime.
The error message is
2021-04-16T18:58:02.029785222Z E0416 18:58:02.029733       1 leaderelection.go:325] error retrieving resource lock openshift-storage/ab76f4c9.openshift.io: leases.coordination.k8s.io "ab76f4c9.openshift.io" is forbidden: User "system:serviceaccount:openshift-storage:ocs-operator" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "openshift-storage"

Looking at the recent fixes in the kubebuilder repo I see https://github.com/kubernetes-sigs/kubebuilder/pull/1809 which seems to indicate that the kubebuilder wasn't adding the required permissions to the operator. It might be fixed in the next release of the kubebuilder but until then we can probably add the permissions manually.
I have added a commit(https://github.com/openshift/ocs-operator/pull/1152/commits/7852a8146c1f3cf45bd6fe8b6b00c312dc2b0e51) in my test PR which adds the required permissions and is also rebased on top of this PR. If it passes the tests please pick that commit.

@raghavendra-talur yay, your patch fixes it!
I'm going to polish this PR and then we can move forward.

When working on polishing / squashing the patches, I came across the test code that @jarrpa  commented out. Removing the comments lets the unit tests fail this way:
=== RUN   TestCreateSCCs
    ocsinitialization_controller_test.go:234:
                Error Trace:    ocsinitialization_controller_test.go:234
                Error:          Received unexpected error:
                                Operation cannot be fulfilled on ocsinitializations.ocs.openshift.io "ocsinit": object was modified
                Test:           TestCreateSCCs
                Messages:       [Case 1]: failed to update ocsInit status
--- FAIL: TestCreateSCCs (0.00s)

Not sure really how to fix this. But ideally, we should be commenting out such lines...

@jarrpa , @raghavendra-talur , @umangachapagain  - any idea?

When working on polishing / squashing the patches, I came across the test code that @jarrpa commented out. Removing the comments lets the unit tests fail this way:
=== RUN   TestCreateSCCs
    ocsinitialization_controller_test.go:234:
                Error Trace:    ocsinitialization_controller_test.go:234
                Error:          Received unexpected error:
                                Operation cannot be fulfilled on ocsinitializations.ocs.openshift.io "ocsinit": object was modified
                Test:           TestCreateSCCs
                Messages:       [Case 1]: failed to update ocsInit status
--- FAIL: TestCreateSCCs (0.00s)

Not sure really how to fix this. But ideally, we should be commenting out such lines...

No idea. I wasn't able to figure anything further out, hence why I had to leave it like that just to make progress. It has something to do with how the fake reconciler is implemented, but that is still fairly opaque to me.

@jarrpa - do you suggest to move forward with the commented-out lines?
(It just feels odd, since the lines seemed to have been there for a purpose... ;-)

For the sake of expediency, yes. There should at least me a TODO to figure out why they were suddenly a problem.

updated to the latest and greatest rook image rook/ceph:v<IP_ADDRESS>.gf4cfc7a
and polished the patches
still the only weird thing is the commented out test code lines... I added a TODO comment

/override ci/prow/ci-index-dev-master-dependencies


Add dayofyear Spark function

The function returns the day in the year value corresponding to the given number of days since 1970-01-01. It takes a DateType as input and returns an Integer. You can find the implementation of this function in Spark here. The unit test for this Spark function can be found here.

The same, can you add spark's link as reference?

@mbasmanova Can you help to review? Thanks.

Thanks.
The PR title was incorrect. It says "Add ... function", but the function existed before this PR. It should say "Fix ... function" and the description should describe the fix.

@mbasmanova Thanks for your help to fix. I have rebased to fix the conflict. Can you help to review again? Thanks.


This should be a more accurate exception or a type judgment mechanism when method get a wrong parameter

Expected behavior
IMHO，we should add some confirmation mechanism to ensure the useful method SingleThreadEventExecutor.shutdownGracefully(long quietPeriod, long timeout, TimeUnit unit) running in a right way or ensure the exception of Timeunit parameter can be correctly represented.

If this issue is necessary，may I sumbit a PR to fix it?

I am not sure what you are suggesting but I am happy to review a pr

I am not sure what you are suggesting but I am happy to review a pr

I think the easiest way is providing a more accurate exception description,rather than just throw a NPE.

honestly I think a NPE does exactly tell you what is wrong here.

Okay,I will close this issue.Thank you for your answer.


Update provider options doc for Sendinblue

We also update the link to the official API doc.

Advent of PRs? :)

Advent of PRs? :)

:sweat_smile: