stale-while-revalidate plugin memory leak

Some changes are style and clarity ... but by far the largest are based on:

Remove deps on TSHttpTxnNewCacheLookupDo() and re-Lookup of a URL

An old use of TSHttpTxnNewCacheLookupDo() would create multiple "nested" lookups that can fail if two have the same URL.   The nesting count was used to free memory at other times with consequences for a race situation (when the lookup did happen again).

Change TSHttpConnect() re-directed queries to HEAD from GET

This prevents a reliable in-core delivery of an entire asset [upon revalidate] and delegates an actual blocking 200 OK response up to the actual txn that found stale data.

Change the HEAD queries to use If-Modified-Since so the plugin can detect 304 instead of 200.

When those are received, even in the stale-on-error case, we can deliver the asset as fresh.  Without it, there's no distinction between 200 OK from a fresh copy or a local copy.

Hmmm, slightly confused. Is this a cherry-pick from master to 6.2.x branch? Or is this fix only needed on the 6.2.x branch? We generally don't make (and review) PRs on the release branches, everything should go on master first.

I am planning to put this on the master, but we do need it on a release .. custom for 6.2.x specifically.   We are moving to 7.0.x but not soon enough.
The problem does exist in all versions on through master.   This is the one we need first.   What is the best direction to go then if we have that need?   I'm hoping to put in the other PR soon for master directly.

I'm going to close this, since this is not the process we use. Please make a PR against master, once approved and committed, you make a new PR, cherry-picking the commit from master (with -x), and you can make a new PR and assign to Phil.

Ok, I missread, reopening.

Thanks for reopening.   I'm going to close this as is to focus on the Master PR.  It is split into more commits and that may make it more digestible anyway.
I have the "backport" for the "foreport" or whatever ready already ... when I get an opinion on it.


System.IO.Packaging version 8.0.0 transitive reference issue via DocumentFormat.OpenXml version 3.1.0 - CVE-2024-43483, CVE-2024-43484

Describe the bug
I encountered an issue with a transitive reference to System.IO.Packaging version 8.0.0 when using DocumentFormat.OpenXml version 3.1.0. The package reference appears as a warning in the NuGet package manager (screenshot attached).
Screenshots

To Reproduce
Steps to reproduce the behavior:

Add DocumentFormat.OpenXml version 3.1.0 to the project.
Observe the transitive dependency on System.IO.Packaging 8.0.0 in the package manager.

Observed behavior
A warning is displayed in the NuGet package manager.
Expected behavior
No warning or a clear explanation of the transitive dependency being safe to use.
Desktop (please complete the following information):

OS: Windows 11, Alpine 3.20.3
.NET Target: .NET Core 8
DocumentFormat.OpenXml Version: 3.1.0

Duplicate from https://github.com/dotnet/Open-XML-SDK/issues/1805

Hello,
I noticed that you have updated the package, but no release has been made. Could you let me know when it is scheduled?
Regards

I can see that there is now a 3.1.1 release.  However, I don't see the new version on nuget.org.  Any ideas?

Hi @udlose, 3.1.1 is available now on NuGet. Sometimes the feed can take a little while to update.


Idea: Internationalization support

I have been playing with i18next to add i18n to a Remix app, I was able to make it work but there are a few things which are missing or Remix could help solve better.

Loading localized strings (it could be way better)
Route meta (title, description, etc.) (missing)
URLs (add /:locale prefix automatically or change the whole URL based on the language) (missing)

Loading localized strings
The approach I used to load the localized strings for React i18next was to detect the language the user wants based on the request and then read a JSON from disk, to detect the language I used the following approach:

If the query has ?lng=:locale I use that
If there's a cookie i18next I use that
If there's an accept-language header I use that
I default to English

Once I got the locale from query/cookie/header/default I use the package accept-language-parser to set an array of supported languages and check if the user expected language is there, and default again to English if it's not. This means if the browser accept-language comes with es and en and I only support English it will default to English.
Then in entry.server I run this function to get the language and read the JSON from disk, the JSON is on public/locales/:locale.json.
Client-side, in entry.client I initialize i18next using the i18next-browser-languagedetector plugins which will use the same approach to get the user preferred language and the plugin i18next-http-backend to load the locale from /locales/:locale.json (this is why I put it on the public folder), once i18next has initialized I run the ReactDOM.hydrate function.
I need to wait for i18next to finish so the app will render with the correct language and not using the default while it finish to initialize, detect the language and load the locales.
If Remix have a built-in way to do i18n I would be able to remove a lot of code here, I would expect Remix to see what locales JSON files I have in a folder at the build time and then use a similar approach to what I did to know what locale to load. So a localized Remix app would have this folder structure:
/app
/locales
  en.json
  es.json
  ...
remix.config.js

Another option, would be to allow routes to define how to get the locales based on the request, this will be great to avoid loading locales I don't really need but it could mean I have to see how to ask only for those loaders and not just load everything and duplicate all the messages.
export let locales: LocalesFunction = async ({ request }) => {
  // somehow get the locales here
  // I could do whatever I want here like checking the query, a cookie, a header, the session or querying the DB if I stored the user preferred language
  // and finally I return a JSON with the locales
}

Then Remix could call this along the loader, the result (of both approaches) could be passed to the meta function and to the component of the route (in a context), inside that component we could use something like this:
export let meta: MetaFunction = ({ locales }) => {
  return { title: locales.title };
}

export default function View() {
  let t = useTranslations() // API inspired by react-i18next
  return (
    <>

{t("title")}

      {/* this Trans is also from react-i18next, is how to add components in the middle of a translation */}

<Trans message="copy" components={ strong:  } />

    </>
  )
}

Route meta
The title and meta:description of a route should be localized too, with the approach I'm using I can't do it because I only have access to the translations in the view, I would have to load the translations again in every loader to pass them to the meta function.
If Remix automatically loads the locales it could inject it to the route meta function, this could work in a per page or global (all locales in a single file) approach.
URLs
Another things that Remix could support is to add a prefix to all the URLs of the app based on the language, so the app/routes folder could be like this:
/app
  /routes
    index.tsx
    about.tsx
    something.tsx
    another.tsx

And Remix could generate the routes:
/en
/en/about
/en/something
/en/another

/es
/es/about
/es/something
/es/another

And basically add the locale before every route, then use that to detect the locale of the user, if the user access without that locale (e.g. /about) it could use the default language (defined in remix.config.js) and we could handle a redirect to the locale specific or not.
Another thing Remix could allow is to enable translate the pathnames, so /about becomes /sobre-mi in Spanish.

Internationalized Routing just works.  ref

Internationalized Routing just works. ref

That’s Next, this so about Remix

just a ref to look into

Hey !
I'm linking my own experiment here : https://github.com/BenoitAverty/remix-i18n-test
I've done this experiment with these requirements in mind :

Be able to load translations at runtime depending on env var, customer id, ...
Be able to load translations from anywhere (static json file, database, any translation provider)
Avoid loading the same translations twice in a request
Avoid fetching the same translations twice from the brower

I think this is important to consider if remix provides something built-in.
Language detection and routing should be customizable too I think.

I was looking into this as well. I was having a hard time translating my meta tags as they got translated back into the fallback language upon hydration. Turns out the meta function runs both in the server and in the browser and there was a mismatch on the active language. I ended up finding a hack (not too proud of it but it works haha).
Digging into the Remix's source code, turns out the route's meta function runs in a component scope so you're able to make use of hooks. If you have your I18n provider above the Document component you can do something like:
export const meta: MetaFunction = () => {
  const { t } = useTranslation();

  return {
    title: t('core:meta.main.title'),
    description: t('core:meta.main.description'),
  };
};

@olavoasantos what I do is to get the title inside the loader, and return it as data, then read the data inside the meta function

Hum... That's less bad than my hack I think haha
But you lose dynamic language switching, right? You'd need to refresh the page since the tags are coming from the server?

@olavoasantos if you make the language switch submit a form (it should) then all loaders are going to be reloaded after the submit automatically so you don't lose the benefit.

Makes sense. Thanks for the help!

I hope this idea to go into reality. I do love remix and am ready for Korean translation!

I just want to add a few things that I think are very important to consider if a :locale prefix is added to Remix:
Link, useNavigate, and redirect should support the /:locale prefix out of the box
If Remix implements an out-of-the-box localization solution, the Link component and useNavigate hook should add the :locale prefix to the To path property! Similarly, any server-side redirect should also add the :locale prefix to the URL.
Default language should not include an :locale prefix
The default / fallback language should not receive a :locale path prefix to avoid redirects (also important for SEO).
Illustration:
In the remix.config.json (or somewhere else), we define en-US as our default language. An English speaking user visits our domain ourdomain.com. Remix should not redirect the user to ourdomain.com/en-us/ but rather keep the user at ourdomain.com.
Pretty sure this is also important for SEO that there is not immediate redirect on the landing page for the google bots.
So instead of
/en
/en/about
/en/something
/en/another

/es
/es/about
/es/something
/es/another

Remix should generate
/
/about
/something
/another

/es
/es/about
/es/something
/es/another

I think Next.js does it this way as well.

I would be amazing if Remix could allow routes to be marked as "English / default language only". I have always missed that feature in other frameworks. It would allow devs to translate applications gradually and to keep /admin, /documentation, and similar subroutes in a single language.
In routes/__layout/documentation.tsx:
export const i18n = false

Remix then should remove the /:locale path prefix when the user enters the subroutes and switch the document language to "en" or whatever the default language of the application is.
Not sure if there are any major downsides to this (apart from a lot of work obviously) but the feature would be really helpful, I think!

Default language should not include an :locale prefix

I for myself prefer to add the language prefix to all, including the default/fallback, languages, although I know many don't like or do it that way, I prefer this ever since reading some Google Developers content on i18n or something, stating that it's not necessary but recommended for consistency and clarity, though can't find the page right now 😳😴
At the moment I'm doing i18n manually, tho so far I've got only english content 😅 (https://www.canrau.com)

Some recent projects out there with /:locale:

https://github.com/derhub/remix-rosetta-i18n - very simple implementation that basically uses window.location.pathname.slice(0, 4).replace(/\/+$/, '') === '/en'
https://github.com/AnandChowdhary/remix-starter - interesting project with en-us notation, also checks IP-based location as a fallback and some other gimmicks

Neither happens to be based on i18next (and I don't think either of them is meant for production, or at least there are obvious limitations).

Another thing Remix could allow is to enable translate the pathnames, so /about becomes /sobre-mi in Spanish.

That would be awesome!
What I haven't seen implemented in Remix yet is domain routing (like mysite.fr setting the language accordingly), but it wouldn't be far from those examples. Remix is uniquely suited for stuff like this.

What I haven't seen implemented in Remix yet is domain routing (like mysite.fr setting the language accordingly), but it wouldn't be far from those examples. Remix is uniquely suited for stuff like this.

I think you can use the request.url to detect the domain and get the correct locale based on that

What I haven't seen implemented in Remix yet is domain routing (like mysite.fr setting the language accordingly), but it wouldn't be far from those examples. Remix is uniquely suited for stuff like this.

I think you can use the request.url to detect the domain and get the correct locale based on that

That would be great to have a way to create routes like routes/about.tsx and map them to a locale to change its actual path depending on the locale like:

en-US : path = /about
fr-FR : path = /a-propos
etc.

And we would have to be able to set/detect a locale from request URL,domain, headers, (or whatever else) too:

mydomain.fr : locale =  fr-FR
mydomain.com : locale = en-US

I would have loved to try Remix over Next.js for a project, but the fact that routes can not be translated or associated/restricted to a locale 😭
Or maybe there is a way not simplified/config by Remix, but I did not get how to do it with custom code/config? 🤔

I'm doing routes/$lang/about.tsx for example then I get the lang from
params which I verify to be one of the supported languages and if it isn't
redirect to default (or something not exactly sure right now about the
logic) but you can check at https://github.com/CanRau/canrau.com
Also routes/index.tsx redirects to default lang but I might add some
language detection in the future.
On Tue, Feb 8, 2022, 13:58 Mathieu Dutto @.***> wrote:

That would be great to have a way to create routes like routes/about.tsx
and map them to a locale to change its actual path depending on the locale
like:

en-US : path = /about
fr-FR : path = /a-propos
etc.

And we would have to be able to set/detect a locale from request
URL,domain, headers, (or whatever else) too:

mydomain.fr : locale = fr-FR
mydomain.com : locale = en-US

I would have loved to try Remix over Next.js for a project, but the fact
that routes can not be translated or associated/restricted to a locale 😭
Or maybe there is a way not simplified/config by Remix, but I did not get
how to do it with custom code/config? 🤔
—
Reply to this email directly, view it on GitHub
https://github.com/remix-run/remix/issues/202#issuecomment-1032954239,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABHUZKYSIHQC22YSUPFGESDU2FRT5ANCNFSM466O3MUQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

As someone who has used both i18next and lingui, I'd like to give a shoutout to lingui. They let you write your strings inline with React components, including pluralization, and then it extracts ICU strings for you so you don't have to know that syntax yourself.
Unfortunately, it's based on Babel Macros, so if it were to be supported in Remix it would be through something like https://github.com/nativew/esbuild-plugin-babel (though I don't know if one must use the macro, possibly worth looking into or otherwise asking).


Implement mpsc/mpmc queue

cf. https://github.com/crossbeam-rs/crossbeam/pull/189

Note that this implementation is blocking: if a consumer doesn't increment the lap id after receiving a value, producers cannot send values.


add method to get data associated with particular trial

search done by the global trial index.

added to /dev


WebGL: initial documentation of js classes

Motivation and Context

We should document the js API.
How Has This Been Tested

Generated documentation with documentationjs. See below:

Types of changes

[ ] Docs change / refactoring / dependency upgrade
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[X] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[X] I have read the CONTRIBUTING document.
[X] I have completed my CLA (see CONTRIBUTING)
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

This is great!

Cool! Documentation.js has a JSON output format so theoretically this could be later plugged into (and crosslinked) the C++ and Python docs I'm just working on.

Awesome, thanks for getting docs started on the js side as well @msbaines !  This and a conversation with @aclegg3 reminds me that we should also merge in the long-dormant #71 .  I'll take a look into that soon.


Add support of constexpr, noexcept and move semantic for Visual Studio 2015+

Since Visual Studio 2015 constexpr and noexcept keywords are supported. Given commit enables those features.

Is this still relevant? @tmontgomery do you have a view on this?


Add displayName to typings and test cases

It appears displayNames are enabled globally; as such many endpoints include a displayName proprerty however our library does not accurately reflect this.
I would strongly advise updating our unit tests in addition to our typings to avoid issues such as #372, where types were added but our code artificially excluded displayName.

Resources such as jkmdev's API tracker should aid in this effort, using git grep displayName to show all additions of displayName, but we will still need to manually check all of the undocumented endpoints (economy, www, etc).


First integration documentation: Datadog

What changed, and why it matters
The integration with DataDog is in good shape but it handles metrics and logs separately, and they aren't always intuitive to set up. This adds a new "integrations" section, and some starter docs for our DataDog integrations. Fixes #773

No need to re-test, I wrote the docs and then (after I'd slept!) used them to set up the demo for the Let's Talk Aiven Tech session last week so they should be pretty much sorted.


slickRemove method not working

The slickRemove method is returning false and doesn't remove a slide a slide from the DOM.

Version: 1.4.1
Browser: Google Chrome Version 43.0.2323.0 canary (64-bit)

Found out that slick was being initialised  twice.


Update DiagnosticSource to 4.4 stable.

SDK version updated to 2.4.0
Update changelog

Codecov Report

Merging #574 into develop will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop     #574   +/-   ##
========================================
  Coverage    84.01%   84.01%
========================================
  Files          137      137
  Lines         6088     6088
  Branches       874      874
========================================
  Hits          5115     5115
  Misses         644      644
  Partials       329      329

Impacted Files
Coverage Δ

...annel/Shared/Implementation/TransmissionStorage.cs
90.57% <0%> (-1.45%)
:arrow_down:

...Core/Managed/Shared/Extensibility/MetricManager.cs
70.37% <0%> (+1.23%)
:arrow_up:

...metryChannel/Shared/Implementation/PlatformFile.cs
95.65% <0%> (+4.34%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3c655af...6a46332. Read the comment docs.


Bulk event trigger method having problem

When I try to use this method:
client = Novu::Client.new(<API-KEY>)
client.trigger_bulk_event(
  events: [
    {name: 'name', to: {subscriberId: 'asdfasdf'}, payload: {message: 'lorem'}},
    {name: 'name', to: {subscriberId: 'sdfgsdfg'}, payload: {message: 'lorem'}},
  ]
)

I am getting the following error:
{"data"=>[{"status"=>"error", "error"=>["identifier must be a string"], "acknowledged"=>true}]}

I noticed that there is an issue with the HTTParty when you want to submit an array as a body. It does not parse it properly.
https://github.com/jnunemaker/httparty/issues/719
I can help to address the issue if we can agree on a solution.
My suggestion is to make add headers: { 'Content-Type' => 'application/json' } to all or post requests and also, convert post body to_json. I think that is the only work around.
The harder solution is to use Faraday instead of HTTParty which is maintained better.
Thanks

The value of subscriberId should be a string @amirali-ashraf

Hi, I tried it by all means, and actually I am using string for all parts which is not working. I can write a quick spec test which can give you a better insight.

Please can I see your code snippet?

Of course!
require "bundler/inline"

gemfile(true) do
  ruby "3.1.2"
  gem "novu"
end

client = Novu::Client.new('YOUR-API-KEY')

res = client.create_subscriber({subscriberId: 'ghgh456fthy45633efwf324r', email<EMAIL_ADDRESS>puts res

res = client.create_subscriber({subscriberId: '2fwef234rwef4teghrt7657h', email<EMAIL_ADDRESS>puts res

res = client.trigger_bulk_event({events: [
  {name: 'in-app', to: {subscriberId: 'ghgh456fthy45633efwf324r'}, payload: {message: 'lorem'}},
  {name: 'in-app', to: {subscriberId: '2fwef234rwef4teghrt7657h'}, payload: {message: 'lorem'}},
]})
puts res

@amirali-ashraf I guess you will have to use .to_json for now.
If you check the spec/events_spec test file, .to_json was added to the body parameter as well.

@amirali-ashraf I understand your point of view now. I would love to see your PR to resolve this issue with the current httpparty that's being used.

@Eazybright Hey, thanks for you reply. My only workaround this issue is to post by converting the hash into json and set the header content to json where you post the data. Does that work for you?
Also, I think this would not be the only instance that you are sending arrays. Can you tell me if there is so we can fix them in one shot or you want them to be fixed in future?

that's the only instance for now

@amirali-ashraf i had the same issue with the bulk subscriber creation API. You can check my PR for the fix https://github.com/novuhq/novu-ruby/pull/39

raised PR  #45

raised PR #45

@unicodeveloper


html/template: Go 1.8 has a different encoding logic. Intentionally?

What version of Go are you using (go version)?
Go 1.8 beta (130ad87ab7b02a87401246d4a5f0c3f00fc7d709)
What operating system and processor architecture are you using (go env)?
Ubuntu, OS X
What did you do?

Test this script with Go 1.8

What did you expect to see?

Works fine with this version of Go

What did you see instead?

Strings do not match: got '', want ''

If that's how it should look like with Go 1.8, how can I get back the old behaviour?

I don't see any obvious commit during Go 1.8 which would've caused this, unless the change wasn't in the html/template package.
@rsc, do you know off hand?
commit 1e917312511f6ce54bbdcea8cd0c25e66973d49e
Author: Caleb Spare<EMAIL_ADDRESS>Date:   Sun Nov 13 18:06:16 2016 -0800

    html/template: fix multiple Clones of redefined template

    This change redoes the fix for #16101 (CL 31092) in a different way by
    making t.Clone return the template associated with the t.Name() while
    allowing for the case that a template of the same name is define-d.

    Fixes #17735.

    Change-Id: I1e69672390a4c81aa611046a209008ae4a3bb723
    Reviewed-on: https://go-review.googlesource.com/33210
    Run-TryBot: Caleb Spare<EMAIL_ADDRESS>    TryBot-Result: Gobot Gobot<EMAIL_ADDRESS>    Reviewed-by: Rob Pike<EMAIL_ADDRESS>
commit 2442b49c47aa818bbc55e4c064e9ea0ca058735f
Author: Marcel Edmund Franke<EMAIL_ADDRESS>Date:   Mon Nov 14 21:46:25 2016 +0100

    html/template: typo fix

    comment on unexported function starts with wrong functionname

    Change-Id: Ib16c2fe42b5a8d4606ed719f620923c17839d091
    Reviewed-on: https://go-review.googlesource.com/33203
    Reviewed-by: Brad Fitzpatrick<EMAIL_ADDRESS>
commit d8264de8683dac99ffbbbc1f46415e627b73c9ed
Author: Dmitri Shuralyov<EMAIL_ADDRESS>Date:   Wed Nov 9 14:49:12 2016 -0800

    all: spell "marshal" and "unmarshal" consistently

    The tree is inconsistent about single l vs double l in those
    words in documentation, test messages, and one error value text.

            $ git grep -E '[Mm]arshall(|s|er|ers|ed|ing)' | wc -l
                  42
            $ git grep -E '[Mm]arshal(|s|er|ers|ed|ing)' | wc -l
                1694

    Make it consistently a single l, per earlier decisions. This means
    contributors won't be confused by misleading precedence, and it helps
    consistency.

    Change the spelling in one error value text in newRawAttributes of
    crypto/x509 package to be consistent.

    This change was generated with:

            perl -i -npe 's,([Mm]arshal)l(|s|er|ers|ed|ing),$1$2,' $(git grep -l -E '[Mm]arshall' | grep -v AUTHORS | grep -v CONTRIBUTORS)

    Updates #12431.
    Follows https://golang.org/cl/14150.

    Change-Id: I85d28a2d7692862ccb02d6a09f5d18538b6049a2
    Reviewed-on: https://go-review.googlesource.com/33017
    Run-TryBot: Minux Ma<EMAIL_ADDRESS>    TryBot-Result: Gobot Gobot<EMAIL_ADDRESS>    Reviewed-by: Brad Fitzpatrick<EMAIL_ADDRESS>
commit ef3a9f2dd410ec1672988a72c72919eab2d58e6c
Author: Russ Cox<EMAIL_ADDRESS>Date:   Wed Oct 26 13:12:17 2016 -0400

    html/template, text/template: drop defined template list from errors

    The report in #17414 points out that if you have many many templates,
    then this is an overwhelming list and just hurts the signal-to-noise ratio of the error.

    Even the test of the old behavior also supports the idea that this is noise:

            template: empty: "empty" is an incomplete or empty template; defined templates are: "secondary"

    The chance that someone mistyped "secondary" as "empty" is slim at best.

    Similarly, the compiler does not augment an error like 'unknown variable x'
    by dumping the full list of all the known variables.

    For all these reasons, drop the list.

    Fixes #17414.

    Change-Id: I78f92d2c591df7218385fe723a4abc497913acf8
    Reviewed-on: https://go-review.googlesource.com/32116
    Run-TryBot: Russ Cox<EMAIL_ADDRESS>    TryBot-Result: Gobot Gobot<EMAIL_ADDRESS>    Reviewed-by: Rob Pike<EMAIL_ADDRESS>
commit 2693fa15ee12acd67e45d8fa57626675903ab605
Author: Russ Cox<EMAIL_ADDRESS>Date:   Wed Oct 19 13:34:15 2016 -0400

    html/template: add test case for unbounded template expansion

    Fixed by CL 31092 already, but that change is a few steps away
    from the problem observed here, so add an explicit test.

    Fixes #17019.

    Change-Id: If4ece1418e6596b1976961347889ce12c5969637
    Reviewed-on: https://go-review.googlesource.com/31466
    Run-TryBot: Russ Cox<EMAIL_ADDRESS>    TryBot-Result: Gobot Gobot<EMAIL_ADDRESS>    Reviewed-by: Quentin Smith<EMAIL_ADDRESS>
commit 604146ce8961d32f410949015fc8ee31f9052209
Author: Russ Cox<EMAIL_ADDRESS>Date:   Wed Oct 19 10:27:05 2016 -0400

    html/template, text/template: docs and fixes for template redefinition

    All prior versions of Go have allowed redefining empty templates
    to become non-empty. Unfortunately, that has never consistently
    taken effect in html/template after the first execution:

            // define and execute
            t := template.New("root")
            t.Parse(`{{define "T"}}{{end}}<a href="{{template "T"}}">`)
            t.Execute(w, nil) //

            // redefine
            t.Parse(`{{define "T"}}my.url{{end}}`) // succeeds, but ignored
            t.Execute(w, nil) //

    When Go 1.6 added {{block...}} to text/template, that loosened the
    redefinition rules to allow redefinition at any time. The loosening was
    undone a bit in html/template, although inconsistently:

            // define and execute
            t := template.New("root")
            t.Parse(`{{define "T"}}body{{end}}`)
            t.Lookup("T").Execute(ioutil.Discard, nil)

            // attempt to redefine
            t.Parse(`{{define "T"}}body{{end}}`) // rejected in all Go versions
            t.Lookup("T").Parse("body") // OK as of Go 1.6, likely unintentionally

    Like in the empty->non-empty case, whether future execution takes
    notice of a redefinition basically can't be explained without going into
    the details of the template escape analysis.

    Address both the original inconsistencies in whether a redefinition
    would have any effect and the new inconsistencies about whether a
    redefinition is allowed by adopting a new rule: no parsing or modifying
    any templates after the first execution of any template in the same set.
    Template analysis begins at first execution, and once template analysis
    has begun, we simply don't have the right logic to update the analysis
    for incremental modifications (and never have).

    If this new rule breaks existing uses of templates that we decide need
    to be supported, we can try to invalidate all escape analysis for the
    entire set after any modifications. But let's wait on that until we know
    we need to and why.

    Also fix documentation of text/template redefinition policy
    (redefinition is always OK).

    Fixes #15761.

    Change-Id: I7d58d7c08a7d9df2440ee0d651a5b2ecaff3006c
    Reviewed-on: https://go-review.googlesource.com/31464
    Run-TryBot: Russ Cox<EMAIL_ADDRESS>    TryBot-Result: Gobot Gobot<EMAIL_ADDRESS>    Reviewed-by: Andrew Gerrand<EMAIL_ADDRESS>
commit d2315fdc11ebdf5c0ae94f33cb01ffaab82c00b6
Author: Russ Cox<EMAIL_ADDRESS>Date:   Wed Oct 19 13:14:16 2016 -0400

    html/template: adjust ambiguous URL context text

    Before: ... appears in an ambiguous URL context.
    After:  ... appears in an ambiguous context within a URL.

    It's a minor point, but it's confused multiple people.
    Try to make clearer that the ambiguity is "where exactly inside the URL?"

    Fixes #17319.

    Change-Id: Id834868d1275578036c1b00c2bdfcd733d9d2b7b
    Reviewed-on: https://go-review.googlesource.com/31465
    Run-TryBot: Russ Cox<EMAIL_ADDRESS>    Reviewed-by: Rob Pike<EMAIL_ADDRESS>    TryBot-Result: Gobot Gobot<EMAIL_ADDRESS>
commit 2f7f679c79009137bd34fcc33a6d3a6762f45e75
Author: Russ Cox<EMAIL_ADDRESS>Date:   Tue Oct 18 23:22:38 2016 -0400

    html/template, text/template: clarify template redefinition behavior

    Make two important points clearer:

     - Giving a template definition containing
       nothing but spaces has no effect.
     - Giving a template definition containing
       non-spaces can only be done once per template.

    Fixes #16912.
    Fixes #16913.
    Fixes #17360.

    Change-Id: Ie3971b83ab148b7c8bb800fe4a21579566378e3e
    Reviewed-on: https://go-review.googlesource.com/31459
    Run-TryBot: Russ Cox<EMAIL_ADDRESS>    Reviewed-by: Rob Pike<EMAIL_ADDRESS>    Reviewed-by: Andrew Gerrand<EMAIL_ADDRESS>
commit cd2c9df7612795cad5b56cabe5ec29c7771db5fe
Author: Caleb Spare<EMAIL_ADDRESS>Date:   Fri Oct 14 00:59:19 2016 -0700

    html/template: fix Clone so that t.Lookup(t.Name()) yields t

    Template.escape makes the assumption that t.Lookup(t.Name()) is t
    (escapeTemplate looks up the associated template by name and sets
    escapeErr appropriately).

    This assumption did not hold for a Cloned template, because the template
    associated with t.Name() was a second copy of the original.

    Add a test for the assumption that t.Lookup(t.Name()) == t.

    One effect of this broken assumption was #16101: parallel Executes
    racily accessed the template namespace because each Execute call saw
    t.escapeErr == nil and re-escaped the template concurrently with read
    accesses occurring outside the namespace mutex.

    Add a test for this race.

    Related to #12996 and CL 16104.

    Fixes #16101

    Change-Id: I59831d0847abbabb4ef9135f2912c6ce982f9837
    Reviewed-on: https://go-review.googlesource.com/31092
    Run-TryBot: Rob Pike<EMAIL_ADDRESS>    TryBot-Result: Gobot Gobot<EMAIL_ADDRESS>    Reviewed-by: Rob Pike<EMAIL_ADDRESS>
commit ffd1c781b77aab542713b66ef387fa9307e4060b
Author: Nodir Turakulov<EMAIL_ADDRESS>Date:   Sat Sep 5 06:38:13 2015 -0700

    html/template: check "type" attribute in

As of 1.8 the EJS template gets mangled due to escaping:

If this is desired behavior, is there a workaround?

@anthonybishopric, you're commenting on a closed bug. If you want somebody to see it and consider it for Go 1.8, please file a new bug.

@bradfitz Thanks, sorry I didn't see the closed indicator. Will open a new issue


round tripping multiline header results in single line header

Subject of the issue
text
text-
-
text

is stringified as:
## text\
text-

text

which changes the structure
Your environment

OS: Ubuntu
Packages: mdast-util-to-markdown 0.6.4
Env: node v15.5.1, npm 7.3.0

Steps to reproduce
parse
text
text-
-
text

which has the structure
{
    "type": "root",
    "children": [
        {
            "type": "heading",
            "depth": 2,
            "children": [
                {
                    "type": "text"
                },
                {
                    "type": "break"
                },
                {
                    "type": "text"
                }
            ]
        },
        {
            "type": "paragraph",
            "children": [
                {
                    "type": "text"
                }
            ]
        }
    ]
}

and stringify it:
## text\
text-

text

the resulting markdown has a different structure than the original
{
    "type": "root",
    "children": [
        {
            "type": "heading",
            "depth": 2,
            "children": [
                {
                    "type": "text"
                }
            ]
        },
        {
            "type": "paragraph",
            "children": [
                {
                    "type": "text"
                }
            ]
        },
        {
            "type": "paragraph",
            "children": [
                {
                    "type": "text"
                }
            ]
        }
    ]
}

:notebook: comparing how the two pieces of markdown text are being parsed with https://spec.commonmark.org/dingus it appears in both cases it is parsed as expected.
Expected behavior
structure is the same
Actual behavior
structure is different

This is another case where I think the AST should change: the style of headings is ATX by default: breaks can’t occur in ATX headings, going 'a  \nb\n=' (or 'a\\\nb\n=') to # a b makes the most sense to me


Can't change sidebar position from right to left

Like title says. In options, I've changed sidebar position from right to left but nothing happened. Reopening browser didn't help either.

Can you post a screenshot of this page.
Which Chrome version are you on?


Add fizzbuzz route

Everyone's favorite CS challenge needs its own route, imho, for issue #6

Awesome :) can you resolve conflicts and update! I think everyone is trying to add routes today!

@jkup fixed! Thanks man


Hypertext

https://www.nngroup.com/articles/hypertext-history/#:~:text=The actual word "hypertext" was,has been developing ever since.&text=Nelson views hypertext as a,has to be online together.


Add announcement area/banner using a flatpage

We're going to need to communicate with users periodically during 2022 about the state of the application, and we've received a request to add an announcement banner, something similar to what is currently in Remote Office Hours Queue.

Communicate to SE campus users:
Canvas data UDW will be stopped supporting toward the end of the year.. UDW will be there but no update to it. Unizin advises UDW users to switch to UDP.  Unizin is planning to roll out canvas data in fall.
Create a banner on the SE app home page: SE will not be receiving new data (refreshed data) beyond December 2023.


fix: CDataTable: v-else compiler warning #143

What kind of change does this PR introduce? (check at least one)

[x] Bugfix
[ ] Feature
[ ] Code style update
[ ] Refactor
[ ] Build-related changes
[ ] Other, please describe:

Does this PR introduce a breaking change? (check one)

[ ] Yes
[x] No

If yes, please describe the impact and migration path for existing applications:
The PR fulfills these requirements:

[x] It's submitted to the develop branch (or to a previous version branch), not the master branch
[x] When resolving a specific issue, it's referenced in the PR's title (e.g. fix #xxx[,#xxx], where "xxx" is the issue number)
[x] All tests are passing
[x] New/updated tests are included

If adding a new feature, the PR's description includes:

[ ] A convincing reason for adding this feature (to avoid wasting your time, it's best to open a suggestion issue first and wait for approval before working on it)

Other information:

thanks @adamshaylor - closed with https://github.com/coreui/coreui-vue/commit/652cac8d21acb2425766497d36fac50f04612986


Develop

Deploy preview for oscartegiffel processing.
Built with commit 72768de37d046be9d1a67b64234e406d3c6eb52f
https://app.netlify.com/sites/oscartegiffel/deploys/5b71a160fdd72a20792449d1


Small fixes to swagger in Microsoft.Sql AdvancedThreatProtectionSetti…

Small fixes to swagger in Microsoft.Sql AdvancedThreatProtectionSettings API for 2021-11-01-preview
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[ ] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[*] Other, please clarify - bug fixes to already committed API

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? This should be published with Microsoft.Sql 2021-11-01-preview
If updating an existing version, please select the specific language SDKs and CLIs that must be refreshed after the swagger is published.

[ ] SDK of .NET (need service team to ensure code readiness)
[ ] SDK of Python
[ ] SDK of Java
[ ] SDK of Js
[ ] SDK of Go
[*] PowerShell
[*] CLI
[ ] Terraform
[ ] No refresh required for updates in this PR

Contribution checklist:

[*] I commit to follow the Breaking Change Policy of "no breaking changes"
[*] I have reviewed the documentation for the workflow.
[*] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[*] Check this box if any of the following apply to the PR so that label "WaitForARMFeedback" will be added automatically to begin ARM API Review. Failure to comply may result in delays to the manifest.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you are using OpenAPIHub to initialize the PR for adding a new version. More details, refer to the wiki.

[*] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If any of the following scenarios apply to the PR, request approval from the Breaking Change Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in a stable version
[ ] Removing properties in a stable version
[ ] Removing API version(s) in a stable version
[ ] Updating API in a stable or public preview version with Breaking Change Validation errors
[ ] Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
Please follow the link to find more details on PR review process.

/azp run


Duplicates deletion error

Bug description
I found the issue with duplicates deletion. When I enabled this feature I started to face such errors sometimes during report uploading:
{"message":"Internal server error, check logs for details"}
In django logs I found the error:
ERROR [dojo.api_v2.exception_handler:32] insert or update on table "dojo_test_import_finding_action" violates foreign key constraint "dojo_test_import_fin_finding_id_28fe8e2d_fk_dojo_find"
DETAIL: Key (finding_id)=(742929) is not present in table "dojo_finding".
django.db.utils.IntegrityError: insert or update on table "dojo_test_import_finding_action" violates foreign key constraint "dojo_test_import_fin_finding_id_28fe8e2d_fk_dojo_find"

As I understand DD tried to find a finding that was deleted as a duplicate.
@valentijnscholten commented on Slack:
"The job to delete duplicates runs every minute, so if you have an import that runs longer than that, there may be a race condition that newly imported findings which are duplicates of existing findings are deleted before the import is complete. I can't remember exactly what the transactional model is in django, but probably  not everything runs in a transaction so duplicates become visible to the job that deletes duplicates before the import is completed."
Steps to reproduce
Steps to reproduce the behavior:

Create a project.
Enable Deduplicate findings and Delete duplicates features in settings.
Import a report with a lot of findings (more than 100).
Import a few reports with duplicates to the project.
After some imports you will face an error in logs:
{"message":"Internal server error, check logs for details"}

Expected behavior
The report should be imported successfully.
Deployment method (select with an X)

[ ] Docker Compose
[X] Kubernetes
[ ] GoDojo

Environment information

DefectDojo version 2.9.1

Logs
ERROR [dojo.api_v2.exception_handler:32] insert or update on table "dojo_test_import_finding_action" violates foreign key constraint "dojo_test_import_fin_finding_id_28fe8e2d_fk_dojo_find"
DETAIL: Key (finding_id)=(742929) is not present in table "dojo_finding".
django.db.utils.IntegrityError: insert or update on table "dojo_test_import_finding_action" violates foreign key constraint "dojo_test_import_fin_finding_id_28fe8e2d_fk_dojo_find"

we also have the same issue. any plan to fix it soon :-(

@Maffooch

There is a possible solution in the linked Draft Pull Request, it's currently having trouble with the JIRA integration's unit tests. This is holding it back from leaving its draft state. Feel free to take a look - maybe you have a idea for that situation.

Is there any update on this or plan when this will be fixed?
Having the same issue with newest version :/

Also affects uploading large trivy.json results via pipeline if import takes over 1 minute.

A hot fix of changing the following settings allowed me to upload the 5mb trivy scan import.
DD_ASYNC_FINDING_IMPORT=True
DD_ASYNC_FINDING_IMPORT_CHUNK_SIZE=100

I faced with the same problem. Main reason is race condition. It's already been discussed and @fhoeborn offer pull request to fix it. But it still not implemented.
https://github.com/DefectDojo/django-DefectDojo/pull/7444


Add GroupsAreSortedRule

Adds a rule to ensure group children are sorted alphabetically (WIP)

~Add support to sort by type instead of name?~ later
~Opt in to rule somehow~ config issue
[x] sorted() is different than Xcode "Sort by Name"

Xcode: [10, 2, 1] -> [1, 2, 10]
Sorted: [10, 2, 1] -> [1, 10, 2]

[ ] sorted(using: .localizedStandard) isn't available on linux 🤔

@mattmassicotte feel free to nitpick/replace/overwrite/etc, mostly just was curious how hard it would be to implement (very easy it turns out!)
#7

🤯 holy macaroni this is awesome
long-term, it is probably going to be difficult to manage a single Xcode project for test input, but I'm very happy to burn that bridge when we get to it.
I think it is completely find to just merge this, if you are ok with it. The opt-in is really part of the whole configuration story. And, the sort-by-type idea is cool, but we can tackle that in the future, if needed.

Wait never mind! I saw that the stock macOS one was modified and thought that's what you were using for testing, but isn't. Let's do this!

Sounds good, just want to take a look at that last bullet point I added before merging

Huh, interesting! sorted(using: .localizedStandard) isn't available in the Linux version of Foundation. Well this stinks! I assume that kind of comparison is essential for this to work correctly?

Just a little typo missing the use of sortedChildren. Easy fix!

The comparison sort of needs to be the same on linux, otherwise CI runs will give different errors.  I switched to specifying sort options, but it's unclear to me what is available on linux and what isn't.

but it's unclear to me what is available on linux and what isn't

I have frequently had this problem too. I tried to help mitigate this by explicitly installing Swift 5.9 on the linux CI instance before running. I think the best thing to do is have a VM set up, but who has time for that. A few back-and-forths with CI is no big deal.

Hi all! It would be super awesome to have additional configuration on this:

Folders on top.
Exclude extension.

Just wondering if you've considered that and already given any through to this. If there are any caveats? Happy to work on a separate PR for this, just want to gather some feedback and intel.

@iby Hello! I'd like to better understand what you are after. How are you controlling these two things? I can see manual sorting for folders, but how does exclusing an extension work?


fail to build contract under near-sdk 5.0.0 and cargo near 0.6.1

The build instructions are as follows：

this is cargo.toml

These are the problems I have when compiling

I am confused as to why near_abi and schemars cannot be found during compilation. I have even set their versions in cargo.toml.

@WuDi329 Just remove the explicit abi feature enablement from Cargo.toml:
near-sdk = "5.0.0"

I will investigate what exactly breaks there when you enable it explicitly, but cargo-near will enable it automatically.

Just remove the explicit abi feature enablement from Cargo.toml:
near-sdk = "5.0.0"

Thanks for your reply, I've modified cargo.toml as you said, but it doesn't seem to have changed the build result.

https://github.com/near/near-sdk-rs/blob/9bb16c2d08d522baf52ddd126e3fe0f01386f2ca/near-sdk/Cargo.toml#L40-L42
Just for the context, I remembered why it fails to compile, we don't even include schemars / near-abi for Wasm target to avoid unexpected dependencies. ABI is generated and extracted ahead of time as a separate step that compiles the code to the host platform first.
@WuDi329 I cannot reproduce your compilation error. Also, remove near-abi and schemars, prefer using near_sdk::borsh over direct dependency on borsh, and MOST IMPORTANTLY, remove rand and getrandom from your dependencies (you can use them in dev-dependencies), they won't work in Wasm environment; use near-rng if you need random numbers inside the contract.

Also, remove near-abi and schemars, prefer using near_sdk::borsh over direct dependency on borsh, and MOST IMPORTANTLY, remove rand and getrandom from your dependencies (you can use them only in dev-dependencies, for tests), they won't work in Wasm environment; use near-rng if you need random numbers inside the contract.

Thank you, @frol ! I have seen your suggestion and removed near-abi,schemars etc. from cargo.toml, I will also use near_sdk::borsh instead of borsh, use near-rng instead of rand and getrandom.

https://github.com/near/near-sdk-rs/blob/9bb16c2d08d522baf52ddd126e3fe0f01386f2ca/near-sdk/Cargo.toml#L40-L42
Just for the context, I remembered why it fails to compile, we don't even include schemars / near-abi for Wasm target to avoid unexpected dependencies. ABI is generated and extracted ahead of time as a separate step that compiles the code to the host platform first.

I have seen the cargo.toml in the near-sdk, so does this mean that when my compilation target is wasm32, it will fail due to missing near_abi and schemars? I don't know if my understanding is correct, because I am a newbie in rust. If you have any questions, please point them out. Now do you have any suggestions to solve this problem?

Now do you have any suggestions to solve this problem?

I cannot reproduce your error, so I cannot help you unless you provide a minimal reproducible example. Try creating a new project (cargo near new) and copy Cargo.toml from there.

@frol I'm sorry that I was busy with other work some time ago, and now I hope to get your help in solving this problem.

Try creating a new project (cargo near new) and copy Cargo.toml from there.

I've created a minimal repository like this. https://github.com/WuDi329/minimal-near, I wrote what I tried in the readme. This minimal comes from the default project created by cargo near new. I added a function in contract, get_round_status, which will return a type I defined in demo-dependency, nothing more.
But when I try to run cargo near abi, it reports the following error：

In fact, I have added abi to the cargo.toml of dependency-demo.

When I run cargo build --target wasm32-unknown-unknown directly, I get an error like this

I guess you might ask why I put that variable in an additional dependent library, it's because the system I want to implement includes both on-chain and off-chain parts, and they all rely on the same data structure definition, but Their functions do not go together.
Do you have any suggestions? If you have any questions, you are welcome to tell me.

@WuDi329 here is the fix: https://github.com/WuDi329/minimal-near/pull/1


Flask app failure happens days after deployment

I have an Azure Linux App Service running with code deployment and Always On: False. The app service works as expected right after deployment, and failed with 503 errors after a certain amount of time. When a 503 error is observed, the stream log from the app service shows the following error message.
2018-11-13 15:24:58.731 INFO  - Starting container for site
2018-11-13 15:24:58.733 INFO  - docker run -d -p 43438:8000 --name usagedriverdevlinux_0 -e WEBSITE_SITE_NAME=usagedriverdevlinux -e WEBSITE_AUTH_ENABLED=False -e WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_INSTANCE_ID=ecd7b7595c2085f900b2a30ce059050e5bc2dcf3927d1ef3f9745e96bd1fef71 -e HTTP_LOGGING_ENABLED=1 appsvc/python:3.6.6_1809042203
2018-11-13T15:24:59.092415149Z
2018-11-13T15:24:59.092541749Z   _____
2018-11-13T15:24:59.092546749Z   /  _  \ __________ _________   ____
2018-11-13T15:24:59.092550249Z  /  /_\  \___   /  |  \_  __ \_/ __ \
2018-11-13T15:24:59.092553849Z /    |    \/    /|  |  /|  | \/\  ___/
2018-11-13T15:24:59.092557349Z \____|__  /_____ \____/ |__|    \___  >
2018-11-13T15:24:59.092560949Z         \/      \/                  \/
2018-11-13T15:24:59.092564349Z
2018-11-13T15:24:59.092575449Z A P P   S E R V I C E   O N   L I N U X
2018-11-13T15:24:59.092579649Z
2018-11-13T15:24:59.092582849Z Documentation: http://aka.ms/webapp-linux
2018-11-13T15:24:59.092586049Z NodeJS quickstart: https://aka.ms/node-qs
2018-11-13T15:24:59.092589349Z
2018-11-13T15:24:59.149267270Z Starting OpenBSD Secure Shell server: sshd.
2018-11-13T15:24:59.156074972Z Running python /usr/local/bin/entrypoint.py
2018-11-13T15:24:59.214145894Z executing:
2018-11-13T15:24:59.214256794Z python --version
2018-11-13T15:24:59.218902596Z Python 3.6.6
2018-11-13T15:24:59.219048096Z executing:
2018-11-13T15:24:59.219130396Z pip --version
2018-11-13T15:24:59.666356263Z pip 18.0 from /usr/local/lib/python3.6/site-packages/pip (python 3.6)
2018-11-13T15:24:59.679948868Z found flask app
2018-11-13T15:24:59.680570869Z executing:
2018-11-13T15:24:59.680652269Z . antenv3.6/bin/activate
2018-11-13T15:24:59.682776670Z /bin/sh: 1: .: Can't open antenv3.6/bin/activate
2018-11-13T15:24:59.683021870Z
2018-11-13T15:24:59.683119770Z executing:
2018-11-13T15:24:59.683187170Z GUNICORN_CMD_ARGS="--bind=<IP_ADDRESS> --timeout 600" gunicorn application:app
2018-11-13T15:24:59.900779151Z [2018-11-13 15:24:59 +0000] [33] [INFO] Starting gunicorn 19.9.0
2018-11-13T15:24:59.901405651Z [2018-11-13 15:24:59 +0000] [33] [INFO] Listening at: http://<IP_ADDRESS>:8000 (33)
2018-11-13T15:24:59.901604851Z [2018-11-13 15:24:59 +0000] [33] [INFO] Using worker: sync
2018-11-13T15:24:59.905127153Z [2018-11-13 15:24:59 +0000] [36] [INFO] Booting worker with pid: 36
2018-11-13T15:25:00.082357719Z [2018-11-13 15:25:00 +0000] [36] [ERROR] Exception in worker process
2018-11-13T15:25:00.082389219Z Traceback (most recent call last):
2018-11-13T15:25:00.082394819Z   File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
2018-11-13T15:25:00.082400719Z     worker.init_process()
2018-11-13T15:25:00.082404719Z   File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
2018-11-13T15:25:00.082408919Z     self.load_wsgi()
2018-11-13T15:25:00.082412819Z   File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
2018-11-13T15:25:00.082424919Z     self.wsgi = self.app.wsgi()
2018-11-13T15:25:00.082429119Z   File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
2018-11-13T15:25:00.082433019Z     self.callable = self.load()
2018-11-13T15:25:00.082436819Z   File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
2018-11-13T15:25:00.082440719Z     return self.load_wsgiapp()
2018-11-13T15:25:00.082444419Z   File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
2018-11-13T15:25:00.082448319Z     return util.import_app(self.app_uri)
2018-11-13T15:25:00.082452119Z   File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
2018-11-13T15:25:00.082456019Z     __import__(module)
2018-11-13T15:25:00.082459719Z   File "/home/site/wwwroot/application.py", line 6, in <module>
2018-11-13T15:25:00.082464019Z     from flask_httpauth import HTTPBasicAuth
2018-11-13T15:25:00.082467919Z ModuleNotFoundError: No module named 'flask_httpauth'
2018-11-13T15:25:00.082950219Z [2018-11-13 15:25:00 +0000] [36] [INFO] Worker exiting (pid: 36)
2018-11-13T15:25:00.124758235Z [2018-11-13 15:25:00 +0000] [33] [INFO] Shutting down: Master
2018-11-13T15:25:00.125010935Z [2018-11-13 15:25:00 +0000] [33] [INFO] Reason: Worker failed to boot.
2018-11-13T15:25:00.157341247Z
2018-11-13 15:25:01.155 ERROR - Container usagedriverdevlinux_0 for site usagedriverdevlinux has exited, failing site startStatus_WatchFile :: Error Error: ENOENT: no such file or directory, open '/appsvctmp/status.txt'

The problem is solved by switching Always On: True. I suspect that the right virtual environment works right after the deployment. When Always On is turned off, the container is destroyed after a certain amount of time. Then when a new request coming, a container is being created but fails in creating the virtual environment. Then the default Python, who did not install dependencies from requirements.txt,  is called. Then ModuleNotFoundError is shown.
By setting Always On to True, the container is always running and the virtual environment created in the container works as well. So that problem is temporary solved.
For more detailed information, please check support ticket [REG:118111321002576]. I really appreciate the help from MSFT support engineers Ramya Gangula and Prashanth Madi in figuring out the possible causes.

Any update on this one?

@KevinXTian
I'll have to reproduce this behavior and see for myself why this  failure happens. Are you deploying a simple flask app? Also, was the initial deployment done with git or FTP?

Yes, it is a simple flask app. And the initial deployment is done with git.

From: JennyLawrance<EMAIL_ADDRESS>Sent: Friday, December 7, 2018 8:57 PM
To: Azure-App-Service/python
Cc: Tian, Xueli (Kevin); Mention
Subject: Re: [Azure-App-Service/python] Flask app failure happens days after deployment (#13)
@KevinXTianhttps://github.com/KevinXTian
I'll have to reproduce this behavior and see for myself why this failure happens. Are you deploying a simple flask app? Also, was the initial deployment done with git or FTP?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/Azure-App-Service/python/issues/13#issuecomment-445420458, or mute the threadhttps://github.com/notifications/unsubscribe-auth/Afr9NAg9JxI85SD8jWHKwnPDkhHqzBtAks5u2xx1gaJpZM4Ym231.

I have a theory for this failure: Essentially, we have an issue / disconnect between the UX and the runtime, where UX allowed python 3.6 to be selected while the runtime was not yet upgraded. This meant that, for this site, the runtime 3.6 version was only available after 11/14 on that scale unit, and I believe you also did a Virtual Environmet create with SSH portal to get the site back to an operational state.
When the site was deployed, the Virtual Environment created was for 3.7/ But when 3.6 runtime was loaded after the upgrade, the site failed to start, since it looks for a different virtual environment. I'll continue to investigate this next week, but I'm pretty certain that alwaysOn in this case is a red-herring, and is not what has resolved the issue.


[WIP] - Reloading of server certificates for server endpoints

This PR resolves the reloading of certs for HTTP and gRPC servers
Resolves #4554

This PR currently contains only HTTP server cert reload. Working on gRPC server cert reload.
cc. @yurishkuro


Log elapsed durations with ms units label

Hoist P/R Checklist
Pull request authors: Review and check off the below. Items that do not apply can also be
checked off to indicate they have been considered. If unclear if a step is relevant, please leave
unchecked and note in comments.

[x] Caught up with develop branch as of last change.
[x] Added CHANGELOG entry, or determined not required.
[x] Reviewed for breaking changes, added breaking-change label + CHANGELOG if so.
[x] Updated doc comments / prop-types, or determined not required.
[x] Reviewed and tested on Mobile, or determined not required.
[x] Created Toolbox branch / PR, or determined not required.

If your change is still a WIP, please use the "Create draft pull request" option in the split
button below to indicate it is not ready yet for a final review.

Pull request reviewers: when merging this P/R, please consider using a squash commit to
collapse multiple intermediate commits into a single commit representing the overall feature
change. This helps keep the commit log clean and easy to scan across releases. PRs containing a
single commit should be rebased when possible.

Did not think this warranted a changelog entry, but I found it helpful to remind me / underscore what the numbers after our track/debug logs represent.
Also note minor re-order on special LoadSupport log to align with track logging.

I think this is a great change, and well worth the extra 2 characters.  We should make the same change in hoist-core for its logging -- I'll add at ticket


fix kafka bootstrap servers for native dev

Commit https://github.com/HumanSignal/infra/commit/cef400f7d475c39463a416a27646f5ccfd1b5bab is created.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 57.75%. Comparing base (bc69ce6) to head (4d62adf).
Report is 10 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #159   +/-   ##
=======================================
  Coverage   57.75%   57.75%
=======================================
  Files          40       40
  Lines        1754     1754
=======================================
  Hits         1013     1013
  Misses        741      741

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Commit https://github.com/HumanSignal/infra/commit/55c139cbea7aaab760a18e26020f0471b69e4a4d is created.


mission and vision page added

Description
This PR is under GSSOC'21. For routing my component, I have installed react-router-dom and have made use of semantic-ui.
Closes #56
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] Any dependent changes have been merged and published in downstream modules

Screenshot:
https://user-images.githubusercontent.com/67699840/112588557-9afae480-8e25-11eb-984d-bfb112178063.mov

@vedikaag99 can you please upload a screenshot?

@codewithvk hi, i'm closing this PR because I am having the hardest time updating my branch patch-1 wrt master. So I am making a new branch and copy pasting my code there as you suggested and hence opening up a new PR. Sorry for the inconvenience caused.


Fix bug leading to openai failure call

The open ai key was not given into the actual API call

Thanks!


(feature request): add support to ignore generated classes

right now the plugin is flagging generated classes (for example from moshi) as unstable. maybe we can ignore that

Thanks, @karandeepsingh-cred for raising the query. We'll definitely take a look into this

@karandeepsingh-cred Instead of ignoring a specific class, we have added a parameter to exclude the generation of classes in the report as of now in v1.1.0. Till the time actual support for specific class comes, maybe give it a try!


Please support catch TypeError if occurred

this is an assumption about a case:
app.get("/", wrap((req, res) => {
	  let str = [];
	  res.status(200).json({ data: str[0].data });
          // it will crash app with message: TypeError: Cannot read property 'data' of undefined.
}));

It should be including exception and response to client as "Something went wrong" instead of error message default of TypeError.
Need update:
wrap.js:
const defaultOptions = {
  nextOnce: true,
  defaultJsonResponse: false,
  errorUnknownMessage: 'Something went wrong!'  // add new option for default message error unknown
}

middleware.js:
const AppError = require('./custom-exception')

function exceptionMiddleware (err, _, res, __) {
    if (err.originError instanceof Error) {
        return res.status(err.status).send(err.errorUnknownMessage)
    }
    res.status(err.status).send(err.response || err.message);
}

module.exports = exceptionMiddleware

Usage:
wrap(fn, { errorUnknownMessage: 'Oops! An error occurred on this server' })
Please update this soon.
Thankyou @kanekotic !

thanks @phatdotcom sounds like a very good idea. Do you mind making a PR as it seems you have the idea pin down?.


add client for making tsa requests

Summary
First step toward support for requesting timestamps from the Timestamp Authority.
This just adds a client for interacting w/ the TSA API. Plenty more work to do here to fully enable this feature.

Closing until we work through some issues w/ the TSA API interface


Change app name @ homescreen

Manifest

Fixed. App label and title will display Tasbeeh

Fixed. App label and title will display Tasbeeh


Fix protobuf version

See https://github.com/Lightning-AI/lightning-transformers/pull/284#issuecomment-1243050284
This is based upon @rohitgr7 's awesome https://github.com/Lightning-AI/lightning-transformers/pull/284
I have done a merge request into his branch. I am not trying to take credit for his awesome work. I submit this PR because I'd like to see CI run with this fix (but I can't as a first time contributor to this repo), so I'd like CI/CD permissions.

@rohitgr7 @carmocca @Borda Any idea why my CI is failing? Is it the skipped tests or the warnings? I didn't grep for any errors.
Even if @rohitgr7 's https://github.com/Lightning-AI/lightning-transformers/pull/284 gets merged, I still would like to understand how to placate CI in case I do future PRs.

Looks like the tests pass, but there's an error either in the coverage computation or on pytest teardown.
I would start by removing coverage and see if the result is different: https://github.com/Lightning-AI/lightning-transformers/actions/runs/3033498716/workflow#L76-L77

Looks like the tests pass, but there's an error either in the coverage computation or on pytest teardown.
I would start by removing coverage and see if the result is different: https://github.com/Lightning-AI/lightning-transformers/actions/runs/3033498716/workflow#L76-L77

Thanks for the pointer. But where in the CI logs are you looking specifically, besides the final non-descript 'Error'?

besides the final non-descript 'Error'?

Just here, there's nothing else. This is why I suspect the command does not finish properly. As I pointed out it's coverage calling pytest. So even if pytest passes, coverage could fail (my hypothesis)
To check it, do
-coverage run --source lightning_transformers -m py.test lightning_transformers tests -v --junitxml=junit/test-results-${{ runner.os }}-${{ matrix.python-version }}-${{ matrix.requires }}.xml
+python -m py.test lightning_transformers tests -v
-coverage XML

@turian the needed proval is GH policy notning that we came up :)

I guess this is not needed any more, correct? :rabbit:

@turian, I guess this is not needed anymore after #284 was merged, correct? 🐰

Correct


[FLINK-23832][docs] Update DataStream API Integration page

What is the purpose of the change
Updates the DataStream API Integration page for all feedback and features for 1.14.
Brief change log

Update explanations for configuration
add batch mode
help for RAW type in case of Row

Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? docs

Thanks everyone. I addressed the last remaining feedback and synced this with Chinese version. I will merge this once the build turns green.


Recorded talk

Hello,
Is it possible to access to the recorded talk? I'm really interested.
Thank you!
Best,
Gustavo

Hi @GEJ1,
As stated on the original Pie&AI registration link with Eventbrite, this was an on-site only event without recording, sorry.
If there some more demand, we might try some recordings for the next events.
However, you can access the presented material and demos online freely:

Slides: https://pieandai-grenoble.github.io/2024-02-23-ml-models-web/
Samples:

https://pieandai-grenoble.github.io/2024-02-23-ml-models-web/samples/housing/index.html
https://pieandai-grenoble.github.io/2024-02-23-ml-models-web/samples/sentiment/index.html
https://pieandai-grenoble.github.io/2024-02-23-ml-models-web/samples/imaging/index.html
https://pieandai-grenoble.github.io/2024-02-23-ml-models-web/samples/imaging/live.html


Make ResolveMetaclass into a closure type

Fixes #116

Abandoning because this use of closures is not idiomatic in Swift. A protocol would be better.


[QUESTION] after change utils\make to support\make，the ide hints of functions is lost

Has anybody meets the same problem，how to solve this？Give me some help，thanks~

.phpstorm.meta.php not updated，it is my error


export more than 1M record.

is posibble to export more than 1M record to one .xls or .xlsx file?

It isn't in a single worksheet.
BIFF-format (.xls) files have a limit of 65,536 rows per worksheet; OfficeOpenXML (.xlsx) files have a limit of 1,048,576 rows per worksheet

We are closing all old tickets, because version 2.* is reaching the End-of-Life stage. Our full attention is now allocated to version 3.0. It's still possible to use 2.*, but there will be no active support anymore.


Add a changelog

We should add a changelog and maybe add instructions for writing changelog entries in the README.
cc @danswick

Changelog is going over here: https://github.com/mapbox/dr-ui/blob/master/CHANGELOG.md


Crash de-referencing null pointer in call to notifyError

Description

Issue
The library occasionally crashes de-referencing a null pointer. Here's the trace we have (only the part inside of Bugsnag):
EXC_BAD_ACCESS: Attempted to dereference null pointer.

0  libsystem_platform.dylib __platform_strcmp
1  PlanGrid                 bsg_kscrashstate_i_onIntegerElement (BSG_KSCrashState.c:105:9)
2  PlanGrid                 bsg_ksjsondecode (BSG_KSJSONCodec.c:1016:18)
3  PlanGrid                 bsg_kscrashstate_i_loadState (BSG_KSCrashState.c:202:9)
4  PlanGrid                 bsg_kscrashstate_init (BSG_KSCrashState.c:292:5)
5  PlanGrid                 bsg_kscrash_reinstall (BSG_KSCrashC.c:198:10)
6  PlanGrid                 -[BSG_KSCrash reportUserException:reason:originalException:handledState:appState:callbackOverrides:metadata:config:discardDepth:terminateProgram:] (BSG_KSCrash.m:385:5)
7  PlanGrid                 -[BugsnagCrashSentry reportUserException:reason:originalException:handledState:appState:callbackOverrides:metadata:config:discardDepth:] (BugsnagCrashSentry.m:54:5)
8  PlanGrid                 -[BugsnagNotifier notify:handledState:block:] (BugsnagNotifier.m:634:5)
9  PlanGrid                 -[BugsnagNotifier notifyError:block:] (BugsnagNotifier.m:516:5)
10 PlanGrid                 +[Bugsnag notifyError:block:] (Bugsnag.m:119:9)

Environment
Library versions:

Bugsnag version (from your Podfile, Podfile.lock, or elsewhere): 5.22.2 (going back to at least 5.19.1)
CocoaPods version (if any) (pod -v): 1.7.2
Carthage version (if any):
iOS/tvOS/macOS version(s): iOS 11+
debug mode or production?: Production
How are you initializing Bugsnag? Can you share a snippet of your
AppDelegate where Bugsnag is configured?

Example code snippet
import Bugsnag
// title is a String
// message is a String
// severity is an enum, which we map to a Bugsnag severity
        let bugsnagError = NSError(domain: "com.loupe.PlanGrid", code: 0, userInfo: [
            NSLocalizedDescriptionKey: title,
            NSLocalizedFailureReasonErrorKey: message
        ])

        Bugsnag.notifyError(bugsnagError) { report in
            report.errorClass = title
            report.errorMessage = message
            report.severity = severity.bugsnagSeverity
            additionalData?.forEach { key, value in
                report.metaData[key] = value
            }
            report.context = "ErrorLogger.logError"
        }

Here's the ID of the error in the plangrid iOS Bugsnag project :) 5c816b620bf2620019385141. Feel free to go have a look!

Thanks for the report @benasher44! This issue should be fixed as of 5.22.3 (in PR https://github.com/bugsnag/bugsnag-cocoa/pull/382). Feel free to add an update if it comes up again though and we’ll look into this.

Ah great thanks! We'll be on the lookout in our release with 5.22.3 in it then :)

Confirmed fixed!


Add a variable casing fixer

Rule request
A fixer for consistent variable casing. This should be configurable with snake_case and camel_case.

Change variable casing in code;
Change variable casing in comments/ docblocks;
Is considered risky;

Code snippet that shows how the rule would fix PHP code
Turning
$fooBar = 'baz';

into
$foo_bar = 'baz';

with config snake_case and vice versa for camel_case

Hi and thanks for sharing your idea,
I think a proposal for this has been done here https://github.com/FriendsOfPHP/PHP-CS-Fixer/pull/5097

@SpacePossum I think it is yes.

Then, let me close this one as duplicate ;)


enable admins to configure the slack help text

The help text displayed when a user enters /discourse into slack is hardcoded as:
`/discourse [watch|follow|mute|help|status] [category|all]`
*watch* – notify this channel for new topics and new replies
*follow* – notify this channel for new topics
*mute* – stop notifying this channel
*status* – show current notification state and categories

in plugin.rb self.help
Discourse admins should be able to modify this default help text.
We don't refer to our discourse instance as "discourse" and so have renamed the slash command in slack to the name more familiar with our users. It'd be nice to reflect this in the help text (which has hardcoded the command to /discourse).

This request does not make sense; the tool is a Slack to Discourse
connector. The instance name does not change the fact that it is Discourse
underneath..
On Wed, Mar 15, 2017 at 4:13 AM stu-b-doo<EMAIL_ADDRESS>wrote:

The help text displayed when a user enters /discourse into slack is
hardcoded as:
/discourse [watch|follow|mute|help|status] [category|all]
watch – notify this channel for new topics and new replies
follow – notify this channel for new topics
mute – stop notifying this channel
status – show current notification state and categories
in plugin.rb self.help
https://github.com/discourse/discourse-slack-official/blob/0c4e8611c19edc3e3247dfa31a927635fa775ecc/plugin.rb#L251
Discourse admins should be able to modify this default help text.
We don't refer to our discourse instance as "discourse" and so have
renamed the slash command in slack to the name more familiar with our
users. It'd be nice to reflect this in the help text (which has hardcoded
the command to /discourse).
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/discourse/discourse-slack-official/issues/21, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABc7Vbj4YdwGz5YyXY-j7nAdM98lwaUyks5rl8fpgaJpZM4MdzVJ
.

I would welcome this change.  On the slack side, when you set up the integration you can choose what the keyword is for the Slack command, and we have also chosen to match our site name, rather than discourse.
So to actually follow a category, you type something like the following:
/mycoolforum watch foo-category

and if you want help, you type:
/mycoolforum help

but then it responds with help saying:
/discourse [watch|follow|mute|help|status] [category|all]

which, if you type it, will do nothing....
So yeah, being able to make the help text match how you configure it in Slack makes total sense to me.

I suppose it is more defensible if there are multiple sites, you would need
to disambiguate among them. That makes sense, so I withdraw my objection.
On Wed, Mar 15, 2017 at 10:12 PM David McClure<EMAIL_ADDRESS>wrote:

I would welcome this change. On the slack side, when you set up the
integration you can choose what the keyword is for the Slack command, and
we have also chosen to match our site name, rather than discourse.
So to actually follow a category, you type something like the following:
/mycoolforum watch foo-category
and if you want help, you type:
/mycoolforum help
but then it responds with help saying:
/discourse [watch|follow|mute|help|status] [category|all]
which, if you type it, will do nothing....
So yeah, being able to make the help text match how you configure it in
Slack makes total sense to me.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/discourse/discourse-slack-official/issues/21#issuecomment-286959826,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABc7VW1RM8aETgQiFIbVJhlzV8h2R0W-ks5rmMSmgaJpZM4MdzVJ
.

This is already available in my PR which will be merged soon.

@coding-horror even if you only have one discourse site, as @mcwumbly  mentioned, our users are used to referring to discourse as mycoolforum . So our slack command reflects that: /mycoolforum.
But in the command's help it's assumed the slack command is called /discourse, adding minor confusion.
To be honest, the most of the users who know about the slack command are probably well aware theat mycoolforum is an instance of this other thing called discourse, so it's not that big a deal. More for completeness really 😃

@vinothkannans in your PR, the message still looks hardcoded in the yml. Is this file going to be modifyable by admins or would it need to be changed at source before the app is installed?

@stu-b-doo you can modify all the strings which are inside .yml file from the admin panel of your Discourse installation in the page /admin/customize/site_texts

so config/locales/server.en.yml would become available as a site text?  Sounds good!

Not just server.en.yml.
All .yml files inside /config/locales/ folder.


how to submit the sequence of the target protein to the model

Thanks a lot for your work.
May I ask a naive question on how to submit the sequence of the target protein to the model? Kind for your reply

Same here. :p I have no clue what this repo even attempts to do


⚠ Bofrost.at has degraded performance

In de35c11, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1351 ms

Resolved: Bofrost.at performance has improved in 26f4783.


Auto-submit CTF flag codes to CTFd

[ ] Add a config or command line parameter to pass in the URL to a CTFd instance. Must be undefined by default
[ ] If above URL is set and CTF flags are enabled upon solving a challenge its solution flag is submitted to the CTFd API
[ ] Should happen together with the websocket notification to benefit from existing replay feature for solution notifications (e.g. if the CTFd was not reachable at the right moment)

Alternative to config and automatic submit, a submit button could be added to the flag notification itself.

Can you share how CTFd API looks like? I've look for it, but could not find any documentation...

There is a chance the CTF tool will get a second option to generate FBCTF data instead of CTFd. That would invalidate or complicate this enhancement, so I'm setting it to blocked for the time being.


Fix syntax errors in setup.py

There are minor syntax errors in setup.py. This Pull request solve there errors.

It seems like it is already opened in PR #19. Thus, closing this one.


Apim 7497 api secrets

https://gravitee.atlassian.net/browse/APIM-7497
Description

Use gravitee-secret-api
Remove constraint on URL starting with provider plugin
Adapt to new SecretProvider contract

Additional context

Gravitee.io Automatic Deployment
🚀 A prerelease version of this package has been published on Gravitee's private artifactory, you can:

use it directly by updating your project with version: 2.0.0-APIM-7497-api-secrets-SNAPSHOT
download it from Artifactory here

:tada: This PR is included in version 2.0.0-alpha.1 :tada:
The release is available on:

2.0.0-alpha.1
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.0.0 :tada:
The release is available on:

2.0.0
GitHub release

Your semantic-release bot :package::rocket:


Add health check endpoint

Add a simple /health endpoint returning 200.

Sure, and maybe we should follow the Hasura convention of /healthz to bee as cool as them and Google!


Release test versions such as nightly, canary, dev, beta, ...

Changes in the core may unintentionally and unexpectedly impact many users and plugins. We may want to release testing versions before actual releases.
See also:

Chrome (Chromium)'s release channels: https://www.chromium.org/getting-involved/dev-channel

@dmikurube
It's a nice idea!.
This is a little off-topic.
Please take care #318.
I'm not sure about current(Un-JRuby) selfupdate algorithm.
Maybe you may need to implement the latest version checker.

Jfyi, the new selfupdate is implemented based on Implementation-Version in JAR meta data. It's implemented before v0.4.0, so I think it's fine.
https://github.com/embulk/embulk/commit/16aa280e41a4439f3e9ff1f4135247c864a96469

Sorry, I now understood your mention. I just confirmed that Bintray's _latestVersion looks taking care of only uploaded date/time as mentioned in #318. It means that:

When 0.5.2 exists,
And then 0.5.3-ALPHA is uploaded,
0.5.3-ALPHA will be _latestVersion.

It's not very good as selfupdate depends on _latestVersion. Even if we change selfupdate from a certain version, older versions (which are already installed at users) take a look into Bintray's _latestVersion.
We may not have a deprecation period unfortunately since we want this kind of unstable versions from 0.9.* soon. One solution may be to have another repository for unstable versions, such as:
https://bintray.com/embulk/maven/embulk-unstable
After that, we may want to have a better solution such as, 1) changing selfupdate or 2) switching to another repository, with an enough deprecation period.


Add process and pull request template for nominating voting members

Based on  https://github.com/wipfli/maplibre/pull/1

This adds a pull request template for creating a nomination for a voting members, so people can select this template when making a pull request, so they don't need to copy paste the template by hand.
Explanation of the process written by @wipfli is added to VOTING_MEMBERS.md.
I created a new label voting-member-nomination which is added by the template link so people have an overview of all nominations and can vote on the ones they want.

Draft pull request, the HubSpot link needs to be added in two places.

Thanks for your help with this @louwers. Can you target the repo wipfli/maplibre with this pull request such that we can iterate on it there before we merge it here and announce it all in one go?


[Perf Tracker] Falcon-180B-GPTQ

Current Perf with 4-way sharding: 79.5 seconds/token
Current Perf with single layer sharding: 94 seconds/token
Tracy Profile Screenshot (4-way sharding): For the first shard out of the four shards -
https://drive.google.com/file/d/1nLZBYR-o6yb1tHLOqYgg569n3QOoMWKV/view?usp=sharing
Tracy Profile File: https://storage.googleapis.com/shark_tank/falcon/sharded/falcon_180b/falcon_180b_layer_0_20_int4_rocm.tracy
Dispatches are available at: shark_tank/falcon/sharded/falcon_180b/falcon_180b_layer_0_20_int4_rocm_dispatches/
URL: https://console.cloud.google.com/storage/browser/shark_tank/falcon/sharded/falcon_180b/falcon_180b_layer_0_20_int4_rocm_dispatches?pageState=("StorageObjectListTable":("f":"%255B%255D"))&authuser=1&prefix=&forceOnObjectsSortingFiltering=false
Commands to run the model:
1.) 4-way sharded:
python apps/language_models/src/pipelines/falcon_pipeline.py --falcon_variant_to_use="180b" --precision="int4" --device="rocm" --sharded --compressed
2.) Single layer sharded:
python apps/language_models/src/pipelines/falcon_pipeline.py --falcon_variant_to_use="180b" --precision="int4" --device="rocm" --sharded --compressed

This is supposed to be int4 quantized, but the matmuls are all f32. The first major step will be avoiding the materialization of the full f32 weights and keep things in i4.

Adding a follow up comment from an offline discussion. The IR that I have seen for this model will be quite difficult for IREE to handle. The first thing that needs to happen to improve performance here is to fuse the dequantization with the matmul (thereby removing the f32 matmuls), however the IR I remember seeing for this was using bitwise arithmetic to extract the i4 values from the i32 weight tensor into an i8 tensor (wasting 4 bits because pytorch can't represent i4 values).  Additionally, to handle the quantization grouping, it was broadcasting the scale/zero point values with a gather. For example,
broadcast 32x1 -> 32x128
torch.broadcast op

broadcast 32x1 -> 4096
index_tensor = [0, 1, 2, 3, ..., 31, 0, 1, ..., 31, ...] (tensor<index x 4096>)

@vivekkhandelwal1 are you able to upload the full elided IR either at the Torch or Linalg level? (or both). That would help with verifying that my claims about the IR are correct. Then we can see about improving the implementations within autogptq if need be.

@qedawkins, here's the torch and linalg elided IR:
Torch IR: https://storage.googleapis.com/shark_tank/falcon/sharded/falcon_180b/falcon_180b_gptq_layer_0_20_int4_elided_torch.mlir
Linalg IR: https://storage.googleapis.com/shark_tank/falcon/sharded/falcon_180b/falcon_180b_gptq_layer_0_20_int4_elided_linalg.mlir
Also, the matmuls were originally in fp16 which were upcasted to fp32 because of the numerical precision issue. You can see in the IR that the results of the matmuls are downcasted to fp16.

Looking at the IR, quite a few issues:

What @qedawkins said in the above is to the point. We have quite some logic to process the i32 tensor weights on the fly, using i8 as an subsitute where i4 should exist. I'm expecting the weight should be directly encoded as i4 tensor. Fixing this would help us to avoid the redundant computation and make the IR less confusing.

    %expanded = tensor.expand_shape %cst_184 [[0], [1, 2]] : tensor<1x1152xi32> into tensor<1x1152x1xi32>
    %27 = tensor.empty() : tensor<1x1152x8xi32>
    %28 = linalg.generic {indexing_maps = [#map2, #map1], iterator_types = ["parallel", "parallel", "parallel"]} ins(%expanded : tensor<1x1152x1xi32>) outs(%27 : tensor<1x1152x8xi32>) {
    ^bb0(%in: i32, %out: i32):
      linalg.yield %in : i32
    } -> tensor<1x1152x8xi32>
    %expanded_201 = tensor.expand_shape %cst_183 [[0, 1], [2]] : tensor<1x8xi32> into tensor<1x1x8xi32>
    %29 = linalg.generic {indexing_maps = [#map, #map5, #map1], iterator_types = ["parallel", "parallel", "parallel"]} ins(%28, %expanded_201 : tensor<1x1152x8xi32>, tensor<1x1x8xi32>) outs(%27 : tensor<1x1152x8xi32>) {
    ^bb0(%in: i32, %in_984: i32, %out: i32):
      %1610 = arith.shrsi %in, %in_984 : i32
      linalg.yield %1610 : i32
    } -> tensor<1x1152x8xi32>
    %30 = tensor.empty() : tensor<1x1152x8xi8>
    %31 = linalg.generic {indexing_maps = [#map, #map1], iterator_types = ["parallel", "parallel", "parallel"]} ins(%29 : tensor<1x1152x8xi32>) outs(%30 : tensor<1x1152x8xi8>) {
    ^bb0(%in: i32, %out: i8):
      %1610 = arith.trunci %in : i32 to i8
      linalg.yield %1610 : i8
    } -> tensor<1x1152x8xi8>
    %32 = linalg.generic {indexing_maps = [#map, #map1], iterator_types = ["parallel", "parallel", "parallel"]} ins(%31 : tensor<1x1152x8xi8>) outs(%30 : tensor<1x1152x8xi8>) {
    ^bb0(%in: i8, %out: i8):
      %1610 = arith.andi %in, %c15_i8 : i8
      linalg.yield %1610 : i8
    } -> tensor<1x1152x8xi8>
    %33 = linalg.generic {indexing_maps = [#map, #map1], iterator_types = ["parallel", "parallel", "parallel"]} ins(%32 : tensor<1x1152x8xi8>) outs(%30 : tensor<1x1152x8xi8>) {
    ^bb0(%in: i8, %out: i8):
      %1610 = arith.addi %in, %c1_i8 : i8
      linalg.yield %1610 : i8
    } -> tensor<1x1152x8xi8>
    %collapsed_202 = tensor.collapse_shape %33 [[0], [1, 2]] : tensor<1x1152x8xi8> into tensor<1x9216xi8>
    %expanded_203 = tensor.expand_shape %collapsed_202 [[0], [1, 2]] : tensor<1x9216xi8> into tensor<1x1x9216xi8>
    %expanded_204 = tensor.expand_shape %cst_182 [[0], [1, 2]] : tensor<1x9216xf16> into tensor<1x1x9216xf16>
    %expanded_205 = tensor.expand_shape %cst_181 [[0], [1, 2]] : tensor<1024x9216xi32> into tensor<1024x1x9216xi32>
    %34 = tensor.empty() : tensor<1024x8x9216xi32>
    %35 = linalg.generic {indexing_maps = [#map6, #map1], iterator_types = ["parallel", "parallel", "parallel"]} ins(%expanded_205 : tensor<1024x1x9216xi32>) outs(%34 : tensor<1024x8x9216xi32>) {
    ^bb0(%in: i32, %out: i32):
      linalg.yield %in : i32
    } -> tensor<1024x8x9216xi32>
    %expanded_206 = tensor.expand_shape %cst_183 [[0], [1, 2]] : tensor<1x8xi32> into tensor<1x8x1xi32>
    %36 = linalg.generic {indexing_maps = [#map1, #map3, #map1], iterator_types = ["parallel", "parallel", "parallel"]} ins(%35, %expanded_206 : tensor<1024x8x9216xi32>, tensor<1x8x1xi32>) outs(%34 : tensor<1024x8x9216xi32>) {
    ^bb0(%in: i32, %in_984: i32, %out: i32):
      %1610 = arith.shrsi %in, %in_984 : i32
      linalg.yield %1610 : i32
    } -> tensor<1024x8x9216xi32>
    %37 = tensor.empty() : tensor<1024x8x9216xi8>
    %38 = linalg.generic {indexing_maps = [#map1, #map1], iterator_types = ["parallel", "parallel", "parallel"]} ins(%36 : tensor<1024x8x9216xi32>) outs(%37 : tensor<1024x8x9216xi8>) {
    ^bb0(%in: i32, %out: i8):
      %1610 = arith.trunci %in : i32 to i8
      linalg.yield %1610 : i8
    } -> tensor<1024x8x9216xi8>
    %39 = linalg.generic {indexing_maps = [#map1, #map1], iterator_types = ["parallel", "parallel", "parallel"]} ins(%38 : tensor<1024x8x9216xi8>) outs(%37 : tensor<1024x8x9216xi8>) {
    ^bb0(%in: i8, %out: i8):
      %1610 = arith.andi %in, %c15_i8 : i8
      linalg.yield %1610 : i8
    } -> tensor<1024x8x9216xi8>
    %collapsed_207 = tensor.collapse_shape %39 [[0, 1], [2]] : tensor<1024x8x9216xi8> into tensor<8192x9216xi8>
    %expanded_208 = tensor.expand_shape %collapsed_207 [[0, 1], [2]] : tensor<8192x9216xi8> into tensor<1x8192x9216xi8>
    %40 = tensor.empty() : tensor<1x8192x9216xi8>
    %41 = linalg.generic {indexing_maps = [#map, #map5, #map1], iterator_types = ["parallel", "parallel", "parallel"]} ins(%expanded_208, %expanded_203 : tensor<1x8192x9216xi8>, tensor<1x1x9216xi8>) outs(%40 : tensor<1x8192x9216xi8>) {
    ^bb0(%in: i8, %in_984: i8, %out: i8):
      %1610 = arith.subi %in, %in_984 : i8
      linalg.yield %1610 : i8
    } -> tensor<1x8192x9216xi8>
  ...

Once with the above done we need to fuse the i4 dequant linalg.generic togther the matmul linalg.generic together to avoid materializing the f32 inputs, which costs compute and memory bandwidth.
The upcasting from f16 to f32 and then downcasting could be problematic given that it may require extra allocations depending how we fuse stuff.. I haven't verified this is indeed an issue though. I guess it's necessary for 180B. Do we also need that for 40B (https://github.com/openxla/iree/issues/15461)?
Even looking at the current f32 matmul pipeline, the heuristics is not good -- for MxNxK=100x8192x32768, we are using a workgorup size of 32 threads.. ROCm basically is reusing the hardcoded tile sizes chosen for CUDA A100.. We should clean that up.

This issue look stale. Can we close it?


Added favicon

Related Issue/Addition to code

#53

Type of change
Added favicon
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Proposed Changes

Added favicon image in images folder.

Edited the code to add the favicon image in index.html

Additional Info

any additional info for context

Checklist:

[x] My code follows the style guidelines of this project and have read CONTRIBUTING.md
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules
[x] I have checked my code and corrected any misspellings

Screenshots

Original
Updated

**

**  **

Sure.

Any updates @sunny2329?

Sorry but I am stuck with my end semester papers.

Hope you did well in them.. Any update?

Someone has begun work on this issue.


Konveyor Addon

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

hello @elamaran11
This is Celestino Bellone, I am an employee of Claranet Switzerland, and I am one of the maintainers of https://github.com/claranet-ch/konveyor-eks-blueprint-addon .
As requested, this is to confirm that Claranet will support the ongoing maintenance of the Konveyor EKS Addon.
Let me know if you need anything else from our side.
Thanks,
Celestino


Add GitHub Action

Add GitHub Action

Add GitHub Action

Build test using Python3.7, 3.8, 3.9, 3.10. This is because we already merge libyt yt frontend to yt-4.1.dev, which supports Python >= 3.7 only. (link)
Test on ubuntu-latest.
Test:

Can it build successfully.
Can libyt/example run in serial.
Can libyt/example run in parallel (MPI=3).

Update Makefile. Make it more consistent.
Update Readme.md.

There are so many commits because I'm testing GitHub Action. This is my first time using this. :sweat:


Rethinking IArgumentModel

While writing the feature tests for IArgumentModel I realized the design does not guarantee a consistent order for the arguments.  .Net does not guarantee the order properties are reflected.  It's generally consistent but not guaranteed.
Additionally, as a developer, I don't think about the order of properties in a class.  I wouldn't expect moving properties around within a class to break a contract.  The current design violates the principle of least surprise.
When we think about methods, we understand that position is an integral part of the contract.
Given that operands are only positional and options are only named, IArgumentModel is currently only well suited for Options.  To be suitable for operands, we'd need a new ArgumentAttribute (or OperandAttribute) that requires position when used on properties.  I wonder if it would be simpler & easier to let IArgumentModel only be used for options and force operands to be defined as method parameters.
thoughts @bilal-fazlani ?
What was the original intention for IArgumentModel?  Was it to enable fluent validators?

Finally got some time to read this.. sorry for my late responses.

.Net does not guarantee the order properties are reflected. It's generally consistent but not guaranteed.

I didn't know this!

What was the original intention for IArgumentModel? Was it to enable fluent validators?

Yes. Exactly that.

I wonder if it would be simpler & easier to let IArgumentModel only be used for options and force operands to be defined as method parameters.

This will solve this issue. But it will take away fluent validations from operands.

IArgumentModel is currently only well suited for Options. To be suitable for operands, we'd need a new ArgumentAttribute (or OperandAttribute) that requires position when used on properties.

I wish we could do something without adding any responsibility to app devs... not sure what at this moment..

This is mostly theoretical.   We can wait until someone raises it as an issue or we come up with an ideal solution.  Maybe it's worth calling out in the documentation?
If we wanted to address this in a backward compatible fashion, I think we add an optional Argument.Position property that can be used in cases when needed, but mostly ignored.
On the topic of validation: I think validation should be a middleware.  Fluent validations would be one option.  Another would be System.ComponentModel.DataAnnotations which would work for both property and parameter.  DataAnnotations cover a lot of scenarios and also make it easy to provide hints in the help output.    It's too bad there isn't an option to declare validations for methods using expressions.


CNNDM test set output

Hi, can you please release the decoded summary for CNNDM test set?

Hi, can you please release the decoded summary for CNNDM test set?

ProphetNet-large (160GB) outputs on CNN/DM: Link

Thank you!


modified the compiler to parse and consume the external keyword.

The compiler is now able to parse the external keyword. However, I did add an argument to the function add_primitive in builder.rs, which was used extensively in other files.

One small question. Use your judgement to resolve it and then merge!


feat(es/lint): implement yoda rule

Description:
implement yoda rule
BREAKING CHANGE:
No

This is also about code style, isn't it?
I'm not sure about this rule

Yes. This rule restrict using literals on left side (e.g. 10 === a)

@kdy1 Also you can always ping/assign me in an issue and I will give advice about rule/naming/options/etc

Thanks! I'll review this soon.

CI is failing.
Can you run UPDATE=1 cargo test from ./crates/swc after pulling the changes (merge commits for rebasing)