chore(deps): update module github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper to v1.42.7

This PR contains the following updates:

Package
Type
Update
Change

github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper
require
patch
v1.42.6 -> v1.42.7

Release Notes

terraform-ibm-modules/ibmcloud-terratest-wrapper (github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper)
v1.42.7
Compare Source
Bug Fixes

deps: update module github.com/ibm/vpc-go-sdk to v0.64.0 (#906) (1d061e2)

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

:tada: This PR is included in version 1.0.25 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


More files

Added more filelists (cross sections are work in progress but in the meantime I think this could be merged).
BTW also fixes small typo in bbbbSkimmer.py

LGTM


migrate everything to jinja2

fixes #283

[ ] TODO: There's too many unwraps/panics in the template code (although it really should be infallible...)

I can clean up the panics more later, they are genuinely "you messed up cargo-dist's code" errors, and getting this in now would be Good


Adds a new explorerViewer sort order configuration item, barrelFirst

This setting sorts barrel files (index.js, index.ts, etc.) in alphabetical order above all other files. Folders remain above all files including barrel files.
Partially addresses #27286 (still open but not specifically calling for this) and a comment requesting this feature on a now closed issue, #12345.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: SimonDever sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Thanks for this PR. However it is usually great to first start a discussion before imlpementing the whole PR.
We will not accept this since this is very javascript specific and not a lot of users requested for this.
Leaving this open as we might change our mind in the future. Thanks

That's fair enough. It doesn't specifically look at file extensions like .ts or .js, just any file starting with index. Would you be able to advise if this functionality (changing file explorer sort order) is possible from an extension? I have looked for how to achieve it but found nothing at all so far about changing file explorer like that.

Unfortunetely this is not possible using an extension. And that is one of the reason why I will keep this PR open for now, since there is no other way to achieve this.

Clossing as out of scope.
Thank you again for your PR.

Is there another way to achieve this yet? Is there a plan to open up API to allow this or anything further?

There is currently no way to achieve this. We do not plan to open up the API.
There is this feature request which you can comment on and upvote https://github.com/microsoft/vscode/issues/27286

Recommend changing 'barrel' to 'index' in every use across this PR to make the name more intuitive and specific. Hoping this gets resolved some day...


Docker container will not start in Unraid

I'm running into issues getting the mailrise docker container to run in Unraid. I am running Unraid 6.12.10 with Docker
24.0.9. When I try to start the container this is the error that I receive.
CRITICAL:mailrise.skeleton:No Apprise targets are configured

I know that Unraid is a bit different with mapping the config file, so I'm guessing it's something related to that. After looking at the solution to this issue #50, below is the current config that I have set up.
mailrise.conf
configs:
  discord:
    urls:
      - discord://webhook

config.yml
version: '1.0'

services:
  mailrise:
    container_name: mailrise
    image: yoryan/mailrise:latest
    ports:
      - 8025:8025
    environment:
      - PUID=1000
      - PGID=1000
    volumes:
      - /mnt/user/appdata/mailrise/mailrise.conf:/etc/mailrise.conf

docker run
docker run
  -d
  --name='mailrise'
  --net='bridge'
  -e TZ="America/Chicago"
  -e HOST_OS="Unraid"
  -e HOST_HOSTNAME="Name"
  -e HOST_CONTAINERNAME="mailrise"
  -l net.unraid.docker.managed=dockerman
  -p '8025:8025/tcp'
  -v '/mnt/user/appdata/mailrise/':'/etc/mailrise':'ro' 'yoryan/mailrise:latest' /etc/mailrise/config.yml

I'm sure there is something simple that I am overlooking. Any ideas what I am doing wrong?

you mailrise.conf looks wrong. discord://webhook needs to be replaced by your config.  It needs your specific discord://webhook_id/webhook_token

My bad I should have added more detail there. That is how I have it formatted in the actual config file. I just put webhook there when I redacted the actual values.
I don't remember exactly where I left this off, but I did try other services besides Discord as was getting the same error. I ended up deciding it was something I had configured strange on my Unraid server preventing the container from starting. I ended up writing a python script instead to route the messages.


As a registered and logged-in user, I want to be able to share links to pictures that I like with my friends to get other people to see this specific picture.

Acceptance Criteria

[x] sopra-fs22-group-20/client#109
[x] sopra-fs22-group-20/client#110
[x] sopra-fs22-group-20/client#111
[x] sopra-fs22-group-20/client#112

Development Tasks FE

[x] sopra-fs22-group-20/server#198

Development Tasks BE

We decide not to implement this function because of the principle of randomness in our application. We don't want famous people to post the link on Instagram and get many likes for a specific picture.


Mojolicious::Lite provides no shutdown hooks

Would be great to have such a feature.
Appending already registered INT/TERM/QUIT signal callback functions instead of just "overwriting" them would be ok, too.
--- /opt/perl-5.14.2/lib/site_perl/5.14.2/Mojo/Server/Daemon.pm	2011-12-23 03:46:01.000000000 +0100
+++ ../Mojo/Server/Daemon.pm	2012-01-26 17:17:15.385124609 +0100
@@ -68,7 +68,9 @@
$self->setuidgid;
Signals

$SIG{INT} = $SIG{TERM} = sub { exit 0 };

if(!defined($SIG{INT}) && !defined($SIG{TERM})) {

 $SIG{INT} = $SIG{TERM} = sub { exit 0 };

}
Start loop
$self->ioloop->start;

That seems like a very poor solution, considering signals are platform specific.

Definitely.
I'm not really into platforms other than Linux. But given that only INT
and TERM are overwritten, it is an appropriate quick fix. :-)
On Thu, January 26, 2012 17:26, Sebastian Riedel wrote:

That seems like a very poor solution, considering signals are platform
specific.

Reply to this email directly or view it on GitHub:
https://github.com/kraih/mojo/issues/271#issuecomment-3671538

--
The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC-2119
[http://www.ietf.org/rfc/rfc2119.txt].

It's not just platform dependent in the sense that signals work differently between operating systems, every web server we support also handles signals differently. This is not a solution at all.

Anyway, i'm going to close this issue now, you're welcome to reopen it should there be any actual reasons to hack up signals instead of just using END/DESTROY blocks.

END/DESTROY are not really useful for controlled, ordered shutdowns where one part needs other parts still working, thereby implying some ordering. However, in my simple case, it could work out just fine.


[BUG] su inside a pot does not work due to wrong root fs permissions

Describe the bug
Trying to su to another user account inside a jail created with Pot will fail due to wrong access permissions for /.
To Reproduce
Steps to reproduce the behavior:

Create a pot
Try to change to another user with su
See error

Expected behavior
I expect to get a shell for that other user
Additional context
The workaround is to run chmod 0755 / inside the jail.

This is a duplicate of #233
It has already been fixed in master (#234)
@pizzamig what about doing a new release soon (maybe after #240 has been merged).

Duplicate of #233


How can I finetune Olmo-7B using this repo?

❓ The question
Hi @epwalsh, @2015aroras
I am trying to setup finetuning for OLMo-7B  from huggingface model, but seems like that might not be possible, as the checkpoint requires trainer and optimizer states e.t.c.
Is it possible to use this repo for finetuning from the public checkpoint? or if I can download the checkpoint that is compatible with this repo?
Thanks.

Hey @rajasekharmekala, see: https://github.com/allenai/OLMo/pull/500.

Thanks @epwalsh and @2015aroras. I was able to load the model from the checkpoints in #500


WorkAround #2179

WorkAround #2179.
Basically, this is a technical issue that I've recreated in a virtual environment.
I consider this as a bug, so on the documentation itself, there's nothing related to this issue that should be added without a full revision from an expert on Microsoft.

@nenonix @JohanFreelancer9 - kindly do the copy editor review for this PR. Thanks.

@andreabichsel - can you please assist to check, approve and merge the content for this PR based on issue #2179?
Thank you.

@j0rt3g4 : Thank you for adding the corrections. The 2 redundant blank lines are still present, though. (Maybe there is a missing commit you have not pushed to Github yet?)

OK I'm gonna check it what lines numbers

258 + 259 seem to be the redundant line numbers in your current PR.

@j0rt3g4 - please see comment from illfated on the redundant lines and do the necessary changes to this PR. Thank you.

@mypil  (done).

@andreabichsel - Can you please assist to check, approve and merge the content for this PR based on issue #2179?
Thank you.

@AndreaBarr - would you be able to merge the content of this PR based on issue #2179? Thank you.


Some questions about AttributeProvider

I want to make it possible, that a developer can use the Spring Expression Language within policies. For example to get the username of the current principal or get parameters of a method invocation.
My first idea was to use attributeId of AttributeDesignator, so the developer e.g. can write attributeId="${principal.username}" or attributeId="${context.parameter[0]}". Therefor I would only need an AttribteProvider. But after a look in the code of their implementations and use in DepthLimitingExpressionFactory I realized that they seemed very static.
Did I understand it right, that DesignatedAttributeProvider doesn't support dynamic AttributeDesignator like in the example above?
Is it possible to pass something like a context object into a request so that an SpEL-AttributeProvider could use it?
Or is there a better way to solve this problem?

The ABAC/XACML-Integration for Spring Security is getting ready soon and I found a solution for my problem:
I map my security context to an identifier and put this identifier as an attribut to the request context. Then I write an AttributeProvider (PIP), that should resolve all missing attributes by asking my context handler (like an the flowchart above).
The new problem is, that the AttributeProvider only knows if he can resolve the missing Attribute to an existing Attribute on runtime when get(final AttributeFqn attributeGUID, final BagDatatype<AV> returnDatatype, final EvaluationContext context) get called. So it could be possible that he can't resolve it and he also can never return sth. to getProvidedAttributes().
Where should I start?

Do you mean: Where should I start to implement the AttributeProvider?
See the dedicated wiki page. The recommended way is to start with a separate Maven project and XML schema for your AttributeProvider's configuration. In your case, since it is quite generic, this configuration should enable the user/integrator to define (or control somehow) which XACML Attribute(s) will be provided by the AttributeProvider, so that he/she can make sure they can be used in the policy.

I mean that currently it seems not easy to write an own AttributeProvider that is not based on DesignatedAttributeProvider and I dont't know where to start without writing a new ExpressionFactory.
Important: Actually I use AuthZForce completely without XSD and XML because I personally believe that this is not a good way to integrate it in Spring Boot with .properties-files etc.

OK, I guess you want to use the BasePdpEngine(ExpressionFactory, RootPolicyProvider, ...) constructor to instantiate the PDP (without using XML). So yes indeed, you need an ExpressionFactory. You can use the default implementation (DepthLimitingExpressionFactory) like in the code of PdpEngineConfiguration constructor:
xacmlExpressionFactory = new DepthLimitingExpressionFactory(attValFactoryRegistry, functionRegistry,
attProviderFactories, maxVarRefDepth, enableXPath, strictAttributeIssuerMatch);

The argument that matters to you here is attProviderFactories which is where the AttributeProviders come in. It's a list of CloseableDesignatedAttributeProvider.DependencyAwareFactory. So in order to plug-in your new AttributeProvider directly (without using XML stuff), just make your AttributeProvider's factory class implement this DependencyAwareFactory interface directly. For that, you don't need XML, so you can handle the configuration in your own way. Make sure that your implementation of getDependencies() returns the special attribute (designator) holding the security context, that you mentioned in your second to last comment. This will tell the PDP engine that your AttributeProvider requires / depends on this attribute to work.

Thank you for the explanation, but I already did that a month ago and it works perfect :-)
The point is, that implementing DependencyAwareFactory will cause that I must use CloseableDesignatedAttributeProvider as AttributeProvider implementation. The DepthLimitingExpressionFactory doensn't allow to use completely custom AttributeProvider without the getProvidedAttributes()-method, but that's what I need.

This method is there for a reason. Just to understand why you are unable to implement it, could you explain how does one know which XACML attributes will be provided by your AttributeProvider (in a given configuration), in terms of Attribute Category, AttributeId, Datatype, etc. ?  What are the configuration parameters? An example would be very welcome.

As mentioned previously the devs, who will use my Spring-ABAC-Framework should not take care of the specific XACML-Framework (e.g. AuthZForce).
There are options for the devs (e.g. config files or classes) to e.g. map SpEL Expressions to AttributeDesignators. The attributes are provided by a central PIP of my Framework. Because the attribute mapping ideally should be configurable on runtime, my PIP never knows if or if not there will be a value for an attribute query.
My PIP now only need to be connected to the PDP-Framework (in AuthZForce AttributeProvider). And that's why getProvidedAttributes() can't be used.
If something isn't clear let me know.

Just to clarify: you want to enable the devs to use your ABAC framework without using any XACML-specific stuff; except that if they want to configure the mappings of SpEL expressions to attributes, they have to use XACML AttributeDesignators (for the latter). Am I correct?
Then one more question: are you not using XACML for policies at all? Your own policy language then?

Not without using XACML stuff but without using XACML-Framework related stuff like AuthZForce's AttributeProvider.
For the SpEL mapping there is a class named RequestAttributeMetadate that is very similar to AttributeDesignator.
I think XACML is perfect as policy language, so I didn't defined an own one. If someone want his own language, he can integrate it very easily and only has to change the policies and not the PEP or PIP stuff

In that case, it does not seem difficult to implement getProvidedAttributes(). From the SpEL mappings, you know the list of AttributeDesignators potentially provided - i.e. supported - by the AttributeProvider. This is all you need to return in  getProvidedAttributes().  It does not matter if some of them are never used in the policies (and therefore not actually provided/used at runtime). What matters is that your AttributeProvider can provide them if requested (by some policy evaluation), using the SpEL-to-XACML-attribute (AttributeDesignator or RequestAttributeMetadata if you prefer) mappings defined  for them in configuration. (Maybe I should rename this method to getSupportedAttributes at some point to make it less ambiguous.)  By the way, AttributeDesignator is standard XACML, not AuthzForce-specific, so there's nothing wrong with using it if you want stay framework-agnostic. And if you still use XACML policies, you have to use AttributeDesignator anyway. (In AuthzForce, the corresponding Java class is directly derived from the standard XACML schema using JAXB, like in AT&T implementation.)

In that case, it does not seem difficult to implement getProvidedAttributes(). From the SpEL mappings, you know the list of AttributeDesignators potentially provided - i.e. supported - by the AttributeProvider. This is all you need to return in getProvidedAttributes(). It does not matter if some of them are never used in the policies (and therefore not actually provided/used at runtime). What matters is that your AttributeProvider can provide them if requested (by some policy evaluation), using the SpEL-to-XACML-attribute mappings defined in configuration.

And this will also work, when someone adds an attribute mapping on runtime? I'm asking that because I'm also planning an integrated online tool to create and edit the XACML policies and the attribute mappings on rumtine without a restart of the application.

By the way, AttributeDesignator is standard XACML, not AuthzForce-specific, so there's nothing wrong with using it if you want stay framework-agnostic. And if you still use XACML policies, you have to use AttributeDesignator anyway.

Yeah I know that, but I implemented an ABAC-API for Spring that is completely independent from XACML. If the dev chooses XACML as one possible implementation for this API, he ideally only has to define the policies in XACML, the mapping is independent of it. If he switches to another implementation, he only needs to migrate the policies.

It will work fine if you re-instantiate the PDP engine after changing the attribute mapping. There may be a more efficient way, but at this point, I think I cannot help you efficiently unless I can see some  code. Could you share the source code, even in early prototype stage ?
Anyway, this is a quite ambitious and promising goal you have here :+1:


Fixed #34560 -- Updated admin's jQuery to 3.7.1.

ticket-34560
Merge just before the future freeze on September 18, 2023.

@sarahboyce Thanks for checking :+1:


Markdown Preview + HTML can manipulate Atom itself

Woo hoo!

Thanks for the report!  Since this issue is related to Markdown preview, can you re-open in https://github.com/atom/markdown-preview and completely fill out the issue template in the new issue?

Already filed at https://github.com/atom/markdown-preview/issues/276.


Fix Update Lease Conflict

Description
This pr fixes a bug already addressed in the upstream virtual kubelet repository. It can be dropped when upgrading the kubelet code.
This bug prevents the node lease to be correctly updated by the node provider after a transitory API server issue.
That is the log of the virtual kubelet in that case:
I0624 14:28:48.845622       1 namespaceMapper.go:72] namespaceNattingTable cache initialized
I0624 14:28:48.950864       1 root.go:287] advertisement not found, setting empty owner reference
I0624 14:28:49.046036       1 root.go:277] setup ended
I0624 14:28:49.046061       1 resourceWatcher.go:59] Liqo informers started
I0624 14:28:49.049781       1 reconciler.go:79] node correctly updated from resourceOffer
I0624 14:28:49.060621       1 reconciler.go:194] correctly set pod CIDR from tunnel endpoint
I0624 14:28:49.062189       1 podcontroller.go:282] Pod cache in-sync
I0624 14:28:49.062452       1 podcontroller.go:324] starting workers
I0624 14:28:49.062464       1 podcontroller.go:338] started workers
I0624 14:28:49.269875       1 reconciler.go:79] node correctly updated from resourceOffer
I0625 00:00:42.694839       1 reconciler.go:79] node correctly updated from resourceOffer
I0625 00:25:04.211517       1 reconciler.go:79] node correctly updated from resourceOffer
I0625 00:25:04.258284       1 reconciler.go:79] node correctly updated from resourceOffer
E0625 01:57:58.216709       1 node.go:294] etcdserver: request timed out - Error while handling node ping
E0625 01:58:09.602193       1 node.go:294] Operation cannot be fulfilled on leases.coordination.k8s.io "liqo-2aac7bf5-0349-47db-836a-13df66cec619": the object has been modified; please apply your changes to the latest version and try again - Error while handling node ping
E0625 01:58:20.514292       1 node.go:294] Operation cannot be fulfilled on leases.coordination.k8s.io "liqo-2aac7bf5-0349-47db-836a-13df66cec619": the object has been modified; please apply your changes to the latest version and try again - Error while handling node ping
E0625 01:58:32.150979       1 node.go:294] Operation cannot be fulfilled on leases.coordination.k8s.io "liqo-2aac7bf5-0349-47db-836a-13df66cec619": the object has been modified; please apply your changes to the latest version and try again - Error while handling node ping
E0625 01:58:42.153110       1 node.go:294] Operation cannot be fulfilled on leases.coordination.k8s.io "liqo-2aac7bf5-0349-47db-836a-13df66cec619": the object has been modified; please apply your changes to the latest version and try again - Error while handling node ping
E0625 01:58:52.212428       1 node.go:294] Operation cannot be fulfilled on leases.coordination.k8s.io "liqo-2aac7bf5-0349-47db-836a-13df66cec619": the object has been modified; please apply your changes to the latest version and try again - Error while handling node ping
...

/rebase

/merge


added stacked_gql package

added stacked_gql package

awesome! I can't wait to migrate all the client apps to using this over the next two weeks.


Fixed image file extension detection

Previously this would fail on jpeg and would detect something such as notajpg as jpg.
This fixes issue #929.

Looks good. :+1: :cat2:

:+1:

This PR has been added to my merge list, meaning it will be merged soon into the branch 2.3.x, unless someone raises any concerns.

Merged in df99d5fee223d018e1a7a4c4a1f3372f76d4edf5 on branch 2.3.x


Allow compiling without JUCE_MODAL_LOOPS_ENABLED

I'm not using these parameter tests, so I've been having this in my fork for a while, so I don't need to compile with JUCE_MODAL_LOOPS_ENABLED.
What do you think of it?

Hm, I don't even remember why that's needed... I'll look into it to see what action I might need to take if we remove this...


react-navigation 和 react-native-swiper 冲突

在demo中使用了 react-navigation 作为分屏组件，在Home屏中使用了 react-native-swiper，所有配置项正确有效的情况下，autoplay = true 时，出现了底部圆点标识在切换但是页面不切换。
const Tab = TabNavigator(   {     Home: {       screen: HomeScene,       navigationOptions: ({ navigation }) => ({         tabBarLabel: '首页',         tabBarIcon: ({ focused, tintColor }) => (           <TabBarItem             tintColor={tintColor}             focused={focused}             normalImage={require('./assets/img/tabbar_home.png')}             selectedImage={require('./assets/img/tabbar_home_selected.png')}           />         )       })     }) export default class Root extends React.Component {   render() {     return (       <HomeScene/>     )   } }
在 HomeScene组件中使用了 react-native-swiper
let renderSwipeView = () => {       return (         sliderImgs.map((item, i) => {           return (             <View style={[{width: w, height: h}, styles.SwiperItem]}>               <Image key={i} source={{uri: item}} style={{width: utils.os().width,height: utils.scaleSize(380)}}/>             </View>           )         }     ) } <Swiper               height={utils.scaleSize(380)}               autoplay={true}               loop={false}>               {renderSwipeView()}             </Swiper>

What is the actual issue?

Can you describe your issue in English please ?

when demo used as TabScreen of  TabNavigator
text content not show , but arrows and buttons can see

@arribbar @mrdezzods
i also confused with this.
React-native-swiper seems to conflict with React-navigation , when React-navigation use the module of tabNavigator, Swiper seems to stuck when drag ,and when the swiper button show and when you click , the dot slide , but the content does not change . it look like:

and what is more, when React-navigation enable option of swipeEnable(set 'swipeEnable' to true)

Swiper show nothing. it look like:

the code of using Swiper is the Example code.
so that is it.

Encounter the same problem, looking for help ! @walkerXiong @arribbar

@dx51job go to the lib react-navigation and search issue of key word 'swiper', there is an issue about it, the solution is add an setTimeout, i dont know why, but it did work. see the url i post above, copy the url to browser(click seems to 404), and find the solution.

@walkerXiong It works when I use tabnavigator only once. After I use tabnavigator component in anathor tabnavigator, the problem shows again.
Code suchas：
const page1 = TabNavigator({
    A : {screen : A},
    B : {screen : B}
});
const page2= TabNavigator({
    C : {screen : C},
    D : {screen : D}
});
const page3 = TabNavigator({
    p1 : {screen : page1},
    p2 : {screen : page2}
});

The swiper component in A stucked when drag.
Furthermore, when I enable the animationEnabled of page1,2 and 3, I find that only the animation in page3 works, while the tab in page1 and page2 become invalid.

I have the same problem. Set react-navigation option  swipeEnable=true, can fix it

This is my fixed method
add a state in constructor
constructor(props) {
       super(props)
       this.state={
             showSwiper:false
       }
}

and then
componentDidMount() {
        setTimeout(() => {
            this.setState({
                showSwiper:true
            })
            //fix the image no show or can drag  in android
        }, 0)
    }

At last
render() {

        if (this.state.showSwiper) {
            return (
                    <Swiper
                        autoplay={true}
                        autoPlatTimeout={1}
                        loop={true}
                        style={styles.wrapper} showsButtons>
                        <View style={styles.slide1}>
                            <Text style={styles.text}>Hello Swiper</Text>
                        </View>
                        <View style={styles.slide2}>
                            <Text style={styles.text}>Beautiful</Text>
                        </View>
                        <View style={styles.slide3}>
                            <Text style={styles.text}>And simple</Text>
                        </View>
                    </Swiper>
            )
        } else {
            return null

        }

    }

@huangsongyan it did not work at all

Mine is  that the Swipper could only play automatically. I removed the "width" in the style of "Swipper",leaving only the "height" of the style. Then, it working well

@jialiujia That's because the swiper acturely is a scrollview in row. That means it's width should be like the amount of your swipers multiply your device's width.


Can't "Add Another Item" on Our Team page

Current Behavior
If you click "Add Another Item" on the "our Team" page, the page layout breaks and images cannot be referenced.
Desired Behavior
Adding another item does not cause the page to break...
Steps to Replicate

Go to: https://thor.dosomething.org/us/node/556/edit
Scroll to the bottom and click the "Add another item" button
Watch layout break.

Why This Matters
Sam can't add new staff photos until this is fixed.
Relevant Screenshots + Links
https://thor.dosomething.org/us/node/556/edit

Click "add another item" and then this happens:

This doesn't seem to be a problem on static content pages like "Our Partners" https://thor.dosomething.org/us/node/532/edit

Anddddd Luke made a video: https://trello.com/c/mhxEsXYc/1528-drupal-bug-adding-new-item-to-gallery-stops-page-functions

@namimody this actually does allow you to add new things, not sure what's wrong with the admin theme when you add a new gallery item, but if you save without an image and go back it works fine
before a save, you can still insert content, the theme just looks a little strange

this is after a save, and you can add an image

Hi @angaither -- wait so your'e saying that a new item can't be added with both an image and content at once, but can be added piecemeal?

@namimody yeah, you could technically paste in the correct image thing, but it's easier to save and wait for the autocomplete.
I don't see any errors or js or otherwise when it happens, I'm not sure it's worth investing a lot of time into fixing.

Hmm @angaither trying to understand -- when you say you don't see any errors in Js or anywhere else you can't see what's going wrong in the code, so fixing will be really difficult?
(By the way, is there any thing different about this page vs. other static content pages? Curious why we're not seeing this happen more places...)

when you say you don't see any errors in Js or anywhere else you can't see what's going wrong in the code, so fixing will be really difficult?
correct

(By the way, is there any thing different about this page vs. other static content pages? Curious why we're not seeing this happen more places...)
there's just the most items on this page

Alright thanks @angaither. I'll close and let Sam know.


Peer connection fails when LND is still starting up and tries to create offer

INFO:squeaknode.network.peer:Received msg msg_getdata(inv=[CInv(type=SecretKey hash=dd8b7b5ec80884cec50b7dc558a26b98dc9dc4fffe5cdc75b4df7417a5282131)]) from Peer(PeerAddress(host='<IP_ADDRESS>', port=42404))
INFO:squeaknode.network.peer:Stopping peer socket: <socket.socket fd=25, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('<IP_ADDRESS>', 18555), raddr=('<IP_ADDRESS>', 42404)>
INFO:squeaknode.network.connection_manager:Current number of peers 0
INFO:squeaknode.network.connection_manager:Current peers:--------
INFO:squeaknode.network.connection_manager:--------------
INFO:squeaknode.network.peer:Stopping peer socket: <socket.socket fd=25, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('<IP_ADDRESS>', 18555), raddr=('<IP_ADDRESS>', 42404)>
Exception in thread Thread-48:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/venv/lib/python3.8/site-packages/squeaknode-0.1.153-py3.8.egg/squeaknode/network/peer_handler.py", line 87, in start_connection
    connection.handle_connection()
  File "/opt/venv/lib/python3.8/site-packages/squeaknode-0.1.153-py3.8.egg/squeaknode/network/connection.py", line 100, in handle_connection
    self.handle_msgs()
  File "/opt/venv/lib/python3.8/site-packages/squeaknode-0.1.153-py3.8.egg/squeaknode/network/connection.py", line 137, in handle_msgs
    self.handle_peer_message(msg)
  File "/opt/venv/lib/python3.8/site-packages/squeaknode-0.1.153-py3.8.egg/squeaknode/network/connection.py", line 158, in handle_peer_message
    self.handle_getdata(msg)
  File "/opt/venv/lib/python3.8/site-packages/squeaknode-0.1.153-py3.8.egg/squeaknode/network/connection.py", line 215, in handle_getdata
    reply_msg = self._get_inv_reply(inv)
  File "/opt/venv/lib/python3.8/site-packages/squeaknode-0.1.153-py3.8.egg/squeaknode/network/connection.py", line 298, in _get_inv_reply
    return self._get_inv_reply_for_secret_key(inv)
  File "/opt/venv/lib/python3.8/site-packages/squeaknode-0.1.153-py3.8.egg/squeaknode/network/connection.py", line 306, in _get_inv_reply_for_secret_key
    resp = self.squeak_controller.get_offer_or_secret_key(
  File "/opt/venv/lib/python3.8/site-packages/squeaknode-0.1.153-py3.8.egg/squeaknode/node/squeak_controller.py", line 186, in get_offer_or_secret_key
    return self.get_offer(
  File "/opt/venv/lib/python3.8/site-packages/squeaknode-0.1.153-py3.8.egg/squeaknode/node/squeak_controller.py", line 206, in get_offer
    return self.squeak_core.package_offer(
  File "/opt/venv/lib/python3.8/site-packages/squeaknode-0.1.153-py3.8.egg/squeaknode/core/squeak_core.py", line 231, in package_offer
    lnd_external_address = self.get_lnd_external_address()
  File "/opt/venv/lib/python3.8/site-packages/squeaknode-0.1.153-py3.8.egg/squeaknode/core/squeak_core.py", line 208, in get_lnd_external_address
    info = self.lightning_client.get_info()
  File "/opt/venv/lib/python3.8/site-packages/squeaknode-0.1.153-py3.8.egg/squeaknode/lightning/lnd_lightning_client.py", line 142, in get_info
    return self.stub.GetInfo(
  File "/opt/venv/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/opt/venv/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.UNKNOWN
	details = "the RPC server is in the process of starting up, but not yet ready to accept calls"
	debug_error_string = "{"created":"@1632346278.080537452","description":"Error received from peer ipv4:<IP_ADDRESS>:10009","file":"src/core/lib/surface/call.cc","file_line":1070,"grpc_message":"the RPC server is in the process of starting up, but not yet ready to accept calls","grpc_status":2}"
>

Should fail gracefully.

https://github.com/yzernik/squeaknode/pull/1389


Add reminder to vote

Closes #72. Adds a reminder to vote onto the "Now Chillin'" message which plays before every song.

I just realized that this does not include a link to the form as #72 initially suggested. To do that we would most likely have to pass this in while busting, or wait until #12 is complete. We could also add a !formlink command or something which we can run in any channel during the bust, setting an internal form_link variable, and have the "chillin" message check to see if this is None or not each time. That way we don't have to remember to pass it in on !bust, which I could def see myself forgetting.
Or we could ignore that part for now.


Fix CapturedCallable to work better with kwargs

Description
#114 uncovered a bug with CapturedCallable. In short, this would work ok:
function = lambda **kwargs: kwargs["a"]

CapturedCallable(function)(a=2) # a=2 only provided at runtime

And this would not:
CapturedCallable(function, a=2)() # a=2 is provided upfront

Now they should both work ok.
We have already had a couple of very small bugs that I've known about for a while, where the following sorts of functions do not work with CapturedCallable:

a function with variadic positional args like def function(*args)
a function with positional-only arguments like def function(a, /)

Fixing the new **kwargs bug, which is an important case, makes it much harder to fix these much less important cases, and so I've decided to just not support them at all. There's now an explicit check to make sure no one tries to use CapturedCallable with those sorts of functions (very unlikely they'd want to do so anyway).
Tests have been updated to be much more thorough so that we now test every case on all 3 sorts of valid function positional_or_keyword_function, keyword_only_function, var_keyword_function.
Thanks to @maxschulz-COL for spotting the bug and for his initial fix which is the basis for this!
Checklist

[x] I have not referenced individuals, products or companies in any commits, directly or indirectly
[x] I have not added data or restricted code in any commits, directly or indirectly
[x] I have updated the docstring of any public function/class/model changed
[x] I have added the PR number to the change description in the changelog fragment, e.g. Enable feature XXX ([#1](https://github.com/mckinsey/vizro/pull/1)) (if applicable)
[x] I have added tests to cover my changes (if applicable)

Types of changes

[ ] Docs/refactoring (non-breaking change which improves codebase)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Notice

[x] I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":

I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.
I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorized to submit this contribution on behalf of the original creator(s) or their licensees.
I certify that the use of this contribution as authorized by the Apache 2.0 license does not violate the intellectual property rights of anyone else.

Thanks for this, looks great! 🚀 Was mostly reading through this out of interest, the tests really helped understanding it. Is it worth adding a note to the docs for the custom charts/actions or do you think the error messages suffice to provide guidance? Either way is fine for me 👍

I think no need for anything in the narrative docs, since a user is so unlikely to ever try to do CapturedCallable on a function where it doesn't work that having any note on this is more likely to just confuse things. I will make a note of it in the docstring though!


Bug:

It does not work. Nothing happens when i start a game

@GamingDuck121 , what website are you trying to use it on?


Quarkus client won't connect to configured Artemis server

We get this error trying to connect to our embedded Artemis server in a Quarkus app.  It worked fine until 2.16.6 and we deleted our artemis working directory.
ERROR: Failed to start application (with profile [prod])
java.lang.IllegalStateException: Configuration : url is not set and devservices is activated. This is a bug. Please report it.
at io.quarkus.artemis.core.runtime.ArtemisUtil.validateIntegrity(ArtemisUtil.java:30)
at io.quarkus.artemis.jms.runtime.ArtemisJmsRecorder.getConnectionFactoryProducer(ArtemisJmsRecorder.java:33)
at io.quarkus.deployment.steps.ArtemisJmsProcessor$configure1492597324.deploy_0(Unknown Source)
at io.quarkus.deployment.steps.ArtemisJmsProcessor$configure1492597324.deploy(Unknown Source)
at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
at io.quarkus.runtime.Application.start(Application.java:101)
application.yml is:
quarkus:
devservices:
enabled: false
artemis:
url: vm://0
The ArtemisJmsRecorder completely ignores the above configuration, it thinks the values are the opposite of the file.
Here are some of our dependencies:
[INFO] +- io.quarkiverse.artemis:quarkus-artemis-jms:jar:2.1.1:compile
[INFO] |  +- io.quarkiverse.artemis:quarkus-artemis-core:jar:2.1.1:compile
[INFO] |  - org.apache.activemq:artemis-jms-client:jar:2.27.1:compile
[INFO] |     - jakarta.jms:jakarta.jms-api:jar:2.0.3:compile
INFO] +- org.apache.activemq:artemis-server:jar:2.27.1:compile
[INFO] |  +- org.apache.activemq:artemis-commons:jar:2.27.1:compile
[INFO] |  +- org.apache.activemq:artemis-selector:jar:2.27.1:compile
[INFO] |  +- org.apache.activemq:artemis-journal:jar:2.27.1:compile
[INFO] |  +- org.apache.activemq:artemis-core-client:jar:2.27.1:compile
[INFO] |  |  +- org.jgroups:jgroups:jar:5.2.0.Final:compile
[INFO] |  |  +- io.netty:netty-handler-proxy:jar:4.1.86.Final:compile
[INFO] |  |  - io.netty:netty-codec-socks:jar:4.1.86.Final:compile
[INFO] |  +- org.apache.activemq:artemis-quorum-api:jar:2.27.1:compile
[INFO] |  +- org.apache.activemq:activemq-artemis-native:jar:2.0.0:compile
[INFO] |  +- org.jctools:jctools-core:jar:2.1.2:compile
[INFO] |  +- io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.86.Final:compile
[INFO] |  +- io.netty:netty-transport-native-kqueue:jar:osx-x86_64:4.1.86.Final:compile
[INFO] |  |  - io.netty:netty-transport-classes-kqueue:jar:4.1.86.Final:compile
[INFO] |  +- org.apache.commons:commons-configuration2:jar:2.8.0:compile
[INFO] |  - io.micrometer:micrometer-core:jar:1.10.2:compile
[INFO] |     +- io.micrometer:micrometer-commons:jar:1.10.2:compile
[INFO] |     - io.micrometer:micrometer-observation:jar:1.10.2:compile
[INFO] +- org.apache.activemq:artemis-amqp-protocol:jar:2.27.1:compile
[INFO] |  - org.apache.qpid:proton-j:jar:0.34.0:compile

@dhoffer Thanks for raising an issue. Can you provide a reproducer?

this could be related to this issue (quarkusio.zulipchat.com) which was fixed by commit b1ecd0b065bdc0f783b55581ea872aee38152db3 in quarkus. It seems that his fix hasn't made it down to any 2.x branch:
git branch -r --contains "b1ecd0b065bdc0f783b55581ea872aee38152db3" | cat
  fork/add-jms-spi-to-bom
  fork/feature/cleanup-codestarter-tests
  fork/feature/improve-csrf-documentation
  fork/feature/quiet-docker-command-for-determining-java-version
  fork/foo
  fork/main
  origin/3.0
  origin/3.1
  origin/3.2
  origin/HEAD -> origin/main
  origin/dependabot/maven/com.google.errorprone-error_prone_annotations-2.20.0
  origin/dependabot/maven/flyway.version-9.20.0
  origin/dependabot/maven/io.opentelemetry-opentelemetry-bom-1.27.0
  origin/dependabot/maven/kafka3.version-3.4.1
  origin/dependabot/maven/kafka3.version-3.5.0
  origin/dependabot/maven/org.checkerframework-checker-qual-3.35.0
  origin/dmlloyd-patch-1
  origin/main
  origin/revert-34339-rr-instance

Could you try prefixing the property with the dev-profile, i.e.:
quarkus:
  devservices:
    enabled: false
  artemis:
    url: vm://0
"%dev":
  quarkus
    artemis:
      url: vm://0

and see if this fixes the issue?
I'll see if i can get a cherry-pick of the commit to 2.13.x and 2.16.x.

Backport-PRs:

2.13: https://github.com/quarkusio/quarkus/pull/34495
2.16: https://github.com/quarkusio/quarkus/pull/34496

I was able to reproduce the issue. See https://github.com/quarkusio/quarkus/pull/34495#issuecomment-1619123628
The problem is only partially on our end, we cannot fix it. Ideally, the beforementioned cherry-picks are merged; this should resolve the issue.


fix: update chevrotain

Why
Came across this issue while developing a new nextjs project. It seems like the bundler does't find a default export from api.js. Chevrotain 11 is pure ESM, so there are only named exports, which means it shouldn't look for a default export at all.
./node_modules/chevrotain/lib_esm/api_esm.mjs
Attempted import error: '../lib/src/api.js' does not contain a default export (imported as 'mod').

Import trace for requested module:
./node_modules/chevrotain/lib_esm/api_esm.mjs
./node_modules/three-stdlib/loaders/VRMLLoader.js
./node_modules/three-stdlib/index.js
<EMAIL_ADDRESS><EMAIL_ADDRESS>./components/Graphic.tsx

What
Updated Chevrotain version to 11
Checklist

[ ] Documentation updated
[ ] Storybook entry added
[x] Ready to be merged

Chevrotain being ESM-only means that we have to bundle it ourselves for CJS. I can update the build config to do so.

@CodyJasonBennett got it - thanks


data-goto not functioning in Firefox 34, 35

I don't see any issues-section in the new repo, so I guess my best option is to post this here:
I've tried several demo's, from the site and ones I've built myself or built by collegues. Recent Firefox versions don't seem to do any navigation done with data-goto.
The section on platforms in docs doesn't list anything related to Firefox. Perhaps it should mention that compatibility with Firefox is limited. If this isn't fixed, I think the majority of all ChUI apps won't work properly on Firefox.

ChocolateChip-UI targets iOS, Android and Windows Phone 8. The defaults on these platforms are mobile Safari, mobile Chrome and mobile IE. To enable development on the desktop, we also support desktop Safari for iOS, desktop Chrome for Android, and desktop IE 10-11 for Windows Phone 8. Both the iOS and Android versions depend on Webkit specific features which are not present in Firefox and therefore there will be issues. In the case of Windows Phone 8, it is possible to run that in desktop Firefox, however, if you are doing that on a Mac, some icons will not display properly because ChocolateChip-UI for Windows Phone 8 uses the same internal icon resources as Windows 8 and Windows Phone 8. Otherwise, the WP8 version should be running without problem in Firefox.
The sad fact of mobile development is that depending on what platform you wish to target, you will need an appropriate desktop operating system: Mac for iOS and Windows for Windows Phone 8. Trying to simulate all the differences between all the browsers and operating system to be the same in all browsers is simply not possible. We therefore restrain the baseline for developing for iOS to Safari, for Android to Chrome and for Windows Phone 8 to Windows 8.

By the way, our bad. We missed the check box for Issue tracking on the repos. They now have it.

Thank you for your responses and suggestions.
Robert: we might try running the WP8-version, I must discuss that with our team.

I'm curious to know why you are interested in Firefox support. Are you thinking of targeting Firefox OS?

We're searching for a way to build an app that supports the recent versions of all major browsers, including Firefox on the desktop on both Windows and OS X. Although we wish to support desktop browsers, the app's main focus is mobile, so that is why we ended up exploring ChocolateChip-UI.
I think I expected some of the things not to work on Firefox since the support for other desktop browsers isn't that bad, and Firefox does render fine in our experiment. But the whole data-goto thing makes the apps break hard :-)

Martijntraa, data-goto works fine for me using Firefox on both Mac and Windows, any of the themes. I'm curious why this isn't working for you. Are you having this issue with the examples or is this a file you've put together? If it's your own code, would you mind sending me some code snippets to see if there's some minor error.

By the way, I'm currently working on a version of the themes that will play nice with Firefox. I'm ditching the Webkit image masks for a different approach that will work with Firefox.


some tracks for improvement!

Actually, I just fixed the other issue in place.
In fact, it was just a b tag that was added to HTML so I had to change the indexes here str(h2)[5:11] from 5:11 to 8:14 to extract the correct total number of movies.
I think this tool does nothing but download the .torrent file, not the movie and this is not quite practical.
I didn't expect that!
This tool should use some torrent client with (magnet link constructed from data returned from YTS API or the downloaded torrent).
I suggest using transmission as it's open-source and it has a nice command line!

Thanks a lot for your suggestion! I agree the title might be misleading but I had created it just to put some "good" use to my Python knowledge (which was very limited at that time, and so is now if I'm being honest) by creating a small CLI for me to use personally instead of having to visit the website.
Coming to your solution, although I've used bs4 in my implementation, I saw yesterday that the API supports query_term as a parameter which would effectively render bs4 implementation useless. I'll update the code for torrent file and make it at least usable.
Regarding the torrent client part, I'll have a look into it and get back to you. Sorry for the delay I might have caused.
Thanks for taking the time and filing such helpful issues and giving your suggestions.

@flashblaze sorry for the title I forgot to edit it, I was editing the issue because I was thinking the app didn't work properly...
and yeah the API was improved so I Imagine much of the code is useless by now, about the CLI, I looked into it.
And I experience a little bit with transmission-remote, but there is a problem with download it's very low I don't know, but I will investigate it further to see what is the problem if it's ok we will have a full solution :)
Of course, the problem now is with the exact name of the movie that I have to type it correctly otherwise "Title Is Not Found".
We have to make it much smarter by using Regex and give suggestion and support of autocomplete :)
Many stuffs will come...


Added clearing of old note entries

1. Why is this change necessary?
There are currently many old expired notes, which cant be used anymore
2. What does this change do, exactly?
Removes expired notes
3. Describe each step to reproduce the issue or behaviour.
4. Please link to the relevant issues (if any).
https://issues.shopware.com/issues/SW-21514
5. Which documentation changes (if any) need to be made because of this PR?
6. Checklist

[ ] I have written tests and verified that they fail without my change
[ ] I have squashed any insignificant commits
[ ] This change has comments for package types, values, functions, and non-obvious lines of code
[X] I have read the contribution requirements and fulfil them.

Please run php-cs-fixer with the actual settings :-)

Hey @shyim,
please squash your commits.

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/SW-21650
Please use this issue to track the state of your pull request.

CLA already signed.

Thanks once again, @shyim! 💙


window.XSLTProcessor not found as expected

<EMAIL_ADDRESS>commented:

Which version of PhantomJS are you using?
1.7.0 (also recreated in 1.6.1)
What steps will reproduce the problem?

Open a page

Call "page.evaluate(function () {var xslProc = new XSLTProcessor();});

What is the expected output?
A new XSLTProcessor instance should be created, as per: http://www.w3schools.com/xsl/xsl_client.asp
What do you see instead?
Error message "Can't find variable: XSLTProcessor"
Which operating system are you using?
Windows 7 (Also tested on Ubuntu 10.04)
Did you use binary PhantomJS or did you compile it from source?
Binary
Please provide any additional information below.

Attached is a simple script which reproduces the error.
I found XSLTProcessor referenced in the webkit source in the phantomjs github:
https://github.com/ariya/phantomjs/blob/master/src/qt/src/3rdparty/webkit/Source/WebCore/xml/XSLTProcessor.cpp
The Webkit XSLT documentation mentions that it uses the libxslt library:
http://www.webkit.org/projects/xslt/index.html

Disclaimer:
This issue was migrated on 2013-03-15 from the project's former issue tracker on Google Code, Issue #917.
:star2:   2 people had starred this issue at the time of migration.

Has the status of this changed for PhantomJS 2.X, @ariya or @Vitallium?

@bedney Unfortunately no.

@Vitallium: Darn - but thanks for your reply. And thank you for all of your hard work on PhantomJS 2!


illegal callback type invocation from native module. This callback type only permits a single invocation from native code

I developpe a react native application using sqlite.
My Android.node.js :

var databaseName = 'react.db'
//initialiser base de donnée
 sqlite.init(databaseName)
     .then((_) => {
         console.log('database initialized.')
     }
 )
//declarer une requette
var sql = 'INSERT INTO principale(id, nom) VALUES (1, bonjour)'
 var params = ["Create react native android sqlite", 1]
//fermer la base de donnée

export default class sql2 extends Component {
  //function
  insert() {

sqlite.query(sql, params)
 .then((data) => {
     console.log('retrieved: ', data)
 }
)
     }
  render() {
    return (
      <View style={styles.container}>

        <Button title='salut' onPress={this.insert}>Cliquz sur moi</Button>

      </View>
    );
  }
}

I have this error : illegal callback type invocation from native module. This callback type only permits a single invocation from native code

Getting this same error when using a custom NativeModule. But only on the android side.
My iOS method of the same name..
RCT_EXPORT_METHOD(sendGroupMsg: (NSDictionary *)params errorCallback:(RCTResponseSenderBlock)failureCallback)

And my failing Android method
public void sendGroupMsg(ReadableMap params, Callback errorCallback) {

Assuming that because they're identical method setups, there shouldn't be an issue..

Facebook anyone?

Hi there! This issue is being closed because it has been inactive for a while. Maybe the issue has been fixed in a recent release, or perhaps it is not affecting a lot of people. Either way, we're automatically closing issues after a period of inactivity. Please do not take it personally!
If you think this issue should definitely remain open, please let us know. The following information is helpful when it comes to determining if the issue should be re-opened:

Does the issue still reproduce on the latest release candidate? Post a comment with the version you tested.
If so, is there any information missing from the bug report? Post a comment with all the information required by the issue template.
Is there a pull request that addresses this issue? Post a comment with the PR number so we can follow up.

If you would like to work on a patch to fix the issue, contributions are very welcome! Read through the contribution guide, and feel free to hop into #react-native if you need help planning your contribution.


Add simple sample for service invocation

Description
Add a sample which performs stateless service invocation which shows a simple example of how Dapr can be used with any statestore or docker dependency.
Issue reference
We strive to have all PRs being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #247
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[X] The sample code compiles correctly
[X] You've tested new builds of the sample if you changed sample code
[X] You've updated the sample's README if necessary

Should this actually be "sample 1"?  The number should be lower since simple or "slim" setups are typically first.

Should this actually be "sample 1"? The number should be lower since simple or "slim" setups are typically first.

Hi Leon, Both the hello-world and the hello-kubernetes samples are straightforward and easy to use and modify. This sample though is a bit more involved in the setup and use, that is why I put it as sample 11.

Should this actually be "sample 1"? The number should be lower since simple or "slim" setups are typically first.

Hi Leon, Both the hello-world and the hello-kubernetes samples are straightforward and easy to use and modify. This sample though is a bit more involved in the setup and use, that is why I put it as sample 11.

We'll be removing the numbering from samples in the near future so for now it is fine


Alternative implementation for JSMA

I know there is currently another open pull request for the JSMA, but I just want to share the work I have done and hopefully it can help too.
My motivation for the change is to improve the efficiency to generate adversarial examples for JSMA. With the change, the time needed for generation should be greatly reduced. I haven't done a precise measurement, but it roughly takes about 15 sec to generate 500 adversarial samples with GPU.
The change tries to keeps the same interface of the original implementation, so that no modification is needed for tutorial.
I have run the tests in the test_tf and the tutorial. Some of the tests in test_tf is failing, but I think it fails even without my change. I also tried to fix the "test_generate_np_targeted_gives_adversarial_example" test, but I can't. Maybe someone can comment on what the test is trying to do.
Any comments and suggestions are appreciated.

I signed it!

I accidentally submitted the change with a wrong email. Closing this one and will create another one.


TickTick not working (Ferdi ToDo's)

Bug description:
TickTick will not load. It sometimes will load after the initial setup (fresh install), but won't load at any point afterwards. This has been happening for months for me on multiple builds and computers, so figured I would make a ticket. Other To Do lists load fine
Steps to reproduce:

Go to File > Settings > General Tab
Change Todo Server to TickTick (or use the custom web addres)
TickTick gets stuck on a loading screen.

Expected behavior
TickTick should load.
Screenshots

Environment:

Operating System: Windows 10

Ferdi Version: 5.6 - Nightly 15

Server: Franz (I think)

Debug information:

I'm facing the same issue. Already tried clearing cache and reloading Ferdi and ToDos. Did you figure out any way to fix it @EgoDogg ?

I'm facing the same issue. I already tried clearing cache and reloading Ferdi and ToDos. Did you figure out any way to fix it @EgoDogg ?

No luck in fixing it. It's been an issue for months and through various releases, betas, and nightly updates. If there is a manual fix that can be made, it's outside my tech-knowhow.

I have the same issue. It works with the 5.5. version though.

anyone knows how to solve this issue?

anyone knows how to solve this issue?

Not that I know of. For the time being I have Tick Tick as a service, but that's not the most ideal option. I'd like to use the Todos sidebar since everything else works. lol

@vraravam  any clue please?

This seems to be cause by the fact that we don't expose the Chromium version of the User-Agent being used in the todos webview. Exposing the Chromium version causes Google Tasks to reject the login, because Google is hostile to "non-official" (read: not Chrome, Safari, or maybe Firefox) browsers. However, TickTick expects the Chrome version number and fails to load without it.
I'll move the Google login logic over from the service webview to fix fix, and selectively suppress the Chrome version number only for the Google login.

Thank you thank you thank you!


Fix release URL in webhooks

Resolves #27180
URL points to the API URL, HTMLURL points to the web page.
Notably, however, for PRs they are the same URL. I switched them to use HTMLURL to match the rest of the codebase terminology.

I was unable to create a backport for 1.21. @jolheiser, please send one manually. :tea:
go run ./contrib/backport 27182
...  // fix git conflicts if any
go run ./contrib/backport --continue


CudnnLSTM variable sequence length sometimes fails with CUDNN_STATUS_EXECUTION_FAILED

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Debian/Sid (2020-07-01), Ubuntu 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
TensorFlow installed from (source or binary): source and binary
TensorFlow version (use command below): 1.15
Python version: 3.6, 3.7.8
Bazel version (if compiling from source): 0.26.1
GCC/Compiler version (if compiling from source): 9.0
CUDA/cuDNN version: 10.0/7.4.1 ; 10.0/<IP_ADDRESS> ; 10.0/<IP_ADDRESS> ; 10.0/<IP_ADDRESS>
GPU model and memory: 2x RTX 2080 Ti ; 4x GTX 1080 Ti ;

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with:

TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)": v1.15.3-0-g4386a6640c

Describe the current behavior
Training with some dataset triggers:
2020-07-22 16:15:42.108252: E tensorflow/stream_executor/dnn.cc:588] CUDNN_STATUS_EXECUTION_FAILED
in tensorflow/stream_executor/cuda/cuda_dnn.cc(1778): 'cudnnRNNForwardTrainingEx( cudnn.handle(), rnn_desc.handle(), input_desc.data_handle(), input_data.opaque(), input_h_desc.handle(), input_h_data.opaque(), input_c_desc.handle(), input_c_data.opaque(), rnn_desc.params_handle(), params.opaque(), output_desc.data_handle(), output_data->opaque(), output_h_desc.handle(), output_h_data->opaque(), output_c_de
sc.handle(), output_c_data->opaque(), nullptr, nullptr, nullptr, nullptr, nullptr, nullptr, nullptr, nullptr, workspace.opaque(), workspace.size(), reserve_space.opaque(), reserve_space.size())'
2020-07-22 16:15:42.108385: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at cudnn_rnn_ops.cc:1527 : Internal: Failed to call ThenRnnForward with model config: [rnn_mode, rnn_input_mode, rnn_direction_mode]: 2, 0, 0 , [num_layers, input_size, num_units, dir_count, max_seq_length, batch_size, cell_num_units]: [1, 2048, 2048, 1, 75, 2, 2048]

Describe the expected behavior
Training should succeed, or TensorFlow or CUDNN should expose a more actionable error
Standalone code to reproduce the issue
Will be provided after.
Other info / logs
Will be provided after. Some noisy debugging session can be seen at https://github.com/mozilla/DeepSpeech/issues/3088

Repro material.
To repro with r1.15 tensorflow on current Debian Sid, I needed to use pyenv to have a local 3.7 python (latest version supported by r1.15), but if you run it on a system-provided 3.7, you'd prefer remove those line and just setup a virtualenv.
STR:

Create ~/tmp/issue3308
Placed this script in ~/tmp/issue3088/

$ cat run_local.sh
#!/bin/bash

set -xe

COMMAND_PREFIX=$1
echo $COMMAND_PREFIX

rm -fr ~/.local/share/deepspeech/

export PYENV_VERSION=3.7.8
export PYENV_ROOT=$HOME/pyenv/
export PATH=$PYENV_ROOT/bin/:$PATH

eval "$(pyenv init -)"

pip --version

for p in tensorboard tensorflow tensorflow-estimator tensorflow-gpu-local;
do
    pip list $p && pip uninstall --yes $p
done;

cd ds/ && \
        pip3 install --upgrade pip==20.0.2 wheel==0.34.2 setuptools==46.1.3 && \
        DS_NOTENSORFLOW=y pip3 install --upgrade -e . && \
cd ..

#pip install --upgrade /home/alexandre/tmp/issue3088/wheel_dst/tensorflow_gpu_local-1.15.0-cp37-cp37m-linux_x86_64.whl
pip install --upgrade /home/alexandre/tmp/issue3088/wheel_dst/tensorflow_gpu_local-*-cp37-cp37m-linux_x86_64.whl

### 1.13
#patch -p1 -d $PYENV_ROOT/versions/$PYENV_VERSION < disable_tensorflow_contrib_cloud.patch
# tpu hack
#pip install --upgrade tensorflow-estimator==1.14.0

### after
#patch -p1 -d $PYENV_ROOT/versions/$PYENV_VERSION < disable_tensorflow_core_contrib_cloud.patch

nvidia-smi

python --version

export LD_LIBRARY_PATH=/home/alexandre/Documents/codaz/Mozilla/DeepSpeech/CUDA-10.0/lib64/

#TF_CPP_MIN_VLOG_LEVEL=1
python -c 'import tensorflow as tf; tf.test.is_gpu_available()'

$COMMAND_PREFIX $PYENV_ROOT/versions/$PYENV_VERSION/bin/python ds/DeepSpeech.py \
        --show_progressbar true \
        --log_level 0 \
        --train_cudnn true \
        --alphabet_config_path ds/data/alphabet.txt \
        --scorer "" \
        --train_files csvs/train_debug_mini_As_Bs_Cs_local.csv \
        --train_batch_size 2 \
        --n_hidden 2048 \
        --audio_sample_rate 16000 \
        --epochs 3 \
        --learning_rate 0.0001 \
        --dropout_rate 0.4 \
        --lm_alpha 0.75 \
        --lm_beta 1.85

git clone https://github.com/mozilla/DeepSpeech/ ds/ (you might also need git-lfs installed prior)
Extract attached zip file for repro dataset

The zip file also features some Dockerfile, if it helps for repro.
tensorflow_issue_41630.zip

More infos / logs
Failure log is (with some extra debug by myself):
D Session opened.
I Could not find best validating checkpoint.
I Could not find most recent checkpoint.
I Initializing all variables.
2020-07-22 16:06:28.419358: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
I STARTING Optimization
Epoch 0 |   Training | Elapsed Time: 0:00:00 | Steps: 0 | Loss: 0.000000                                                                                2
020-07-22 16:06:29.819427: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
generate_values <deepspeech_training.util.sample_collections.CSV object at 0x7f497418e610>
yield generate_values 0 csvs/../data/A/163_5029_3498779ce37873475394654801cc3888-8fddd9522baf442463171802a7e57489.wav <deepspeech_training.util.sample_collections.LabeledSample object at 0x7f4d1710a8d0>
yield generate_values 1 csvs/../data/A/155_4757_9bc6d6f754547a09bbcf70e42d8e2a27-b112945da6818223ab8e1daf80313a62.wav <deepspeech_training.util.sample_collections.LabeledSample object at 0x7f4d1710a850>
yield generate_values 2 csvs/../data/B/98_2923_a387275540ba5f2159c37eaee3e4e9a0-651926517a6241fd9bb5942777b1f0ff.wav <deepspeech_training.util.sample_collections.LabeledSample object at 0x7f4d1710ab10>
batch_fn <_VariantDataset shapes: <unknown>, types: tf.string> 2 <_VariantDataset shapes: (?, 26), types: tf.float32> <_VariantDataset shapes: (), types: tf.int32>
yield generate_values 3 csvs/../data/B/154_4738_2f841fb1af523c579414e0358ab16295-6aea9aa95b1bdbfd80703754cd8a180c.wav <deepspeech_training.util.sample_collections.LabeledSample object at 0x7f4d1710a9d0>
yield generate_values 4 csvs/../data/C/175_5429_67ed7914b9a3bac4e46dd42a5721a95f-e31a33c85ca8249476596c1ff7fc2f67.wav <deepspeech_training.util.sample_collections.LabeledSample object at 0x7f4d1710a910>
yield generate_values 5 csvs/../data/C/169_5271_3210ac3e97626f9c1515cb019e5fa36e-dd839274af12610f137398ddd01f85f8.wav <deepspeech_training.util.sample_collections.LabeledSample object at 0x7f4d1710abd0>
batch_fn <_VariantDataset shapes: <unknown>, types: tf.string> 2 <_VariantDataset shapes: (?, 26), types: tf.float32> <_VariantDataset shapes: (), types: tf.int32>
CudnnRNNForwardOp
ShouldUsePaddedIO time_major=1
ShouldUsePaddedIO seq_array[0]=74
ShouldUsePaddedIO seq_array[1]=74
ShouldUsePaddedIO [0]: seq_array[i]=74
ShouldUsePaddedIO [0]: model_shapes.max_seq_length=74
ShouldUsePaddedIO [1]: seq_array[i]=74
ShouldUsePaddedIO [1]: model_shapes.max_seq_length=74
ShouldUsePaddedIO rv=false all_max_seq_length=true
files: ["csvs/../data/A/163_5029_3498779ce37873475394654801cc3888-8fddd9522baf442463171802a7e57489.wav" "csvs/../data/A/155_4757_9bc6d6f754547a09bbcf70e42d8e2a27-b112945da6818223ab8e1daf80313a62.wav"] [74 74] [[[-0.245490551 0.00717380643 0.10210821 ... 0.117108196 -0.0276376158 -0.126444399]
  [0.0757513046 -0.0512054712 -0.291382492 ... 0.243732437 -0.0566715039 0.0398223847]]

 [[-0.195820108 0.138200715 0.0146510527 ... 0.177044764 0.0705049634 -0.189360529]
  [-0.028521724 0.25693503 -0.152943641 ... 0.121622048 -0.342009 0.0648547485]]

 [[-0.22990115 0.229410842 0.0562512279 ... 0.0885351449 -0.210647494 -0.115016311]
  [-0.522053659 -0.152149498 -0.0655985773 ... 0.0328139216 -0.0536242127 0.0909850895]]

 ...

 [[0.125317723 -0.201877266 0.342403591 ... -0.461629033 -0.7424016 -0.634134829]
  [-0.495484978 0.0653357953 0.444910228 ... 0.356240422 0.116222136 -0.165624559]]

 [[-0.559318066 -0.360073239 0.386285067 ... 0.250025511 -0.437686384 -0.0856590122]
  [-0.429158121 -0.0336664394 -0.304231912 ... 0.120450795 -0.136865944 -0.0517320298]]

 [[-0.30379957 0.109998763 0.230347365 ... 0.213008478 -0.100003451 -0.244150743]
  [-0.19179742 -0.0341652408 0.393984377 ... 0.300181448 -0.380117238 -0.0264749527]]] 'SparseTensor(indices=[[0 0]
 [0 1]
 [0 2]
 ...
 [1 22]
 [1 23]
 [1 24]], values=[5 14 0 ... 8 9 10], shape=[2 27])'
CudnnRNNBackwardOp
ShouldUsePaddedIO time_major=1
ShouldUsePaddedIO seq_array[0]=74
ShouldUsePaddedIO seq_array[1]=74
ShouldUsePaddedIO [0]: seq_array[i]=74
ShouldUsePaddedIO [0]: model_shapes.max_seq_length=74
ShouldUsePaddedIO [1]: seq_array[i]=74
ShouldUsePaddedIO [1]: model_shapes.max_seq_length=74
ShouldUsePaddedIO rv=false all_max_seq_length=true
Epoch 0 |   Training | Elapsed Time: 0:00:03 | Steps: 1 | Loss: 188.119797                                                                                b
atch_fn <_VariantDataset shapes: <unknown>, types: tf.string> 2 <_VariantDataset shapes: (?, 26), types: tf.float32> <_VariantDataset shapes: (), types: tf.int32>
CudnnRNNForwardOp
ShouldUsePaddedIO time_major=1
ShouldUsePaddedIO seq_array[0]=74
ShouldUsePaddedIO seq_array[1]=75
ShouldUsePaddedIO [0]: seq_array[i]=74
ShouldUsePaddedIO [0]: model_shapes.max_seq_length=75
ShouldUsePaddedIO rv=true all_max_seq_length=false
2020-07-22 16:06:33.612395: E tensorflow/stream_executor/dnn.cc:588] CUDNN_STATUS_EXECUTION_FAILED
in tensorflow/stream_executor/cuda/cuda_dnn.cc(1778): 'cudnnRNNForwardTrainingEx( cudnn.handle(), rnn_desc.handle(), input_desc.data_handle(), input_data.opaque(), input_h_desc.handle(), input_h_data.opaque(), input_c_desc.handle(), input_c_data.opaque(), rnn_desc.params_handle(), params.opaque(), output_desc.data_handle(), output_data->opaque(), output_h_desc.handle(), output_h_data->opaque(), output_c_de
sc.handle(), output_c_data->opaque(), nullptr, nullptr, nullptr, nullptr, nullptr, nullptr, nullptr, nullptr, workspace.opaque(), workspace.size(), reserve_space.opaque(), reserve_space.size())'
2020-07-22 16:06:33.612438: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at cudnn_rnn_ops.cc:1527 : Internal: Failed to call ThenRnnForward with model config: [rnn_mode, rnn_input_mode, rnn_direction_mode]: 2, 0, 0 , [num_layers, input_size, num_units, dir_count, max_seq_length, batch_size, cell_num_units]: [1, 2048, 2048, 1, 75, 2, 2048]

We verified a few things:

fine with tensorflow r1.14
breaks on r1.15 with cudnn 7.6, 7.5, more intermittent with 7.4 (I got severel non repro, reporting contributor still hit the issue)
repro with several versions of the nvidia driver (currently using 440.100 packaged by debian sid, contributor using similar version tested down to 430.64: https://github.com/mozilla/DeepSpeech/issues/3088#issuecomment-656583170)
git bisected tensorflow several time to ensure the first commit really exposing the issue is https://github.com/tensorflow/tensorflow/pull/30889 that @kaixih pushed (confirmed by local revert and no repro at all over 25 runs)
forcing ShouldUsePaddedIO to return false would repro, forcing true would never repro in my case (not verified on big dataset from contributors)
tried changing the batch ordering when we feed / call ctc_loss but this would not help
a lot of other trial/experiments are to be found in the deepspeech bug report

Please do not hesitate to ping myself or @applied-machinelearning if you have troubles reproducing the issue, but with the provided material it should be straightforward.

@timshen91 can you PTAL?

Can you try to fetch the cudnn logs with the following env vars? And attach the somefile.log (If this is too large, we might only need the last part which contains the cudnnRNNForwardTrainingEx). @lissyx
export CUDNN_LOGINFO_DBG=1
export CUDNN_LOGDEST_DBG=somefile.log

More details: https://docs.nvidia.com/deeplearning/sdk/cudnn-developer-guide/index.html#api-logging

issue3088.log
I might not be able to react before tomorrow, it's getting later over there.

More details: https://docs.nvidia.com/deeplearning/sdk/cudnn-developer-guide/index.html#api-logging

I swear I search a long time for a CUDNN-related debug env variable but never found that :/

I! CuDNN (v7600) function cudnnRNNForwardTrainingEx() called:
...
i!         paddingMode: type=cudnnRNNPaddingMode_t; val=CUDNN_RNN_PADDED_IO_DISABLED (0);
i!         plan: type=cudnnPersistentRNNPlan_t; val=NULL_PTR;
i!     xDesc: type=cudnnRNNDataDescriptor_t:
i!         dataType: type=cudnnDataType_t; val=CUDNN_DATA_FLOAT (0);
i!         dimA: type=int; val=[75,2,2048];
i!         seqLengthArray: type=int; val=[74,75];
i!         layout: type=cudnnRNNDataLayout_t; val=CUDNN_RNN_DATA_LAYOUT_SEQ_MAJOR_UNPACKED (0);
i!         paddingFill: type=CUDNN_DATA_FLOAT; val=0.000000;
...

An initial investigation shows the padding mode is suspicious. We shouldn't see CUDNN_RNN_PADDED_IO_DISABLED here, which is incompatible with the CUDNN_RNN_DATA_LAYOUT_SEQ_MAJOR_UNPACKED below.
This case uses the variable sequence lengths [74, 75] and max seq length is 75. The time/seq major layout is used. From the logic here: https://github.com/tensorflow/tensorflow/blob/r1.15/tensorflow/core/kernels/cudnn_rnn_ops.cc#L944-L956, the use_padded_io should be set True and subsequently the CUDNN_RNN_PADDED_IO_ENABLED should have been used here https://github.com/tensorflow/tensorflow/blob/r1.15/tensorflow/stream_executor/cuda/cuda_dnn.cc#L1082. I checked how the use_padded_io gets passed in the TF codebase and it looks no issue to me.

Just one more thing: can you give a shot with TF_CUDNN_RESET_RND_GEN_STATE=1? The only thing that I suspect now is this branch might not be taken. https://github.com/tensorflow/tensorflow/blob/a621845bd0256981a092681eb80f6e2cac6bb65e/tensorflow/core/kernels/cudnn_rnn_ops.cc#L1085

Just one more thing: can you give a shot with TF_CUDNN_RESET_RND_GEN_STATE=1? The only thing that I suspect now is this branch might not be taken.
https://github.com/tensorflow/tensorflow/blob/a621845bd0256981a092681eb80f6e2cac6bb65e/tensorflow/core/kernels/cudnn_rnn_ops.cc#L1085

issue3088_reset_rnd_gen.log
Here is the log, I'll check more in-depth and verify this branch

Wait, I might have made a mistake when generating those logs and left a false being forced for use_padded_io (I should maybe not have done that being sleepy). I'm cross-checking and will provide newer logs.

Here is the log, with debug enabled, after ensuring ShouldUsePaddedIO does not hardcode a false:
issue3088_new_noResetRnnGenState.log
As you can see, this is still failing the same way.
Now, here are the logs with the same TensorFlow setup, and TF_CUDNN_RESET_RND_GEN_STATE=1
issue3088_new_withResetRnnGenState_0.log
issue3088_new_withResetRnnGenState_1.log
issue3088_new_withResetRnnGenState_2.log
issue3088_new_withResetRnnGenState_3.log
issue3088_new_withResetRnnGenState_4.log
As you can see, no repro at all now.
This does makes me wonder if we might not be in trouble because of how we use CudnnLSTM: https://github.com/mozilla/DeepSpeech/blob/9e023660ef1c0947b0f8c8db54b5cc9174c7076a/training/deepspeech_training/train.py#L108-L132
Context, as much as I can remember (I'm not the one who wrote that) is that we needed to handle variables properly for being able to reload checkpoint on non-CuDNN setup. Maybe @reuben can elaborate if that's required.

This does makes me wonder if we might not be in trouble because of how we use CudnnLSTM: https://github.com/mozilla/DeepSpeech/blob/9e023660ef1c0947b0f8c8db54b5cc9174c7076a/training/deepspeech_training/train.py#L108-L132
Context, as much as I can remember (I'm not the one who wrote that) is that we needed to handle variables properly for being able to reload checkpoint on non-CuDNN setup. Maybe @reuben can elaborate if that's required.

So I verified, changed our code to avoid this like that:
diff --git a/training/deepspeech_training/train.py b/training/deepspeech_training/train.py
index 93d0c727..85c581ae 100644
--- a/training/deepspeech_training/train.py
+++ b/training/deepspeech_training/train.py
@@ -115,22 +115,19 @@ def rnn_impl_cudnn_rnn(x, seq_length, previous_state, _):
     # reuse=True to reuse variables, we can't easily make use of the object oriented
     # way CudnnLSTM is implemented, so we save a singleton instance in the function,
     # emulating a static function variable.
-    if not rnn_impl_cudnn_rnn.cell:
+    with tfv1.variable_scope('cudnn_lstm/rnn/multi_rnn_cell'):
         # Forward direction cell:
         fw_cell = tf.contrib.cudnn_rnn.CudnnLSTM(num_layers=1,
                                                  num_units=Config.n_cell_dim,
                                                  input_mode='linear_input',
                                                  direction='unidirectional',
                                                  dtype=tf.float32)
-        rnn_impl_cudnn_rnn.cell = fw_cell

-    output, output_state = rnn_impl_cudnn_rnn.cell(inputs=x,
-                                                   sequence_lengths=seq_length)
+        output, output_state = fw_cell(inputs=x,
+                                       sequence_lengths=seq_length)

     return output, output_state

-rnn_impl_cudnn_rnn.cell = None
-

 def rnn_impl_static_rnn(x, seq_length, previous_state, reuse):
     with tfv1.variable_scope('cudnn_lstm/rnn/multi_rnn_cell'):

Unfortunately, the issue does still repro.

So @kaixih what's your take with those new logs and the extra verification I performed on our code ? Don't hesitate if you need more logging, I'll do it as quickly as I can :)

So, for now two things can be verified:
(1) If the TF_CUDNN_RESET_RND_GEN_STATE=1, there is no repro, correct?
(2) If the TF_CUDNN_RESET_RND_GEN_STATE=0 (default setting), the use_padded_io is True, correct? If yes, can you track it and see if this branch is taken in your training: https://github.com/tensorflow/tensorflow/blob/05ab6a2afa2959410d48aab2336cea9dc1e2c13e/tensorflow/stream_executor/cuda/cuda_dnn.cc#L1082?

(1) If the TF_CUDNN_RESET_RND_GEN_STATE=1, there is no repro, correct?

Yes, that is for sure.

(2) If the TF_CUDNN_RESET_RND_GEN_STATE=0 (default setting), the use_padded_io is True

Should I check that in the cudnn logs or at tensorflow framework level (i.e., return value of ShouldUsePaddedIO) ?

If yes, can you track it and see if this branch is taken in your training:

I will verify that.

(2) If the TF_CUDNN_RESET_RND_GEN_STATE=0 (default setting), the use_padded_io is True, correct? If yes, can you track it and see if this branch is taken in your training:

It is indeed weird: so far, it looks like we do return true from ShouldUsePaddedIO but when we reach the code you link, the value is false there, and so we don't set the proper value for CUDNN side.

So, it would looks like we are calling GetCachedRnnDescriptor() https://github.com/tensorflow/tensorflow/blob/r1.15/tensorflow/core/kernels/cudnn_rnn_ops.cc#L1077-L1096 with proper use_padded_io=true, but the cache returns a hit and the matching rnn_desc we get is with CUDNN_RNN_PADDED_IO_DISABLED.

I STARTING Optimization
Epoch 0 |   Training | Elapsed Time: 0:00:00 | Steps: 0 | Loss: 0.000000                                                                                2
020-07-28 13:51:55.003459: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
generate_values <deepspeech_training.util.sample_collections.CSV object at 0x7fce547bcc10>
yield generate_values 0 csvs/../data/A/163_5029_3498779ce37873475394654801cc3888-8fddd9522baf442463171802a7e57489.wav <deepspeech_training.util.sample_collections.LabeledSample object at 0x7fd2167e4090>
yield generate_values 1 csvs/../data/A/155_4757_9bc6d6f754547a09bbcf70e42d8e2a27-b112945da6818223ab8e1daf80313a62.wav <deepspeech_training.util.sample_collections.LabeledSample object at 0x7fd2167d8fd0>
yield generate_values 2 csvs/../data/B/98_2923_a387275540ba5f2159c37eaee3e4e9a0-651926517a6241fd9bb5942777b1f0ff.wav <deepspeech_training.util.sample_collections.LabeledSample object at 0x7fd2167d8cd0>
yield generate_values 3 csvs/../data/B/154_4738_2f841fb1af523c579414e0358ab16295-6aea9aa95b1bdbfd80703754cd8a180c.wav <deepspeech_training.util.sample_collections.LabeledSample object at 0x7fce547bcc50>
batch_fn <_VariantDataset shapes: <unknown>, types: tf.string> 2 <_VariantDataset shapes: (?, 26), types: tf.float32> <_VariantDataset shapes: (), types: tf.int32>
batch_fn <_VariantDataset shapes: <unknown>, types: tf.string> 2 <_VariantDataset shapes: (?, 26), types: tf.float32> <_VariantDataset shapes: (), types: tf.int32>
yield generate_values 4 csvs/../data/C/175_5429_67ed7914b9a3bac4e46dd42a5721a95f-e31a33c85ca8249476596c1ff7fc2f67.wav <deepspeech_training.util.sample_collections.LabeledSample object at 0x7fd2167d8f90>
yield generate_values 5 csvs/../data/C/169_5271_3210ac3e97626f9c1515cb019e5fa36e-dd839274af12610f137398ddd01f85f8.wav <deepspeech_training.util.sample_collections.LabeledSample object at 0x7fd2167d8e50>
ShouldUsePaddedIO time_major=1
ShouldUsePaddedIO seq_array[0]=74
ShouldUsePaddedIO seq_array[1]=74
ShouldUsePaddedIO [0]: seq_array[i]=74
ShouldUsePaddedIO [0]: model_shapes.max_seq_length=74
ShouldUsePaddedIO [1]: seq_array[i]=74
ShouldUsePaddedIO [1]: model_shapes.max_seq_length=74
ShouldUsePaddedIO rv=false all_max_seq_length=true
CudnnRNNForwardOp use_padded_io=0
CudnnRNNForwardOp continue use_padded_io=0
CudnnRNNForwardOp GetCachedRnnDescriptor use_padded_io=0
GetCachedRnnDescriptor call with use_padded_io=0
--> GetCachedRnnDescriptor[num_layers, input_size, num_units, dir_count, max_seq_length, batch_size, cell_num_units]: [1, 2048, 2048, 1, 74, 2, 2048]
--> GetCachedRnnDescriptor cache miss use_padded_io=0
--> GetCachedRnnDescriptor call CreateRnnDescriptor use_padded_io=0
-->  CreateRnnDescriptor use_padded_io=0
static stream_executor::port::StatusOr<stream_executor::gpu::CudnnRnnDescriptor> stream_executor::gpu::CudnnRnnDescriptor::Create(const stream_executor::gpu::{anonymous}::CudnnHandle&, int, int, int, int, int, cudnnRNNInputMode_t, cudnnDirectionMode_t, cudnnRNNMode_t, cudnnDataType_t, cudnnDataType_t, const stream_executor::dnn::AlgorithmConfig&, float, tensorflow::uint64, stream_executor::ScratchAllocator
*, bool) IN use_padded_io=0
static stream_executor::port::StatusOr<stream_executor::gpu::CudnnRnnDescriptor> stream_executor::gpu::CudnnRnnDescriptor::Create(const stream_executor::gpu::{anonymous}::CudnnHandle&, int, int, int, int, int, cudnnRNNInputMode_t, cudnnDirectionMode_t, cudnnRNNMode_t, cudnnDataType_t, cudnnDataType_t, const stream_executor::dnn::AlgorithmConfig&, float, tensorflow::uint64, stream_executor::ScratchAllocator
*, bool) Create use_padded_io=0
static stream_executor::port::StatusOr<stream_executor::gpu::CudnnRnnDescriptor> stream_executor::gpu::CudnnRnnDescriptor::Create(const stream_executor::gpu::{anonymous}::CudnnHandle&, int, int, int, int, int, cudnnRNNInputMode_t, cudnnDirectionMode_t, cudnnRNNMode_t, cudnnDataType_t, cudnnDataType_t, const stream_executor::dnn::AlgorithmConfig&, float, tensorflow::uint64, stream_executor::ScratchAllocator
*, bool) Set_v6 use_padded_io=0
static stream_executor::port::StatusOr<stream_executor::gpu::CudnnRnnDescriptor> stream_executor::gpu::CudnnRnnDescriptor::Create(const stream_executor::gpu::{anonymous}::CudnnHandle&, int, int, int, int, int, cudnnRNNInputMode_t, cudnnDirectionMode_t, cudnnRNNMode_t, cudnnDataType_t, cudnnDataType_t, const stream_executor::dnn::AlgorithmConfig&, float, tensorflow::uint64, stream_executor::ScratchAllocator
*, bool) checking use_padded_io=0
ShouldUsePaddedIO time_major=1
ShouldUsePaddedIO seq_array[0]=74
ShouldUsePaddedIO seq_array[1]=74
ShouldUsePaddedIO [0]: seq_array[i]=74
ShouldUsePaddedIO [0]: model_shapes.max_seq_length=74
ShouldUsePaddedIO [1]: seq_array[i]=74
ShouldUsePaddedIO [1]: model_shapes.max_seq_length=74
ShouldUsePaddedIO rv=false all_max_seq_length=true
CudnnRNNBackwardOp use_padded_io=0
GetCachedRnnDescriptor call with use_padded_io=0
--> GetCachedRnnDescriptor[num_layers, input_size, num_units, dir_count, max_seq_length, batch_size, cell_num_units]: [1, 2048, 2048, 1, 74, 2, 2048]
--> GetCachedRnnDescriptor cache miss use_padded_io=0
--> GetCachedRnnDescriptor call CreateRnnDescriptor use_padded_io=0
--> CreateRnnDescriptor use_padded_io=0
static stream_executor::port::StatusOr<stream_executor::gpu::CudnnRnnDescriptor> stream_executor::gpu::CudnnRnnDescriptor::Create(const stream_executor::gpu::{anonymous}::CudnnHandle&, int, int, int, int, int, cudnnRNNInputMode_t, cudnnDirectionMode_t, cudnnRNNMode_t, cudnnDataType_t, cudnnDataType_t, const stream_executor::dnn::AlgorithmConfig&, float, tensorflow::uint64, stream_executor::ScratchAllocator
*, bool) IN use_padded_io=0
static stream_executor::port::StatusOr<stream_executor::gpu::CudnnRnnDescriptor> stream_executor::gpu::CudnnRnnDescriptor::Create(const stream_executor::gpu::{anonymous}::CudnnHandle&, int, int, int, int, int, cudnnRNNInputMode_t, cudnnDirectionMode_t, cudnnRNNMode_t, cudnnDataType_t, cudnnDataType_t, const stream_executor::dnn::AlgorithmConfig&, float, tensorflow::uint64, stream_executor::ScratchAllocator
*, bool) Create use_padded_io=0
static stream_executor::port::StatusOr<stream_executor::gpu::CudnnRnnDescriptor> stream_executor::gpu::CudnnRnnDescriptor::Create(const stream_executor::gpu::{anonymous}::CudnnHandle&, int, int, int, int, int, cudnnRNNInputMode_t, cudnnDirectionMode_t, cudnnRNNMode_t, cudnnDataType_t, cudnnDataType_t, const stream_executor::dnn::AlgorithmConfig&, float, tensorflow::uint64, stream_executor::ScratchAllocator
*, bool) Set_v6 use_padded_io=0
static stream_executor::port::StatusOr<stream_executor::gpu::CudnnRnnDescriptor> stream_executor::gpu::CudnnRnnDescriptor::Create(const stream_executor::gpu::{anonymous}::CudnnHandle&, int, int, int, int, int, cudnnRNNInputMode_t, cudnnDirectionMode_t, cudnnRNNMode_t, cudnnDataType_t, cudnnDataType_t, const stream_executor::dnn::AlgorithmConfig&, float, tensorflow::uint64, stream_executor::ScratchAllocator
*, bool) checking use_padded_io=0
Epoch 0 |   Training | Elapsed Time: 0:00:04 | Steps: 1 | Loss: 188.119797                                                                                b
atch_fn <_VariantDataset shapes: <unknown>, types: tf.string> 2 <_VariantDataset shapes: (?, 26), types: tf.float32> <_VariantDataset shapes: (), types: tf.int32>
ShouldUsePaddedIO time_major=1
ShouldUsePaddedIO seq_array[0]=74
ShouldUsePaddedIO seq_array[1]=75
ShouldUsePaddedIO [0]: seq_array[i]=74
ShouldUsePaddedIO [0]: model_shapes.max_seq_length=75
ShouldUsePaddedIO rv=true all_max_seq_length=false
CudnnRNNForwardOp use_padded_io=1
CudnnRNNForwardOp continue use_padded_io=1
CudnnRNNForwardOp GetCachedRnnDescriptor use_padded_io=1
GetCachedRnnDescriptor call with use_padded_io=1
--> GetCachedRnnDescriptor[num_layers, input_size, num_units, dir_count, max_seq_length, batch_size, cell_num_units]: [1, 2048, 2048, 1, 75, 2, 2048]
--> GetCachedRnnDescriptor cache hit use_padded_io=1
2020-07-28 13:51:59.388865: E tensorflow/stream_executor/dnn.cc:588] CUDNN_STATUS_EXECUTION_FAILED
in tensorflow/stream_executor/cuda/cuda_dnn.cc(1783): 'cudnnRNNForwardTrainingEx( cudnn.handle(), rnn_desc.handle(), input_desc.data_handle(), input_data.opaque(), input_h_desc.handle(), input_h_data.opaque(), input_c_desc.handle(), input_c_data.opaque(), rnn_desc.params_handle(), params.opaque(), output_desc.data_handle(), output_data->opaque(), output_h_desc.handle(), output_h_data->opaque(), output_c_de
sc.handle(), output_c_data->opaque(), nullptr, nullptr, nullptr, nullptr, nullptr, nullptr, nullptr, nullptr, workspace.opaque(), workspace.size(), reserve_space.opaque(), reserve_space.size())'
2020-07-28 13:51:59.388909: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at cudnn_rnn_ops.cc:1537 : Internal: Failed to call ThenRnnForward with model config: [rnn_mode, rnn_input_mode, rnn_direction_mode]: 2, 0, 0 , [num_layers, input_size, num_units, dir_count, max_seq_length, batch_size, cell_num_units]: [1, 2048, 2048, 1, 75, 2, 2048]

@kaixih So, after checking, I'm wondering if there's not something missing for creating the cache key:
https://github.com/tensorflow/tensorflow/blob/r1.15/tensorflow/core/kernels/cudnn_rnn_ops.cc#L1083
We use model_shapes there, which should NOT allow cache hit:

GetCachedRnnDescriptor[num_layers, input_size, num_units, dir_count, max_seq_length, batch_size, cell_num_units]: [1, 2048, 2048, 1, 74, 2, 2048]
is obviously different from:
GetCachedRnnDescriptor[num_layers, input_size, num_units, dir_count, max_seq_length, batch_size, cell_num_units]: [1, 2048, 2048, 1, 75, 2, 2048]

But according to the log above, we still had a cache hit on this.
Checking the definition of CudnnRnnModelShapes https://github.com/tensorflow/tensorflow/blob/r1.15/tensorflow/core/kernels/cudnn_rnn_ops.cc#L494-L520, it is my understanding that IsCompatibleWith is used to produce the cache key: https://github.com/tensorflow/tensorflow/blob/r1.15/tensorflow/core/kernels/cudnn_rnn_ops.cc#L507-L512
One can see that this method does not take into account max_seq_length, which would explain why we get a cache hit with padded disabled when we had it enabled.
At least, with that patch, I can't repro anymore the issue, and I can see cache misses and cache hits:
diff --git a/tensorflow/core/kernels/cudnn_rnn_ops.cc b/tensorflow/core/kernels/cudnn_rnn_ops.cc
index 4a27394f28..9098b97462 100644
--- a/tensorflow/core/kernels/cudnn_rnn_ops.cc
+++ b/tensorflow/core/kernels/cudnn_rnn_ops.cc
@@ -508,7 +508,7 @@ struct CudnnRnnModelShapes {
   bool IsCompatibleWith(const CudnnRnnModelShapes& rhs) const {
     return num_layers == rhs.num_layers && input_size == rhs.input_size &&
            num_units == rhs.num_units && dir_count == rhs.dir_count &&
-           cell_num_units == rhs.cell_num_units;
+           cell_num_units == rhs.cell_num_units && max_seq_length == rhs.max_seq_length;
   }
   string DebugString() const {
     return strings::Printf(

@lissyx
Looks very plausible to be the root cause to me!
From a code point of view and it also fits with and explains all the patterns we saw while testing and debugging this issue. The same code seems also still present in TF2.x and master, which correlates with all the other reports you found about LSTM with CUDA/CUDNN being unstable while training and are likely related.
Great catch !

Great catch !

Thanks, no offense but I'll claim victory once I get feedback from @kaixih :)

@lissyx Thanks, I agree with you that this should be the root cause. (This also reminds me that I probably forgot to add corresponding changes to the cached RNN desc searching back when adding the variable sequence length feature for cuDNN).
For your two patches, I prefer to change both or only the first one. The first one about the IsCompatibleWith used here is mainly for the probing in the hash table, but might be used elsewhere to compare the RNN states. And I think it makes sense to distinguish RNNs with different max_seq_length. The second one about the hash function is mainly for generating the hash key for the RNN states and your patch could give us a little faster searching performance (because we can skip the probing). So, I think the first one is probably a necessity or we simply change both.
For the next step, can you create a PR to fix this bug and maybe link to this issue? Thanks.

For the next step, can you create a PR to fix this bug and maybe link to this issue? Thanks.

Sure!

I suspect there won't be a 1.15 dot release including that kind of change, unfortunately?

Right, I think Google only fixes major issues for 1.15. Can you first create a PR against master? And later they might pick it for 1.15 if necessary? @sanjoy

Right, I think Google only fixes major issues for 1.15. Can you first create a PR against master? And later they might pick it for 1.15 if necessary? @sanjoy

Yep, it's done: https://github.com/tensorflow/tensorflow/pull/41832

@kaixih So I have been running experiment on ~1000h of french training data, and setting TF_CUDNN_RESET_RND_GEN_STATE=1 would make one epoch of training about 20 seconds longer, ~23:30 minutes on my hardware from the ~23:10 minutes when using TF_CUDNN_RESET_RND_GEN_STATE=0
Does that sounds reasonnable to you? Are there any side-effects to using that env variable we should know about?

@kaixih So I have been running experiment on ~1000h of french training data, and setting TF_CUDNN_RESET_RND_GEN_STATE=1 would make one epoch of training about 20 seconds longer, ~23:30 minutes on my hardware from the ~23:10 minutes when using TF_CUDNN_RESET_RND_GEN_STATE=0
Does that sounds reasonnable to you? Are there any side-effects to using that env variable we should know about?

@kaixih Gentle ping: we are close to release 1.0, and we'd like to know exactly what are the consequences of that flag so that we can either force it by default until a (potential 1.15.4 is released) or at least recommend people to use it?
Thanks!

@kaixih
Is the fix from @lissyx  fix actually enough ?
I have now rebuild tensorflow with this fix and it now doesn't blow up in the training stage, but in the validation stage.
If i look at the code of:
Status GetCachedRnnDescriptor(
OpKernelContext* context,
const CudnnRnnModelShapes& model_shapes,
const RnnInputMode& input_mode,
const AlgorithmConfig& algo_config,
RnnStateCache* cache, RnnDescriptor** rnn_desc,
bool use_padded_io
)

Is it really enough to only check if the model_shapes match and not input_mode, algo_config and use_padded_io ?
Another question is if the whole caching is really worthwhile, since lissyx has reported on a test run he didn't find a stellar difference in training duration with or without the caching ?

Just let you know guys that in my system I need to set TF_CUDNN_RESET_RND_GEN_STATE=1  flag to make it to work even after upgrading to 1.15.4

me aswell, it does not work without: "export TF_CUDNN_RESET_RND_GEN_STATE=1 " in the .sh file i run to fine-tune the model.
But it does work with it ! so i am a happy camper again.
perhaps the documentation should be updated again @lissyx ?

me aswell, it does not work without: "export TF_CUDNN_RESET_RND_GEN_STATE=1 " in the .sh file i run to fine-tune the model.
But it does work with it ! so i am a happy camper again.
perhaps the documentation should be updated again @lissyx ?

No, it means there might be other cases triggering some issue. You are welcome to investigate the internals of TensorFlow / CUDNN.

I am in the same boat as you, insanely busy with work and my toddler.
i am writing my bachelor thesis now, and it is to be handed 4.th of January.
what i meant is that it might be a good idea to mention this tensorflow cudnn flag in the documentation.
but i guess that must folks will google the error and find these 3 threads.
i might look into this in February though.

I am in the same boat as you, insanely busy with work and my toddler.
i am writing my bachelor thesis now, and it is to be handed 4.th of January.

Good luck, been there, done that, I know how you are right now.

what i meant is that it might be a good idea to mention this tensorflow cudnn flag in the documentation.

As you can see from the discussion, this flag is a workaround not a fix. If there's a documentation that might benefit from it, it would be deepspeech, not tensorflow as much as I can tell.

but i guess that must folks will google the error and find these 3 threads.
i might look into this in February though.

Please understand that tracking and fixing the root cause in TensorFlow to ensure DeepSpeech users are safe already took me weeks, specifically from the lack of reproductibility. That was when I was 100% of the time on the DeepSpeech project, which is not the case anymore, so, I'm sorry, but I really can't look into that.


Check the embedded defaults in release in CI

Following up on this comment:
https://github.com/microsoft/kiota/pull/2184#issuecomment-1405641324
cc. @baywet

Thanks for the additional tests here!


Error for tasks with parameters

I tried to input some parameters (like this @{'BuildWorkingFolderNumber'="$(BuildWorkingFolderNumber)"} and this BuildWorkingFolderNumber=14) and it didn't worked. The psake build step doesn't treat parameters in any way that I've tried.

Hi @alexbwasp,
The syntax for parameters is a comma- or newline-separated list of name=value. In your example it should be:
BuildWorkingFolderNumber=$(BuildWorkingFolderNumber)
If this is not working, can you run your build with the variable system.debug at true and send the verbose log (where all task parameters should have been output) of the psake task?
Thanks,

Hi @qetza,
I tried again and it did worked! Thanks!


Update README.md

Updated readme
Highlighted the inclusion of the recompiled executable more prominently.

Thanks - yeah, good point.  I'd assume the same.


Sign in and sign up frontend

Finished the basics of the sign in and sign up forms

Done with validations


FEATURE: add JDBC connection URL to Warehouse instance

Please don't break compatibility, but provide as much compatibility as is reasonable (new client with old server, old client with new server,...).
The old method has to stay, call the new method and be annotated as deprecated with proper explanation.

please bear in mind java sdk is primary rest api client and every change has to be performed in best interest of the rest api clients


Image transforms detr functionality

What does this PR do?
Adds transforms + tests for the DETR image processor
Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[x] Did you read the contributor guideline,
Pull Request section?
[ ] Was this discussed/approved via a Github issue or the forum? Please add a link
to it if that's the case.
[ ] Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
[ ] Did you write any new necessary tests?

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


Kotlin SDK for Java Customers

The Kotlin language supports interop with Java, including Java programs calling into Kotlin libraries.  This task is to determine how usable the Kotlin SDK is from a Java developer's perspective.  This would include doing some prototyping work from Java to get a sense of how the language boundry impacts readability as well as general programming ergonomics.  If issues are encountered that can be rectified with either codegen or client runtime changes, those tasks need to be identified and scoped.
Acceptance critera:  Be able to anwser the following questions ~

Is the Kotlin SDK usable from Java?  If not what are the issues?
Are there changes we could make that would improve the Java experience w/out regressing on the Kotlin side (see project tenets)  If so create tasks for changes.

internal id: 176347179

Closing in favor of community-focused issue https://github.com/awslabs/aws-sdk-kotlin/issues/433