setting delayed close timer 200ms but still have default 1s

Title: setting delayed close timer but still have default 1s
Description:

i  setting delayed close timer 200ms ，but still have default 1s
why？

Based on what you posted, my best guess is that you have two filter chains, and only set this timeout on one of them. But I can't tell without more data.

@cobb-tx can you post your full envoy configuration?
Based on what you posted, my best guess is that you have two filter chains, and only set this timeout on one of them. But I can't tell without more data.

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: default-backend
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
    - applyTo: NETWORK_FILTER
      match:
        context: GATEWAY
        listener:
          filterChain:
            filter:
              name: "envoy.filters.network.http_connection_manager"
      patch:
        operation: MERGE
        value:
          typed_config:
            "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager"
            common_http_protocol_options:
              idle_timeout: 75s  # 1 hour
            stream_idle_timeout: 75s
            delayed_close_timeout: 200ms
            local_reply_config:
              mappers:
                - filter:
                    status_code_filter:
                      comparison:
                        op: EQ
                        value:
                          default_value: 404 # Replace to 401 if you want to customize for the JWT validation fail response
                          runtime_key: placeholder
                  body:
                    inline_string: "default backend - 404"

---
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: ingress-gateway-health-check
  namespace: istio-system
spec:
  workloadSelector:
    # select by label in the same namespace
    labels:
      app: istio-ingressgateway
  configPatches:
    # The Envoy config you want to modify
    - applyTo: HTTP_FILTER
      match:
        context: GATEWAY
        listener:
          filterChain:
            filter:
              name: "envoy.filters.network.http_connection_manager"
              subFilter:
                name: "envoy.filters.http.router"
      patch:
        operation: INSERT_BEFORE
        value: # lua filter specification
          name: envoy.lua
          typed_config:
            "@type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua"
            inline_code: |-
              function envoy_on_request(request_handle)
                if request_handle:headers():get(":path") == "/healthz"then
                  request_handle:respond(
                  {[":status"] = "200" },
                  "")
                end
              end

Look at the following code: https://github.com/envoyproxy/envoy/blob/main/source/common/http/conn_manager_impl.cc#L148-L154
Do you enable maxConnectionDuration?

@daixiang0  maybe not enable maxConnectionDuration

Do you have a full config dump through localhost:15000/config_dump ?
your EnvoyFilter seems applying delay close to all http  conn manager but I want to confirm at config_dump

we are also seeing similar issue where delay is not getting applied correctly. Since this happens in production so we cant post the config dump but anything specific u want us to check or may be part of the dump ?

``

from logs found out, 1000ms was for static listener to which envoyfilter doesnt apply, so found it


No GoXLR Devices Found

Not sure if I've missed a step somewhere, or if I've missed an already sorted issue, but I've taken a look and been unable to find anything that seems to provide guidance on this.
I've got both a GoXLR Mini and a full sized GoXLR, and have been unable to get either showing up in the utility. Levels adjust as expected, at least on the full sized (haven't tested that on the Mini yet) running the newest firmware, and the Mini powers up, both show in Audio with their individual channels.
Pipewire with Pulse appears to be the default on my OS, and I was unable to see whether this is likely to be what that caused, or if there's something else that may be the cause; I'm slowly working through getting it set up to know what I need to do to swap into my main system, but I'm still gradually working my way through figuring some things out.
Running on NixOS, with the following being all I can find regarding my sound configuration so far
# Enable sound with pipewire.
  sound.enable = true;
  hardware.pulseaudio.enable = false;
  security.rtkit.enable = true;
  services.pipewire = {
    enable = true;
    alsa.enable = true;
    alsa.support32Bit = true;
    pulse.enable = true;
    # If you want to use JACK applications, uncomment this
    jack.enable = true;

Not sure what else may be needed or useful, but if there's anything else me know.

Could you provide the log files and let me know how you installed the utility? If I were to guess, the udev rules for the GoXLR are missing.

Thanks for the quick response, unfortunately it was just as I got to bed so I couldn't respond earlier.
I'd love to provide the logs, but could you specify which ones? I'd hate to provide ones that aren't useful, and I'm still a little green to Linux to know which handle what.
As for how I installed it, I installed it through the NixOS configuration file. Any advice for how to check the udev rules? I think I saw something while going through previous issues posted about it, but my attempts to run the copy through sudo cp 50-goxlr.rules /etc/udev/rules.d/50-goxlr.rules && sudo udevadm control --reload-rules && sudo udevadm trigger failed with cp: cannot stat '50-goxlr.rules': No such file or directory; I'm assuming I'm just running it in the wrong place, but I'm not sure where the right place would be.

The Nix package for the GoXLR Utility is managed by a third party, so it's a little tricky to help support there (I'm personally not a Nix user), but I'll give @Sporesirius and @errnoh a ping (sorry!), as they were responsible for getting the package sorted there.
For the logs, they're available in ~/.local/share/goxlr-utility/logs/, the latest goxlr-daemon.log may provide insight.

Thanks for the ping @FrostyCoolSlug
I'm using the latest version from Nix so "works on my machine", but lets see if I can help with debugging. If nothing else, at least I might be able to rubberduck.
So for the relevant background:

the 50-goxlr.rules are inside the packages etc/udev/rules.d directory as per https://github.com/nixos/nixpkgs/blob/master/pkgs/tools/audio/goxlr-utility/default.nix#L47
the rules are installed on this line: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/audio/goxlr-utility.nix#L33 which installs all rules in «pkg»/etc/udev/rules.d and «pkg»/lib/udev/rules.d

If it worked correctly should be able to see the file installed at /etc/udev/rules.d/50-goxlr.rules, if not, maybe you don't have services.goxlr-utility.enable = true; set?
After confirming that my next question would be about your overall setup. goxlr-utility is using XDG autostart, a feature which I hadn't run into before I added this nix package (but that has worked really well so no complaints there), what I don't know is if XDG autostart has some requirements from the system itself (maybe @FrostyCoolSlug  knows more about that part?). Anyways, if that ends up being the issue back when I created the module I added services.goxlr-utility.autoStart.xdg option that can be disabled in case someone wants to run it manually or using some other method they desire.

@TheRedHerring95 did you resolve your issue?

Sorry, it's been a busy few days at work and I haven't had the chance to jump on my laptop and test this again since your suggestions and guidance. Only one more day till the weekend, so hopefully I'll be able to get back to troubleshooting it in the next day or so.
I appreciate you checking whether I'd solved it though; as much as I like to think I'll jump straight on and confirm it's working with just what it was, I acknowledge that there's definitely a chance I'd get side tracked and end up not getting the comment added with just what happened to get everything working.

I am also having this issue - also on NixOS
goxlr-daemon.log showed the following error repeated:
[ERROR] Couldn't load potential GoXLR on bus 1 address 2: Access denied (insufficient permissions)
Am I perhaps missing a group on my user?

@TheRedHerring95 no worries, just wanted to make sure there's nothing wrong with the package that I'd need to fix :) and to make sure that the issue gets sorted out within decent time.
@Different-Name this does also look like it's due to missing udev rules. Check my message above ( https://github.com/GoXLR-on-Linux/goxlr-utility/issues/172#issuecomment-2187155379 ) and ensure you have the goxlr utilities module enabled.

@errnoh sorry I forgot to mention I checked the rules were in place, the file was indeed there. I have the module enabled

@Different-Name hmm, weird. Assuming the contents of /etc/udev/rules.d/50-goxlr.rules also match https://github.com/GoXLR-on-Linux/goxlr-utility/blob/main/50-goxlr.rules then the next steps in debugging would probably be:

ensure you have restarted since installing, just to be sure
maybe sudo udevadm control --reload-rules && sudo udevadm trigger or similar, though if you have restarted this seems unlikely to be required
re-connect the GoXLR device itself, there could be some initialization that needs to happen but also requires the rules, so attaching the device after rules are already in effect might result in problem being solved.

But do report back even if that doesn't help, the Access denied (insufficient permissions) just seems like quite generic udev error that should be fixed with just turning things on and off in correct order :sweat:

To continue the trend I'm unfortunately not going to have access to my goxlr for the next week 🥲
I'll double check and get back to you when I can - thank you

Hi @errnoh, sorry for the delay in checking back in; life's thrown a few curve balls in the last couple of weeks, so I didn't end up having the chance last weekend to take a look.
I do bare good news though; I didn't have it enabled, and rebuilding with that line you'd mentioned seems to have resolved the issue. I did have an issue at first where it didn't pick up if the device was already connected when I launched the utility, but that appears to be working fine now as well.
I've only tested on the Mini at this point, but will confirm everything is operating as expected on the full sized unit tonight and let you know if there's any complications.
Let me know if there's any additional checking I can do to compare between myself and @Different-Name as well, in case we can identify the difference and what may be causing mine to now work with that additional line while theirs is still having issues.

I assume my issue is because I did not reboot, the udev rules only apply when a device is connected while the system is running, so that's probably the issue I encountered. Always reboot!
Will update when I can though

Huh, interesting; I didn't reboot for mine; just ran the rebuild switch command and went with testing. Had to restart the utility with the device not connected, but after that it seemed solid on my end. Definitely worth a shot though! Good luck :slightly_smiling_face:

After a reboot, it is working as intended

Glad to hear it solved the issue :)
I'm not sure if there's some good way to signal NixOS users if there's a module that's supposed to be used instead of just installing the package. While I could submit a PR in this repo to guide people how to enable the service on NixOS I don't want to burden @FrostyCoolSlug with NixOS related questions when it's not an "officially" supported version, but at the same time I wouldn't be surprised to see this being asked again in the future.
Anyways, @TheRedHerring95 or @FrostyCoolSlug based on the above comments (i.e. both managing to get the service running properly) I'd say that this GH issue can be closed!

It's definitely possible to disable direct usage of a package, I saw it done a couple of days ago while I was looking through the nixpkgs repo, though I was not able to find it again doing a quick search.

Found it, not sure if it helps though
https://github.com/NixOS/nixpkgs/blob/dc14ed91132ee3a26255d01d8fd0c1f5bff27b2f/pkgs/by-name/pr/proton-ge-bin/package.nix#L24-L26


Incorrect wording in section "Upload user data to the inbound provisioning API"

Under section "Upload user data to the inbound provisioning API", in step 4 the bracketted statement next to the second bullet state next to the link "Test-API2AAD.postman_environment.json"  "(Environment collection for API-driven provisioning to on-premises AD)-" and this should be "(Environment collection for API-driven provisioning to Azure AD)"

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 12bbf2b1-94d1-803e-eb08-471942c336a5
Version Independent ID: c84a8620-6aa2-2790-5cf5-cd4fec1d9a8f
Content: Quickstart API-driven inbound provisioning with Postman - Microsoft Entra
Content Source: articles/active-directory/app-provisioning/inbound-provisioning-api-postman.md
Service: active-directory
Sub-service: app-provisioning
GitHub Login: @jenniferf-skc
Microsoft Alias: jfields

@CarlH2
Thanks for your feedback! We will investigate and update as appropriate.

@CarlH2
Thanks for reporting this! We have created a PR for this issue once the author will review and approve the changes should go live soon.


Enable ndims expansion and interpolation into kernel signature

CC @albert-de-montserrat

Merging this, but the syntax for interpolation will be changed in a next PR as discussed with @albert-de-montserrat


chore(deps): upgrade compiler dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-compiler-dependencies-1.x" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 2883

Questions ?
Please refer to the Backport tool documentation


this gem should not be used for validating domain names

README is suggestive of using this gem for validating domains, however it is severely broken for that purpose, as almost any character is allowed in labels. Valid labels in domains should only contain characters in [\-A-Za-z0-9] and are limited to 63 characters. I suggest adding a warning in the README about this.

The purpose of this library is to validate a name according to the PSL. It's not a library to validate URIs or performs syntax validations of strings.

@weppos I appreciate what you are saying there, but the README says this gem is a "domain name parser" (without further qualification), when it most certainly is not. It can only be used to check if a given arbitrary string ends in a valid TLD listed in the PSL, which is not the same thing as parsing a domain. As you can see above, it accepts a newline in the string and the domain method fails to parse the components of what constitutes a valid domain properly.
I would suggest it should either reject invalid input or parse out the part (jwerwe123.com in the above) that is still considered a valid domain. Silently producing invalid output when given invalid input is neither here nor there.
Said another way, a more appropriate name for what the valid? method does would be ends_with_valid_tld?.
If you are not willing to make any changes, I suggest at least mentioning that the inputs to this gem or the output of the domain, subdomain etc methods should be further validated by some other means.  I'm happy to submit a PR if you agree.

@weppos I appreciate what you are saying there, but the README says this gem is a "domain name parser" (without further qualification)

This is incorrect. It says "is a Ruby domain name parser based on the Public Suffix List". That makes a lot of difference.
I also don't write anywhere that this is a domain validator.

I would suggest it should either reject invalid input or parse out the part (jwerwe123.com in the above) that is still considered a valid domain. Silently producing invalid output when given invalid input is neither here nor there.

By design, I decided to not introduce this feature because it would require a non insignificant maintenance effort. There are just plenty of possible ways you can submit invalid string data.
This library expect a formally valid domain name. It is user responsibility to feed a correctly formatted data. If you want an URI validator, I recommend you look into https://www.rubydoc.info/gems/addressable/Addressable

If you are not willing to make any changes, I suggest at least mentioning that the inputs to this gem or the output of the domain, subdomain etc methods should be further validated by some other means. I'm happy to submit a PR if you agree.

I will look into that and see how I can make it even more explicit.
However, please note the documentation already exists. The library is extensively documented, and each method has its own docblock. Here's an example of the parameter explanation for both valid and parse:
  # @param  [String, #to_s] name The domain name or fully qualified domain name to validate.

Note it explicitly references a domain name or FQDN. It doesn't say "arbitrary string that may contain or look like a domain". I understand it may be a subtle difference to grasp, but these terms carry on a pretty well defined specification in the corresponding RFCs. I don't intend to package such RFCs baggage into this library, as it's beyond the scope of a PSL implementation.


Address MultiSelection Keyboard Selection Bug

Summary
This just fixes a bug where mutli-selection on first load wasn't allowing for keyboard shortcuts on selected elements.
This is just because doing the drag gesture with a mouse doesn't actually cause the browser to change focus.
Focus on the canvas or one of its children is necessary to pick up keyboard events on it.
The bug was caused because we were only setting the canvas as the default focus element in focusout which doesn't fire if you multiselect right after initial page load. The only real update here was checking if we needed to update the default focus on the initial render.
Relevant Technical Choices
Changed setTimout duration to be 0 instead of 300 because it will still be queued after any potential focusin events queued up in the IO queue, but be a little more snappy. 🌭 🐊
Also added in depth comments because the approach wasn't very straight forward to me at first. I'd be happy to remove some if they're feeling a little dense
To-do
If we wanted to, we could make a useDefaultFocus(ref) hook that encapsulates this functionality since it feels like it has a separate purpose then the useCanvasKeys(ref) hook. But also feel like the hooks are already pretty spread out and nested here, so understand not wanting to do this.
User-facing changes
Fixes being able to use keyboard commands on mutliselected canvas elements on initial load.
Testing Instructions

On a fresh load of the story editor multiselect several elements before clicking anywhere else on the page. try to use keyboard shortcuts (arrows, delete) on them and they should work.

Also just play around with editor a but and make sure there were no regressions on inputs.

Addresses #1070

@dvoytenko please review this in light of your focus PR - is this overlapping?

I'm gonna close this since I think @dvoytenko 's update to how we we're capturing focus on the canvas fixed this


Background images are being shown incorrectly

sure! this can help narrow down the automatic image process.. the woman image is the one scrambling.  the one with
pencil on guy is working (the pencil color doesnt show up but thats ok for now...)

Thanks
Originally posted by @zartan9 in https://github.com/haroldo-ok/choice4genesis/issues/66#issuecomment-1348872185

Okay, thanks; it seems that, indeed, that image is being displayed incorrectly, no matter how many ways one tries to convert it. I'll have to take a look at this one later.
bug.zip

Starting to look at the bug.
Working branch: https://github.com/haroldo-ok/choice4genesis/tree/fix-background-image-bug

It does not seem to be a tileMap problem; if the tilemap part gets commented out, the bug still happens. VDP_loadTileSet() does return null, though, indicating that it is running out of memory.
void VN_showImage(const Image *image, VDPPlane plane, u16 palNum, u16 x, u16 y) {
	VDP_loadTileSet(image->tileset, imageInfo.tileNumber, DMA);

	TileMap *tmap = unpackTileMap(image->tilemap, NULL);
	VDP_setTileMapEx(plane, tmap, TILE_ATTR_FULL(palNum, FALSE, FALSE, FALSE, imageInfo.tileNumber),
		x, y,  0, 0, tmap->w, tmap->h, CPU);
	MEM_free(tmap);

	VDP_setPalette(palNum, (u16*)image->palette->data);
	imageInfo.tileNumber += image->tileset->numTile;
}

Tested various permutations of the code, with various configurations, even going as far as removing compression; the problem persists.

ok keep me posted im sure youll get it!! Im working on the text in the meantime! thanks!
On Sunday, January 8, 2023 at 05:49:53 PM EST, Haroldo de Oliveira Pinheiro @.***> wrote:
It does not seem to be a tileMap problem; if the tilemap part gets commented out, the bug still happens. VDP_loadTileSet() does return null, though, indicating that it is running out of memory.
void VN_showImage(const Image *image, VDPPlane plane, u16 palNum, u16 x, u16 y) {
VDP_loadTileSet(image->tileset, imageInfo.tileNumber, DMA);
TileMap *tmap = unpackTileMap(image->tilemap, NULL);
VDP_setTileMapEx(plane, tmap, TILE_ATTR_FULL(palNum, FALSE, FALSE, FALSE, imageInfo.tileNumber),
	x, y,  0, 0, tmap->w, tmap->h, CPU);
MEM_free(tmap);

VDP_setPalette(palNum, (u16*)image->palette->data);
imageInfo.tileNumber += image->tileset->numTile;

}
Tested various permutations of the code, with various configurations, even going as far as removing compression; the problem persists.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>

i knew you could do it! it works!
On Sunday, January 8, 2023 at 05:49:53 PM EST, Haroldo de Oliveira Pinheiro @.***> wrote:
It does not seem to be a tileMap problem; if the tilemap part gets commented out, the bug still happens. VDP_loadTileSet() does return null, though, indicating that it is running out of memory.
void VN_showImage(const Image *image, VDPPlane plane, u16 palNum, u16 x, u16 y) {
VDP_loadTileSet(image->tileset, imageInfo.tileNumber, DMA);
TileMap *tmap = unpackTileMap(image->tilemap, NULL);
VDP_setTileMapEx(plane, tmap, TILE_ATTR_FULL(palNum, FALSE, FALSE, FALSE, imageInfo.tileNumber),
	x, y,  0, 0, tmap->w, tmap->h, CPU);
MEM_free(tmap);

VDP_setPalette(palNum, (u16*)image->palette->data);
imageInfo.tileNumber += image->tileset->numTile;

}
Tested various permutations of the code, with various configurations, even going as far as removing compression; the problem persists.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>

Thanks!


auth v3: support jwt as auth token

For production use cases, current simple token of auth v3 is too naive e.g. its token is a randomly generated string and doesn't care about the cost of guessing by attackers.
JWT seems to be useful for the purpose. auth v3 needs to support it as its token. jwt-go would be a suitable implementation for etcd.

What do we expect to put inside each jwt token? Just username? Or also the role details attached to the user?

I think just username would be fine for now. It may be affected by proxy design and including role details would be useful for the case (but it will bring consistency problems).

how are we going to revoke a jwt token? will it be timeout based?

I think there are two ways;

revision based. It requires that AuthStore is revision aware
timeout based. Of course etcd nodes must not use their local clock for this purpose. So we need a new  clock driven by raftpb.MsgHeartbeat.

The second one would be simpler. How do you think?

@mitake I am closing this. We still need to work on the documentation side. There is already a tracking issue.

@xiang90 yes, the doc issues can be tracked here: https://github.com/coreos/etcd/issues/7251
Thanks for closing.


Implement GetRecordsConfig.DoWithContext

Hello, thanks for the nice library :)
We use code like:
records, err := table.GetRecords().
		FromView(viewID).
		WithOffset(offset).
		Do()

and we have issue with Do() that calls Table.GetRecordsWithParams which calls Table.GetRecordsWithParamsContext using default context context.Background(). We would like to have option to pass custom context instance.
The request I have is to implement DoWithContext(context.Context) that does just that. I could not find a way to achieve passing context, as in the end GetRecordsConfig.params are private

@michalb-goflink , thank you
Fixed v0.3.2+

@mehanizm nice, thanks


Build fails looking for antlr-3.5.2

I am getting the following error when trying to build/setup a new environment:
Building and installing the app on the device (cd android && gradlew.bat installDebug)...
Download https://jcenter.bintray.com/org/antlr/antlr/3.5.2/antlr-3.5.2.jar
FAILURE: Build failed with an exception.

What went wrong:
A problem occurred configuring root project 'MyProject'.

Could not resolve all dependencies for configuration ':classpath'.
Could not download antlr.jar (org.antlr:antlr:3.5.2)
Could not get resource 'https://jcenter.bintray.com/org/antlr/antlr/3.5.2/antlr-3.5.2.jar'.
java.lang.NullPointerException (no error message)

Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED
Environment
Environment:
OS: Windows 10
Node: 9.9.0
Yarn: 1.5.1
npm: 5.7.1
Watchman: Not Found
Xcode: N/A
Android Studio: Version  <IP_ADDRESS> AI-171.4443003
Packages: (wanted => installed)
react: 16.3.0-alpha.1 => 16.3.0-alpha.1
react-native: 0.54.2 => 0.54.2
Steps to Reproduce
Have installed the version 23 SDK and build tools

Was thinking the problem could be something to do with environment variables this is what i have:

Expected Behavior
Expected it to build and deploy to the emulator
Actual Behavior
Build fails

Well hitting my head on the wall has fallen broken the wall, I found a solution to this problem.
What I did was type:
react-native link
This downloaded the necessary antlr files it was missing of both 3.5.2 and 4.5.3 and now it works fine.
Maybe a more knowledgeable person can explain what is causing this but here is the solution for those that hit this issue also.


Data cleanup that fixes #42

Adds a few extra SQL statements to clean up the data in the raw_table data before trying to insert it into contributions. Fixes #42 so that mysql_init_db.py runs properly when dealing with MySQL 5.7

Thanks!


Implement lookaside cache client - downloading and uploading

Should this actually be part of this library? It is a necessary functionality for rebase-helper and packit, yes, but it's not really related to spec files.

Should this actually be part of this library? It is a necessary functionality for rebase-helper and packit, yes, but it's not really related to spec files.

What about creating a subpackage/submodule so we can prepare the shared code and Packit/RebaseHelper part but still be able to exclude it later if we want by just fixing the imports. (=Built it on top of Specfile.) It would be really nice to have a different package for this but I am worried it gives us too much maintenance overhead.
But we need to think about this twice. It looks like a precedent for other use cases we will definitely find..;)

On the other hand (after a while of thinking), it can be really handy to call this code directly from Specfile objects.
e.g. (not sure about the real syntax/types/names..;)
for source in my_specfile.sources:
    if source.is_cached:
        source.upload_to_cache()
    else:
        source.download_from_cache()

Or at least make it work with the Specfile objects.
e.g. (not sure about the real syntax/types/names..;)
the_lookaside_cache = LookasideCache()

for source in my_specfile.sources:
    if source not in the_lookaside_cache:
        the_lookaside_cache.upload(source)
    else:
        the_lookaside_cache.download(source)

I think that's a good idea :+1:

We decided not to implement this in specfile and use rpkg for lookaside cache functionality in rebase-helper and packit.


[Suggestion] On the fly Texture switching with only one version(non-duplicate) of a model

Basically you can make or import a list of textures for a model so the user can save space by having one copy of the model that can have many textures to chose between instead of many copies were the only thing changed is the texture.
This would also make it easy to update the model since you would only need to update one model and all the textures would already use the updated model instead of vary slowly updating one at a time.
of course this is only useful for when there is many different textures for one model but it's nicer to just have the option.

It would be even better if this was supported by the ModelFile API.
I'm imagining code like the following:
// File file
ModelFile modelFile = ModelFile.load(file);

// File imageFile
modelFile.setTexture(/* texture_id etc */, imageFile);
// OR
ModelFile anotherModelFile = modelFile.withTexture(/* texture_id etc */, imageFile);


No Module Named 'torch'

When I run pip install flash-attn, it says that. But obviously, it is wrong. See screenshot.

Same issue for me.

Workaround: install the previous version pip install flash-attn==1.0.5

I am seeing the same problem on every flash_attn version. I am using Cuda 12.1 on the new g2 vm instance from gcp. https://cloud.google.com/compute/docs/accelerator-optimized-machines#g2-vms

Workaround: install the previous version pip install flash_attn==1.0.5

This might work in some scenarios but not all.

Can you try python -m pip install flash-attn?
It's possible that pip and python -m pip refer to different environments.
Getting the dependencies right for all setup is hard. We had torch in the dependency in 1.0.5, but for some users it would download a new version of torch instead of using the existing one. So for 1.0.6 we leave torch out of the dependency.

Getting the same issue. I also tried python -m pip install flash-attn as you suggested with the same failure.

same problem here.

I don't know a right solution that works for all setups, happy to hear suggestions.
We recommend the Pytorch container from Nvidia, which has all the required tools to install FlashAttention.

I believe this is an incompatibility issue with cuda 12.1 version of torch.
Using the following torch version solves my probem.
torch==2.0.0+cu117

@smeyerhot I use the exact version, but it doesn't work. See the screenshot.

I believe this is an incompatibility issue with cuda 12.1 version of torch.
Using the following torch version solves my probem.
torch==2.0.0+cu117

Sorry! This didn't fix things... please ignore this comment.

@smeyerhot No problem. Thanks a lot anyway!

same problem

same problem

pip install flash-attn==1.0.5 might help. I am using torch 1.13 and cuda 12.0.

I had the same issue with pip. Workaround was to compile from source, worked as a charm
In [1]: import flash_attn

In [2]: import torch

In [3]: torch.__version__
Out[3]: '2.0.1+cu117'

In [4]: flash_attn.__version__
Out[4]: '1.0.6'

I had the same issue with pip. Workaround was to compile from source, worked as a charm
In [1]: import flash_attn

In [2]: import torch

In [3]: torch.__version__
Out[3]: '2.0.1+cu117'

In [4]: flash_attn.__version__
Out[4]: '1.0.6'

I also had the same issue, but my system needs Cuda 12.1 (2x Nvidia L4). so using torch 117 is not an option.
this is also my workaround and it works like a charm.
my system uses Fedora Server

I compiled it myself using a docker container and I still get
RuntimeError: Expected q_dtype == torch::kFloat16 || ((is_sm8x || is_sm90) && q_dtype == torch::kBFloat16) to be true

try pip install flash-attn --no-build-isolation  fixed my problem.
pip docs
to fix this problem, maybe adding torch dependency into pyproject.toml can help

@xwyzsn Unfortunately this only worked on my windows system, not linux. But I feel we're making progress.

to fix this problem, maybe adding torch dependency into pyproject.toml can help

We had torch in the dependency in 1.0.5, but for some users it would download a new version of torch instead of using the existing one.
I'm not really an expert in Python packaging, so it's possible I'm doing sth wrong.

@xwyzsn Unfortunately this only worked on my windows system, not linux. But I feel we're making progress.

Hi, actually I am using linux. It also worked well. I assume that you may missed some other package  to build this up  in your linux system.

--no-build-isolation ...Build dependencies specified by PEP 518 must be already installed if this option is used.

same problem to me,  i solve by check my device and torch cuda version.

try pip install flash-attn --no-build-isolation fixed my problem. pip docs to fix this problem, maybe adding torch dependency into pyproject.toml can help

This fixed the torch problem, but now I got an other error but might be related to something else.
        435 |         function(_Functor&& __f)
            |                                                                                ^
      /usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
      /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
        530 |         operator=(_Functor&& __f)
            |                                                                                ^
      /usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
      error: command '/usr/bin/nvcc' failed with exit code 255

@xwyzsn ninja was removed, then torch was removed, then ninja was re-added. Next logical step is to re-add torch. right??? 😄

There must be some recent change that caused this issue. @tridao

Same issue in Kubuntu 20 with torch 2.0.1 and cuda 11.8 python 3.9 / python 3.10 and flash-attn versions 0.2.8 / 1.0.4 / 1.0.5 / 1.0.6 / 1.0.7 with and without --no-build-isolation flag.

Thanks to the previous answers, I can install it successfully. Here is my experience:
Environment
torchv2.0.0 + cuda11.7 on Ubuntu

I meet error as ModuleNotFoundError: No module named 'torch', then I install as pip install flash-attn --no-build-isolation
It raises another error as ModuleNotFoundError: No module named 'packaging', then I install this package as pip install packaging
re-run the installation, another error comes RuntimeError: The current installed version of g++ (4.8.5) is less than the minimum required version by CUDA 11.7 (6.0.0). Please make sure to use an adequate version of g++ (>=6.0.0, <12.0).
I use a higher version of g++9.0, and it finally works

Workaround: install the previous version pip install flash_attn==1.0.5

 how do tackle this

I had the same issue with pip. Workaround was to compile from source, worked as a charm

In [1]: import flash_attn

In [2]: import torch

In [3]: torch.version

Out[3]: '2.0.1+cu117'

In [4]: flash_attn.version

Out[4]: '1.0.6'

I also had the same issue, but my system needs Cuda 12.1 (2x Nvidia L4). so using torch 117 is not an option.
this is also my workaround and it works like a charm.
my system uses Fedora Server

What was your solution for cuda 12 and L4 gpu?

I got the same issue here. I only was able to build from the source (clone the repo then run python setup.py install). pip install git+https://github.com/HazyResearch/flash-attention also give me the same error.

try pip install flash-attn --no-build-isolation fixed my problem. pip docs to fix this problem, maybe adding torch dependency into pyproject.toml can help

This fixed the torch problem, but now I got an other error. Might be related to something else tho.
        435 |         function(_Functor&& __f)
            |                                                                                ^
      /usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
      /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
        530 |         operator=(_Functor&& __f)
            |                                                                                ^
      /usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
      error: command '/usr/bin/nvcc' failed with exit code 255

I had the same issue. Could you solve this?

@Martion-z looks the same as #172 and #225. Some version of CUDA doesn't like gcc 11. Downgrading to gcc 10 might work.

conda install -c conda-forge cudatoolkit-dev
pip flash_attn==1.0.5

This woked for me.

pip install flash-attn==1.0.5

Thanks! This solves the above error. But there's still a new one occurs: The detected CUDA version (12.1) mismatches the version that was used to compile.

pip install flash-attn==1.0.5

Thanks! This solves the above error. But there's still a new one occurs: The detected CUDA version (12.1) mismatches the version that was used to compile.

Problem solved. I installed the nightly version of pytorch, and then install flash-attn with the no-build-isolation option.
pip install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu121
# Successfully installed torch-2.1.0.dev20230710+cu121
pip install flash-attn --no-build-isolation
# Successfully built flash-attn
# Installing collected packages: ninja, flash-attn
# Successfully installed flash-attn-1.0.8 ninja-1.11.1

Note the wheel building process takes a long time. Don't kill it and just wait.

There are a lot of things that can go wrong when installing this package.
Next I'm going to share a recipe that should work right now using conda.
Some remarks:

Don't use pip to install any cuda or pytorch libraries
Make sure you don't have several versions of CUDA installed, if that is your case the installation might fail.
Try to install ninja from your distribution's package manager. The process might work without having it installed but it does a much better job detecting the environment.
If you couldn't install it before I strongly recommend to start from a fresh conda environment.
This work with the current torch version and cuda 11.8 no warranties that it will keep working for future versions.

Create a new environment if you don't have already one.
conda create -n flash_attn python=3.10.11
conda activate flash_attn

These are the required packages with the required version.
conda install -c conda-forge gcc=11.3
conda install -c conda-forge gxx=11.3
conda install cuda -c nvidia/label/cuda-11.8.0
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
pip install packaging
pip install flash_attn --no-build-isolation

There are a lot of things that can go wrong when installing this package. Next I'm going to share a recipe that should work right now using conda.
Some remarks:

Don't use pip to install any cuda or pytorch libraries  (this include any package that might reference them before having them installed via conda)
Make sure you don't have several versions of CUDA installed, if that is your case the installation might fail.
Try to install ninja from your distribution's package manager. The process might work without having it installed but it does a much better job detecting the environment.
This work with the current torch version and cuda 11.8 no warranties that it will keep working for future versions.
If you couldn't install it before I strongly recommend to start from a fresh conda environment.

Create a new environment if you don't have already one.
conda create -n flash_attn python=3.10.11
conda activate flash_attn

These are the required packages with their required versions.
conda install -c conda-forge gcc=11.3
conda install -c conda-forge gxx=11.3
conda install cuda -c nvidia/label/cuda-11.8.0
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
pip install packaging
pip install flash_attn --no-build-isolation

Thanks for your kind and detailed recipe! But I still meet this problem...
Collecting flash_attn
  Using cached flash_attn-1.0.9.tar.gz (1.8 MB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [21 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-57_5gf64/flash-attn_939101cdc95c431a947f582f325cfb21/setup.py", line 111, in <module>
          _, bare_metal_version = get_cuda_bare_metal_version(CUDA_HOME)
        File "/tmp/pip-install-57_5gf64/flash-attn_939101cdc95c431a947f582f325cfb21/setup.py", line 26, in get_cuda_bare_metal_version
          raw_output = subprocess.check_output([cuda_dir + "/bin/nvcc", "-V"], universal_newlines=True)
        File "/home/adseadmin/miniconda3/envs/lmtest0/lib/python3.10/subprocess.py", line 421, in check_output
          return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        File "/home/adseadmin/miniconda3/envs/lmtest0/lib/python3.10/subprocess.py", line 503, in run
          with Popen(*popenargs, **kwargs) as process:
        File "/home/adseadmin/miniconda3/envs/lmtest0/lib/python3.10/subprocess.py", line 971, in __init__
          self._execute_child(args, executable, preexec_fn, close_fds,
        File "/home/adseadmin/miniconda3/envs/lmtest0/lib/python3.10/subprocess.py", line 1863, in _execute_child
          raise child_exception_type(errno_num, err_msg, err_filename)
      FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/cuda-11.5/bin/nvcc'

      torch.__version__  = 2.0.1

      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

@jinghan23 See point two, you have several versions of CUDA installed.

The solution I've found working :

Find version of cuda used by torch
Install the right cuda toolkit, for me it was cuda 11.7 -> follow instruction here
Example :

$ wget -d https://developer.download.nvidia.com/compute/cuda/repos/debian11/x86_64/cuda-keyring_1.0-1_all.deb
$ sudo dpkg -i cuda-keyring_1.0-1_all.deb
$ sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/debian11/x86_64/3bf863cc.pub
$ add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/debian11/x86_64/ /"
$ add-apt-repository contrib
$ sudo apt-get update
$ apt-get -y install cuda-11-7

Install g++-10 and gcc-10
Install flash-attn like this : CXX=g++-10 CC=gcc-10 LD=g++-10 pip3 install flash-attn==v1.0.3.post0

I've not tried on other version of flash-attn, but I think it should works too

The workaround that worked for me was to downgrade the cuda runtime version. My driver version is still 12.2 but the runtime version is now 11.7.
conda install -c "nvidia/label/cuda-11.7.0" cuda-toolkit

flash-attn is a very problematic library

you MUST install flash-atten after torch been installed

Solution: Install PyTorch first, then install FA2.
Example:
WRONG:
$ pip install torch flash-attn

CORRECT:
$ pip install torch
$ pip install flash-attn

(source)

Tried adding torch also as a build and not just a runtime dependency in my pyproject.toml. Doubled the install time, but didn't actually work.
My workaround now is to have it as an optional dependency:
[project.optional-dependencies]
# Flash attention cannot be installed alongside normal dependencies,
# since it requires torch during build time. Install with
#     pip install '.[flash-attn]'
# after installing everything else first.
flash-attn = [
    "flash-attn>=2.5.7"
]

And then do two pip installs. One without [flash-attn] and then one with [flash-attn].

pip install flash-attn==1.0.5 might help. I am using torch 1.13 and cuda 12.0.

after I do this, I get this error:
TypeError: MHA.__init__() got an unexpected keyword argument 'num_heads_kv'

We had torch in the dependency in 1.0.5, but for some users it would download a new version of torch instead of using the existing one. I'm not really an expert in Python packaging, so it's possible I'm doing sth wrong.

You are right that omitting in your requirements such an hard to install and popular dependency like torch is the best practice (judging from the wrappers on "classic" ML algos such as xgboost or lightgbm).
The problem here is that your installer tries to import torch, which is a not a good idea, because it fails if unless developers are able to guarantee your exact expected installation sequence: fist torch then flash-attn, which really is the last thing to be expected from batch installation processes or in new environments. This routinely happens in Dockerfiles of GPU-enabled docker containers, because there we install GPU-enabled packages such as torch as the last ones precisely because some of their wrappers still (despite all the educational efforts :) contain CPU-only versions as their requirements and we need to exchange CPU-only torch (tensorflow, xgboost, ...) for the correct GPU-enabled version.

Make sure you're using the latest version of pip, wheel and setuptools.  Then it's fine.
python3 -m venv .venv source .venv/bin/activate python -m pip install --upgrade pip wheel setuptools python -m pip install flash-attn

Does anyone know how we can use the flash_attn current version in T4 GPU? I am using a model that uses flash attn and I am unable to load it in the T4 machine

anyone having problems with macbookpro m3?

Following sequence works:
python -m venv ~/myvenv
source ~/myvenv/bin/activate
pip install torch==2.3.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install wheel
pip install flash-attn --no-build-isolation

Following sequence works:
python -m venv ~/myvenv
source ~/myvenv/bin/activate
pip install torch==2.3.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install wheel
pip install flash-attn --no-build-isolation

It still fails, unfortunately:
❯ pip install flash-attn --no-build-isolation
Collecting flash-attn
  Using cached flash_attn-2.6.3.tar.gz (2.6 MB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [23 lines of output]
      fatal: not a git repository (or any parent up to mount point /)
      Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
      /tmp/pip-install-x4jog5zz/flash-attn_437d70419cfa4722bb9fe12daabe7121/setup.py:95: UserWarning: flash_attn was requested, but nvcc was not found.  Are you sure your environment has nvcc available?  If you're installing within a container from https://hub.docker.com/r/pytorch/pytorch, only images whose names contain 'devel' will provide nvcc.
        warnings.warn(
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-x4jog5zz/flash-attn_437d70419cfa4722bb9fe12daabe7121/setup.py", line 179, in <module>
          CUDAExtension(
        File "/home/koutheir/projects/ml-tts/venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 1077, in CUDAExtension
          library_dirs += library_paths(cuda=True)
                          ^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/koutheir/projects/ml-tts/venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 1204, in library_paths
          if (not os.path.exists(_join_cuda_home(lib_dir)) and
                                 ^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/koutheir/projects/ml-tts/venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 2419, in _join_cuda_home
          raise OSError('CUDA_HOME environment variable is not set. '
      OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.

      torch.__version__  = 2.3.0+cu118

      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

anyone having problems with macbookpro m3?

@ManuelSokolov. Seeing the requirements of the README, it says that this project requires CUDA, so I don't think you can run it with an M3. But also, on the file usage.md it mentions a fork for apple silicon: https://github.com/philipturner/metal-flash-attention

The proposed solutions here are (imho) problematic. Using an nvidia container is a huge dependency and I personally find it difficult to trust a build with no isolation (to not mess up my existing setup) for a library that seems to have a messy installation process.

Dear Professor @tridao and team,
I am having the same issue with these

flash-attn==2.5.6,
causal-conv1d==1.1.3.post1
mamba-ssm==1.1.4

I have torch = 2.1 and CUDA 12.2, I am following this https://github.com/kuleshov-group/mdlm
In my case I solve add in the .toml directly the wheel

mamba-ssm = {url = "https://github.com/state-spaces/mamba/releases/download/v2.2.2/mamba_ssm-2.2.2+cu122torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl"}
flash-attn = {url = "https://github.com/Dao-AILab/flash-attention/releases/download/v2.5.6/flash_attn-2.5.6+cu122torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl"}
causal-conv1d = {url = "https://github.com/Dao-AILab/causal-conv1d/releases/download/v1.1.3.post1/causal_conv1d-1.1.3.post1+cu122torch2.1cxx11abiTRUE-cp310-cp310-linux_x86_64.whl"}

I hope it helps for all the community.


⚠ Bofrost.be has degraded performance

In 29b55e0, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 3397 ms

Resolved: Bofrost.be performance has improved in 1ce5c61.


Error: "Locale not found" when set custom basePath in nextjs config

When basePath: '', everything is fine but when I set for example basePath: '/abc', I get an error on route '/abc/en'

1 of 1 unhandled error
Next.js is up to date

Unhandled Runtime Error
Error: Locale not found

Call Stack
eval
node_modules/next-international/dist/app/client/index.js (1:3060)
mountMemo
node_modules/next/dist/compiled/react-dom-experimental/cjs/react-dom.development.js (12161:0)
Object.useMemo
node_modules/next/dist/compiled/react-dom-experimental/cjs/react-dom.development.js (12817:0)
useMemo
node_modules/next/dist/compiled/react-experimental/cjs/react.development.js (1813:0)
eval
node_modules/next-international/dist/app/client/index.js (1:3000)
e
node_modules/next-international/dist/app/client/index.js (1:1165)
renderWithHooks
node_modules/next/dist/compiled/react-dom-experimental/cjs/react-dom.development.js (10855:0)
mountIndeterminateComponent
node_modules/next/dist/compiled/react-dom-experimental/cjs/react-dom.development.js (16491:0)
beginWork$1
node_modules/next/dist/compiled/react-dom-experimental/cjs/react-dom.development.js (18088:0)
beginWork
node_modules/next/dist/compiled/react-dom-experimental/cjs/react-dom.development.js (26476:0)
performUnitOfWork
node_modules/next/dist/compiled/react-dom-experimental/cjs/react-dom.development.js (25327:0)
workLoopSync
node_modules/next/dist/compiled/react-dom-experimental/cjs/react-dom.development.js (25043:0)
renderRootSync
node_modules/next/dist/compiled/react-dom-experimental/cjs/react-dom.development.js (24998:0)
recoverFromConcurrentError
node_modules/next/dist/compiled/react-dom-experimental/cjs/react-dom.development.js (24229:0)
performConcurrentWorkOnRoot
node_modules/next/dist/compiled/react-dom-experimental/cjs/react-dom.development.js (24135:0)
workLoop
node_modules/next/dist/compiled/scheduler-experimental/cjs/scheduler.development.js (261:0)
flushWork
node_modules/next/dist/compiled/scheduler-experimental/cjs/scheduler.development.js (230:0)
MessagePort.performWorkUntilDeadline
node_modules/next/dist/compiled/scheduler-experimental/cjs/scheduler.development.js (537:0)

Interesting bug. I think this issue might be solved by https://github.com/QuiiBz/next-international/pull/83 (the "Replace usePathname with useParams part")
It's not yet released but will be soon, see: https://github.com/QuiiBz/next-international/issues/62#issuecomment-1659633372

https://github.com/QuiiBz/next-international/pull/83 fixed this error :)


How to make hydrogen use a specific Jupyter console?

I am sorry if its a completely stupid question provided it even makes sense!! I have two different python-2.7s installed on a Mac. The two pythons share the version but differ in certain c libraries. I have two corresponding Jupyter consoles which by default call the two pythons, respectively. How do I make hydrogen bind to a specific Jupyter console  (or equivalently make use of a specific python-2.7 out of the two on my machine)?
Please let me know if the question needs clarification. I apologize for possibly not using the technical terms like 'console', etc. correctly.

I apologize for possibly not using the technical terms like 'console', etc. correctly.

That's ok, there's a lot of terminology here to get through.
How do you have these separate python setups installed? If you have separate environments, you'll want to create a separate kernel installation for each (so that Hydrogen can pick from them). You can create this from the command line by running:
/path/to/python -m pip install ipykernel
/path/to/python -m ipykernel install --user --name SomeNameForEnv

This way they're both able to be selected by name.

How do you have these separate python setups installed?

Let me give an answer in layman terms: I have two python2.7 , let me call them A and B respectively. A is native python2.7 that is preinstalled with MacOS at /usr/bin/python2.7. B was installed using macports at /opt/local/bin/python2.7. Jupyter was installed twice as follows
/usr/local/bin/pip install jupyter
and
/opt/local/bin/pip-2.7 install jupyter.
Then, atom was downloaded from the atom website and installed, followed by installation of Hydrogen via atom package manager. Ideally I would like to have the freedom to use Hydrogen with either of /usr/bin/python2.7 and /opt/local/bin/python2.7 but it seems by default it assumes /usr/bin/python2.7.
To give perspective, let me say that when I run /opt/local/bin/jupyter-2.7 notebook  in the terminal I work with  /opt/local/bin/python2.7 and access all the installed libraries in the notebook, instead when I run /usr/local/bin/jupyter notebook  I work with /usr/bin/python2.7 and can correspondingly call all the installed libraries with this python version.
I apologize for not following your answer. I thought therefore, I should perhaps give a more detailed description of my issue incase I wasn't clear to begin with. It is possible that your aforesaid solution solves my problem- in that case I would be grateful if you could explain your solution in layman terms so I that I can implement it without screwing up existing workflow to access python installations by calling different jupyter notebooks.
Thanks

Thanks for making the write up. It helps for me to see the way in which you want this to work and where the gap is. Sadly, the technical answer to this isn't very pleasing. 😅
Using $PATH to your advantage
Hydrogen will only be able to launch one python by default. It can only run the python it sees on your $PATH. Hydrogen makes a best faith effort to use your default Python kernel (the runtime) for your interactive Python session. If you want to get things working without the approach I suggested originally, there is another workaround. You will have to run atom . explicitly with your PATH including /opt/local/bin/ at the beginning like this:
PATH=/opt/local/bin:$PATH atom .

You have to fully quit atom (at least on the mac) every time you want to switch to your other python environment though. For this reason, I highly recommend learning about Jupyter's kernel installations and how they can benefit you.
Kernel specifications (kernel specs)
To make it so your notebook server, nteract, Hydrogen, or any other frontend can run any language runtime (kernel), the concept of a kernel specification was created. These are installed into a known directory. For example, on my mac they're in ~/Library/Jupyter/kernels:
/Users/kylek/Library/Jupyter/kernels/ir:
kernel.js      kernel.json    logo-64x64.png

/Users/kylek/Library/Jupyter/kernels/python2:
kernel.json    logo-32x32.png logo-64x64.png

/Users/kylek/Library/Jupyter/kernels/python3:
kernel.json    logo-32x32.png logo-64x64.png

/Users/kylek/Library/Jupyter/kernels/scala:
kernel.json    launcher.jar   logo-64x64.png

Here's what is in my python3 kernel's kernel.json:
{
 "argv": [
  "/usr/local/opt/python3/bin/python3.5",
  "-m",
  "ipykernel",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Python 3",
 "language": "python"
}

This provides a description for Hydrogen to know how to launch this specific kernel. When I try to run Python code with Hydrogen, I get to choose which runtime I want:

Note that it shows both the Python 2 and Python 3 kernel. This is based on the current language shown inside of Atom as well as the fact that the kernel spec has a language key of python. What shows in the Atom menu is the display_name from the kernel spec.
There's an excellent post from @mottalrd on installing kernel specs per environment (what I would call your two python installations). It hopefully gives another perspective on this that I can't provide, as well as the means to set up these kernel specs.

Hey! thanks for the elaborate reply! Your first solution worked like a charm- but it was your last post that gave me the understanding and the eventual confidence to implement your first solution. Thanks a lot.


Add info about Cargo.lock.msrv to Contributing.md

This is so that contributors (external or internal) are able to find information about how to deal with failures in min_rust CI job.
Fixes: #912
Pre-review checklist

[ ] I have split my patch into logically separate commits.
[ ] All commit messages clearly explain what they change and why.
[ ] I added relevant tests for new features and bug fixes.
[ ] All commits compile, pass static checks and pass test.
[ ] PR description sums up the changes and reasons why they should be introduced.
[ ] I have provided docstrings for the public items that I want to introduce.
[ ] I have adjusted the documentation in ./docs/source/.
[ ] I added appropriate Fixes: annotations to PR description.

v2:

Addressed review comments
Unrelated small change: removed unnecessary backtick, as it was breaking syntax highlighting


🚀 Feature: (possibly) add eslint-plugin-github-commit-hash

Bug Report Checklist

[X] I have tried restarting my IDE and the issue persists.
[X] I have pulled the latest main branch of the repository.
[X] I have searched for related issues and found none that matched my issue.

Overview
Hey! I learned a lot from this repo regards to making an eslint plugin, and especially one regarding the package.json. That is why the general structure of my code should be quite similar and might nicely integrate into the current project.
I'm not sure if this linting rule is something that fits the package, but since I took quite some inspiration from this repo id seemed fair to mention it here.
Additional Info
No response

Hey thanks, I'm really happy to hear this repo has helped you learn things! ❤
For https://github.com/JustSamuel/eslint-plugin-github-commit-hash: is there a reason why that's a better practice? I rarely use Git specifiers in package.jsons myself (and think that most folks don't?) so I'm not familiar with best practices around them.

Using Git specifiers is not super common, but not unheard of. For example see angular-ci, which also uses the commit hash.
The reason why it is a best practice to include the commit hash is that otherwise your dependency would be "unversioned" and npm i would download the latest commit.  The package-lock also uses the commit hash as a version.
Linking dependencies via Git is also a nice alternative to npm or to hackily setup private package repo's by putting them on GitHub.
But I do admit that this is a very niche case and might not be super applicable to include in the eslint-plugin.

Gotcha, thanks for the explanation! Yeah, it's pretty niche - but if it's a universally agreed-upon best practice then maybe it makes sense here? Leaving in status: in discussion for a while to see if anybody else has inputs.

yarn now requires git dependencies to be the tag, branch or the full 40-characters commit hash. So if this is incorporated, it would be wise to switch for the shortened 7 I used to the full 40.

Just following up, it's been about half a year and I haven't heard of any more strong motivations to add this rule. I think for now it's best to treat it as a niche (but still valid!) use case. Closing out for this plugin.
Thanks for the discussion though! I'm glad https://github.com/JustSamuel/eslint-plugin-github-commit-hash exists for folks that need it. 🙂


Audit logs page

Currently we forward all commands executed to discord.
We'd like to also store those commands in the database and show them on a dedicated page.
Ideally when commands are applied to a specific player we'd like to also have a logical link, but that might prove tricky as API users don't leave in the the same space as the kick/ban commands

done


migration: fix stateful restore

With the new storage api work done {m,um}ounting containers is done on demand
so we need to ensure that storage is available at the right time. In the
restore case, when the container is running it is stopped which will also
unmounted the storage volume for e.g. the zfs backend. So let's make sure that
it is mounted again right after.
Signed-off-by: Christian Brauner<EMAIL_ADDRESS>

Failures look unrelated but I'm surprised we got so many... Looks like there's at least some DB flakiness going on with the import tests.

@stgraber, the flakiness comes from using sqlite3 to access the db which leads to db locked errors. I looked this up and it seems to be a known issue. I'm not sure what we can do about this in the test-suite or if it is even worth it. We might be able to set a 1s wait before calling sqlite3 after LXD is done accessing the db.

@brauner you could kill LXD, modify the DB and spawn it again

Would need a shutdown_lxd() and respawn_lxd() method in main.sh I think to update the PID in the pid file and so on.

Yeah, we do that by hand for the "lxd shutdown" test already


reaction install

Expected behavior
Actual behavior
(HTTP code 404) unexpected - {"message":"manifest for reactioncommerce/reaction:latest not found"}
Information about the Issue
Steps to reproduce the behavior

Install Reaction

Docker Hub is hosted on AWS s3, which is currently down:
https://status.aws.amazon.com/
Sorry for the inconvenience.


How to detect from a video file

Hi @SKEW002 I used a package to read the video file and stream it as an Image topic which can be given to the darknet_ros.
You can check this: http://wiki.ros.org/video_stream_opencv

@SKEW002 Please report on the problem status so far. Did @deigocdl's solution work for you?

Assuming @diegocdl's solution worked. If not, reopen.


Using Resnet50 for FRCN

Sorry to bother you again, I have tried to repalce the VGG to Resnet50 ,so I add two layers: add_layer and batch_normalization layer:
@layer
def add(self,input,name):
    return tf.add(input[0],input[1])

@layer
def batch_normalization(self,input,name,relu=True,is_training=False):
    if relu:
        temp_layer=tf.contrib.layers.batch_norm(input,scale=True,center=True,is_training=is_training,scope=name)
        return tf.nn.relu(temp_layer)
    else:
        return tf.contrib.layers.batch_norm(input,scale=True,center=True,is_training=is_training,scope=name)

the model file is convert from the caffe-tensorflow project on the  github(you can search this on the github),  and I convert the bn_layer's data in the caffe to the moving_average and so on , but the result is really really bad ,map is about 55% , and I don't the reason ,maybe the model is not convert right?
Have you tried to replace VGG to resnet50
below is the resnet50_train.py:
class Resnet50_train(Network):
def init(self, trainable=True):
self.inputs = []
self.data = tf.placeholder(tf.float32, shape=[None, None, None, 3], name='data')
self.im_info = tf.placeholder(tf.float32, shape=[None, 3], name='im_info')
self.gt_boxes = tf.placeholder(tf.float32, shape=[None, 5], name='gt_boxes')
self.gt_ishard = tf.placeholder(tf.int32, shape=[None], name='gt_ishard')
self.dontcare_areas = tf.placeholder(tf.float32, shape=[None, 4], name='dontcare_areas')
self.keep_prob = tf.placeholder(tf.float32)
self.layers = dict({'data':self.data, 'im_info':self.im_info, 'gt_boxes':self.gt_boxes,
'gt_ishard': self.gt_ishard, 'dontcare_areas': self.dontcare_areas})
self.trainable = trainable
self.setup()
def setup(self):

    n_classes = cfg.NCLASSES
    # anchor_scales = [8, 16, 32]
    anchor_scales = cfg.ANCHOR_SCALES
    _feat_stride = [16, ]
    (self.feed('data')
         .conv(7, 7, 64, 2, 2, relu=False, name='conv1')
         .batch_normalization(relu=True, name='bn_conv1', is_training=False)
         .max_pool(3, 3, 2, 2, padding='VALID',name='pool1')
         .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res2a_branch1')
         .batch_normalization(name='bn2a_branch1',is_training=False,relu=False))

    (self.feed('pool1')
         .conv(1, 1, 64, 1, 1, biased=False, relu=False, name='res2a_branch2a')
         .batch_normalization(relu=True, name='bn2a_branch2a',is_training=False)
         .conv(3, 3, 64, 1, 1, biased=False, relu=False, name='res2a_branch2b')
         .batch_normalization(relu=True, name='bn2a_branch2b',is_training=False)
         .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res2a_branch2c')
         .batch_normalization(name='bn2a_branch2c',is_training=False,relu=False))

    (self.feed('bn2a_branch1',
               'bn2a_branch2c')
         .add(name='res2a')
         .relu(name='res2a_relu')
         .conv(1, 1, 64, 1, 1, biased=False, relu=False, name='res2b_branch2a')
         .batch_normalization(relu=True, name='bn2b_branch2a',is_training=False)
         .conv(3, 3, 64, 1, 1, biased=False, relu=False, name='res2b_branch2b')
         .batch_normalization(relu=True, name='bn2b_branch2b',is_training=False)
         .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res2b_branch2c')
         .batch_normalization(name='bn2b_branch2c',is_training=False,relu=False))

    (self.feed('res2a_relu',
               'bn2b_branch2c')
         .add(name='res2b')
         .relu(name='res2b_relu')
         .conv(1, 1, 64, 1, 1, biased=False, relu=False, name='res2c_branch2a')
         .batch_normalization(relu=True, name='bn2c_branch2a',is_training=False)
         .conv(3, 3, 64, 1, 1, biased=False, relu=False, name='res2c_branch2b')
         .batch_normalization(relu=True, name='bn2c_branch2b',is_training=False)
         .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res2c_branch2c')
         .batch_normalization(name='bn2c_branch2c',is_training=False,relu=False))

    (self.feed('res2b_relu',
               'bn2c_branch2c')
         .add(name='res2c')
         .relu(name='res2c_relu')
         .conv(1, 1, 512, 2, 2, biased=False, relu=False, name='res3a_branch1', padding='VALID')
         .batch_normalization(name='bn3a_branch1',is_training=False,relu=False))

    (self.feed('res2c_relu')
         .conv(1, 1, 128, 2, 2, biased=False, relu=False, name='res3a_branch2a', padding='VALID')
         .batch_normalization(relu=True, name='bn3a_branch2a',is_training=False)
         .conv(3, 3, 128, 1, 1, biased=False, relu=False, name='res3a_branch2b')
         .batch_normalization(relu=True, name='bn3a_branch2b',is_training=False)
         .conv(1, 1, 512, 1, 1, biased=False, relu=False, name='res3a_branch2c')
         .batch_normalization(name='bn3a_branch2c',is_training=False,relu=False))

    (self.feed('bn3a_branch1',
               'bn3a_branch2c')
         .add(name='res3a')
         .relu(name='res3a_relu')
         .conv(1, 1, 128, 1, 1, biased=False, relu=False, name='res3b_branch2a')
         .batch_normalization(relu=True, name='bn3b_branch2a',is_training=False)
         .conv(3, 3, 128, 1, 1, biased=False, relu=False, name='res3b_branch2b')
         .batch_normalization(relu=True, name='bn3b_branch2b',is_training=False)
         .conv(1, 1, 512, 1, 1, biased=False, relu=False, name='res3b_branch2c')
         .batch_normalization(name='bn3b_branch2c',is_training=False,relu=False))

    (self.feed('res3a_relu',
               'bn3b_branch2c')
         .add(name='res3b')
         .relu(name='res3b_relu')
         .conv(1, 1, 128, 1, 1, biased=False, relu=False, name='res3c_branch2a')
         .batch_normalization(relu=True, name='bn3c_branch2a',is_training=False)
         .conv(3, 3, 128, 1, 1, biased=False, relu=False, name='res3c_branch2b')
         .batch_normalization(relu=True, name='bn3c_branch2b',is_training=False)
         .conv(1, 1, 512, 1, 1, biased=False, relu=False, name='res3c_branch2c')
         .batch_normalization(name='bn3c_branch2c',is_training=False,relu=False))

    (self.feed('res3b_relu',
               'bn3c_branch2c')
         .add(name='res3c')
         .relu(name='res3c_relu')
         .conv(1, 1, 128, 1, 1, biased=False, relu=False, name='res3d_branch2a')
         .batch_normalization(relu=True, name='bn3d_branch2a',is_training=False)
         .conv(3, 3, 128, 1, 1, biased=False, relu=False, name='res3d_branch2b')
         .batch_normalization(relu=True, name='bn3d_branch2b',is_training=False)
         .conv(1, 1, 512, 1, 1, biased=False, relu=False, name='res3d_branch2c')
         .batch_normalization(name='bn3d_branch2c',is_training=False,relu=False))

    (self.feed('res3c_relu',
               'bn3d_branch2c')
         .add(name='res3d')
         .relu(name='res3d_relu')
         .conv(1, 1, 1024, 2, 2, biased=False, relu=False, name='res4a_branch1', padding='VALID')
         .batch_normalization(name='bn4a_branch1',is_training=False,relu=False))

    (self.feed('res3d_relu')
         .conv(1, 1, 256, 2, 2, biased=False, relu=False, name='res4a_branch2a', padding='VALID')
         .batch_normalization(relu=True, name='bn4a_branch2a',is_training=False)
         .conv(3, 3, 256, 1, 1, biased=False, relu=False, name='res4a_branch2b')
         .batch_normalization(relu=True, name='bn4a_branch2b',is_training=False)
         .conv(1, 1, 1024, 1, 1, biased=False, relu=False, name='res4a_branch2c')
         .batch_normalization(name='bn4a_branch2c',is_training=False,relu=False))

    (self.feed('bn4a_branch1',
               'bn4a_branch2c')
         .add(name='res4a')
         .relu(name='res4a_relu')
         .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res4b_branch2a')
         .batch_normalization(relu=True, name='bn4b_branch2a',is_training=False)
         .conv(3, 3, 256, 1, 1, biased=False, relu=False, name='res4b_branch2b')
         .batch_normalization(relu=True, name='bn4b_branch2b',is_training=False)
         .conv(1, 1, 1024, 1, 1, biased=False, relu=False, name='res4b_branch2c')
         .batch_normalization(name='bn4b_branch2c',is_training=False,relu=False))

    (self.feed('res4a_relu',
               'bn4b_branch2c')
         .add(name='res4b')
         .relu(name='res4b_relu')
         .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res4c_branch2a')
         .batch_normalization(relu=True, name='bn4c_branch2a',is_training=False)
         .conv(3, 3, 256, 1, 1, biased=False, relu=False, name='res4c_branch2b')
         .batch_normalization(relu=True, name='bn4c_branch2b',is_training=False)
         .conv(1, 1, 1024, 1, 1, biased=False, relu=False, name='res4c_branch2c')
         .batch_normalization(name='bn4c_branch2c',is_training=False,relu=False))

    (self.feed('res4b_relu',
               'bn4c_branch2c')
         .add(name='res4c')
         .relu(name='res4c_relu')
         .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res4d_branch2a')
         .batch_normalization(relu=True, name='bn4d_branch2a',is_training=False)
         .conv(3, 3, 256, 1, 1, biased=False, relu=False, name='res4d_branch2b')
         .batch_normalization(relu=True, name='bn4d_branch2b',is_training=False)
         .conv(1, 1, 1024, 1, 1, biased=False, relu=False, name='res4d_branch2c')
         .batch_normalization(name='bn4d_branch2c',is_training=False,relu=False))

    (self.feed('res4c_relu',
               'bn4d_branch2c')
         .add(name='res4d')
         .relu(name='res4d_relu')
         .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res4e_branch2a')
         .batch_normalization(relu=True, name='bn4e_branch2a',is_training=False)
         .conv(3, 3, 256, 1, 1, biased=False, relu=False, name='res4e_branch2b')
         .batch_normalization(relu=True, name='bn4e_branch2b',is_training=False)
         .conv(1, 1, 1024, 1, 1, biased=False, relu=False, name='res4e_branch2c')
         .batch_normalization(name='bn4e_branch2c',is_training=False,relu=False))

    (self.feed('res4d_relu',
               'bn4e_branch2c')
         .add(name='res4e')
         .relu(name='res4e_relu')
         .conv(1, 1, 256, 1, 1, biased=False, relu=False, name='res4f_branch2a')
         .batch_normalization(relu=True, name='bn4f_branch2a',is_training=False)
         .conv(3, 3, 256, 1, 1, biased=False, relu=False, name='res4f_branch2b')
         .batch_normalization(relu=True, name='bn4f_branch2b',is_training=False)
         .conv(1, 1, 1024, 1, 1, biased=False, relu=False, name='res4f_branch2c')
         .batch_normalization(name='bn4f_branch2c',is_training=False,relu=False))

    (self.feed('res4e_relu',
               'bn4f_branch2c')
         .add(name='res4f')
         .relu(name='res4f_relu'))

    #========= RPN ============
    (self.feed('res4f_relu')
         .conv(3,3,512,1,1,name='rpn_conv/3x3')
         .conv(1,1,len(anchor_scales)*3*2 ,1 , 1, padding='VALID', relu = False, name='rpn_cls_score'))

    (self.feed('rpn_cls_score', 'gt_boxes', 'gt_ishard', 'dontcare_areas', 'im_info')
         .anchor_target_layer(_feat_stride, anchor_scales, name = 'rpn-data' ))
    # Loss of rpn_cls & rpn_boxes

    (self.feed('rpn_conv/3x3')
         .conv(1,1,len(anchor_scales)*3*4, 1, 1, padding='VALID', relu = False, name='rpn_bbox_pred'))

    #========= RoI Proposal ============
    (self.feed('rpn_cls_score')
         .spatial_reshape_layer(2, name = 'rpn_cls_score_reshape')
         .spatial_softmax(name='rpn_cls_prob'))

    (self.feed('rpn_cls_prob')
         .spatial_reshape_layer(len(anchor_scales)*3*2, name = 'rpn_cls_prob_reshape'))

    (self.feed('rpn_cls_prob_reshape','rpn_bbox_pred','im_info')
         .proposal_layer(_feat_stride, anchor_scales, 'TRAIN',name = 'rpn_rois'))

    (self.feed('rpn_rois','gt_boxes', 'gt_ishard', 'dontcare_areas')
         .proposal_target_layer(n_classes,name = 'roi-data'))

    #========= RCNN ============
    (self.feed('res4f_relu','roi-data')
         .roi_pool(7,7,1.0/16,name='res5a_branch2a_roipooling')
         .conv(1, 1, 512, 2, 2, biased=False, relu=False, name='res5a_branch2a', padding='VALID')
         .batch_normalization(relu=True, name='bn5a_branch2a',is_training=False)
         .conv(3, 3, 512, 1, 1, biased=False, relu=False, name='res5a_branch2b')
         .batch_normalization(relu=True, name='bn5a_branch2b',is_training=False)
         .conv(1, 1, 2048, 1, 1, biased=False, relu=False, name='res5a_branch2c')
         .batch_normalization(name='bn5a_branch2c',is_training=False,relu=False))

    (self.feed('res5a_branch2a_roipooling')
         .conv(1,1,2048,2,2,biased=False, relu=False, name='res5a_branch1', padding='VALID')
         .batch_normalization(name='bn5a_branch1',is_training=False,relu=False))

    (self.feed('bn5a_branch2c','bn5a_branch1')
         .add(name='res5a')
         .relu(name='res5a_relu')
         .conv(1, 1, 512, 1, 1, biased=False, relu=False, name='res5b_branch2a')
         .batch_normalization(relu=True, name='bn5b_branch2a',is_training=False)
         .conv(3, 3, 512, 1, 1, biased=False, relu=False, name='res5b_branch2b')
         .batch_normalization(relu=True, name='bn5b_branch2b',is_training=False)
         .conv(1, 1, 2048, 1, 1, biased=False, relu=False, name='res5b_branch2c')
         .batch_normalization(name='bn5b_branch2c',is_training=False,relu=False))
    #pdb.set_trace()
    (self.feed('res5a_relu',
               'bn5b_branch2c')
         .add(name='res5b')
         .relu(name='res5b_relu')
         .conv(1, 1, 512, 1, 1, biased=False, relu=False, name='res5c_branch2a')
         .batch_normalization(relu=True, name='bn5c_branch2a',is_training=False)
         .conv(3, 3, 512, 1, 1, biased=False, relu=False, name='res5c_branch2b')
         .batch_normalization(relu=True, name='bn5c_branch2b',is_training=False)
         .conv(1, 1, 2048, 1, 1, biased=False, relu=False, name='res5c_branch2c')
         .batch_normalization(name='bn5c_branch2c',is_training=False,relu=False))
    #pdb.set_trace()
    (self.feed('res5b_relu',
    	       'bn5c_branch2c')
         .add(name='res5c')
         .relu(name='res5c_relu')
         .fc(n_classes, relu=False, name='cls_score')
         .softmax(name='cls_prob'))

    (self.feed('res5c_relu')
         .fc(n_classes*4, relu=False, name='bbox_pred'))

@miraclebiu Thank you for this work.
ResNet has batch_norm layers that keeps a moving_avg and moving_var during each iter. And it works only for mini_batch > 1 (see the orginal paper). In our case, the batchsize is 1. so the moving-avg and moving_var is meanlingless.
To better use ResNet, I think you should keep the bn layer fixed, skipping re-computing mean and var during training.

looking forward for your latest result.

I read your code agian and find you've disabled the updating of bn, by passing is_training=False.
Then I'm not sure about the causes.
Some suggestions:

Try a differen learning rate.
Disable updating of resnet layers, only fine-tuning new layers, like rpn_xx, fc layers.
Add  a histogram summary of each layer to make sure there is no large difference in magnitude between the resnet and new layers

There is a resnet implement, https://github.com/ry/tensorflow-resnet, and also the pretrained models.

I will try and if I can find the error, I will tell you

This evening, I turn my code from the Faster-rcnn_TF to your project ,and I train the resnet50 for about 130k iterations ,the test performance  map is about 70%,maybe is the Faster-rcnn_TF errors that I didn't find .In my previous opinion, since this project can train the VGG well ,so it can train resnet50 well too But I think I'm wrong. Besides , I also get the code about  the psroipooling in R-FCN and implement in the tensorflow ,maybe I can send to you .

I'm new about tensorflow and the FRCN, so I think I should read your code carefully and learn how to use it .

@miraclebiu Congratulations! Very glad you made it. You can try pull request to send you awesome code to this repo. check this link https://help.github.com/articles/about-pull-requests/
And there are something very confusing in the original code, like the inconsistency of axis. some where the rpn_score is ordered by (1, H, W, A, 2) and some where (1, H, W, 2, A).

I write an email to your gmail.
I don't know how to pull request the file since I'm a newcomer about github, all I know is how to download the code.Besides, the tensorflow machine and my computer is not the same machine , I need to copy the file to this computer, so I create the reposity about TFFRCN_Resnet50,you can see the readme.md, I only change the factory.py, init.py, networks.py  in ./lib/networks and add resnet50_train and resnet50_test.py in the same folder.
Thank you for your work on this project!
the website is
https://github.com/miraclebiu/TFFRCN_resnet50

Hi,
Following the training process, I see a lot of
ckpt.meta/.index/.data-00000-of-00001 file generated in output folder. How
do I get plain .ckpt file?
Feeded both *.ckpt.meta and *.ckpt.data-00000-of-00001 to demo and
test_net.py, didn't work. How do you get plain .ckpt weight file? Caffe
will produce both a useable caffemodel and solverstate file.
Thanks,
Dennis
On Fri, Dec 2, 2016 at 5:25 AM, Minyue Jiang<EMAIL_ADDRESS>wrote:

I write an email to your gmail.
I don't know how to pull request the file since I'm a newcomer about
github, all I know is how to download the code.Besides, the tensorflow
machine and my computer is not the same machine , I need to copy the file
to this computer, so I create the reposity about TFFRCN_Resnet50,you can
see the readme.md, I only change the factory.py, init.py, networks.py
in ./lib/networks and add resnet50_train and resnet50_test.py in the same
folder.
Thank you for your work on this project!
the website is
https://github.com/miraclebiu/TFFRCN_resnet50
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/CharlesShang/TFFRCNN/issues/7#issuecomment-264453206,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAM4K3wdicJsuIoFjQf_1xNgYnH_WyOvks5rEBxKgaJpZM4LBAdL
.

Seems there is an issue/bug somewhere. Checked lib/fast_rcnn/train.py and
it should write .ckpt file, added a check to see the file exists at line
84, indeed the ckpt file does not exist after invocation of
self.save.save(sees, filename) statement. Running latest tensorflow 0.12rc0
on ubuntu 16.04.1.
On Sun, Dec 4, 2016 at 6:09 PM, Dennis Su<EMAIL_ADDRESS>wrote:

Hi,
Following the training process, I see a lot of
ckpt.meta/.index/.data-00000-of-00001 file generated in output folder.
How do I get plain .ckpt file?
Feeded both *.ckpt.meta and *.ckpt.data-00000-of-00001 to demo and
test_net.py, didn't work. How do you get plain .ckpt weight file? Caffe
will produce both a useable caffemodel and solverstate file.
Thanks,
Dennis
On Fri, Dec 2, 2016 at 5:25 AM, Minyue Jiang<EMAIL_ADDRESS>wrote:

I write an email to your gmail.
I don't know how to pull request the file since I'm a newcomer about
github, all I know is how to download the code.Besides, the tensorflow
machine and my computer is not the same machine , I need to copy the file
to this computer, so I create the reposity about TFFRCN_Resnet50,you can
see the readme.md, I only change the factory.py, init.py, networks.py
in ./lib/networks and add resnet50_train and resnet50_test.py in the same
folder.
Thank you for your work on this project!
the website is
https://github.com/miraclebiu/TFFRCN_resnet50
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/CharlesShang/TFFRCNN/issues/7#issuecomment-264453206,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAM4K3wdicJsuIoFjQf_1xNgYnH_WyOvks5rEBxKgaJpZM4LBAdL
.

I run this on 0.11 ,you can check what's changed on 0.12 https://github.com/tensorflow/tensorflow/releases

Did some debugging, found ckpt.* (meta/data/index) is new style of
checkpointing. In the demo.py, once passes in the base .ckpt path, it works
(need to disable the ckpt file pathname check since there is no 1 ckpt
file, but a bunch of suffix files).
Indeed 0.12 release note indicated checkpoint syntax has been changed:

New checkpoint format becomes the default in tf.train.Saver. Old V1
checkpoints continue to be readable; controlled by the write_version
argument, tf.train.Saver now by default writes out in the new V2
format. It significantly reduces the peak memory required and latency
incurred during restore.

On Sun, Dec 4, 2016 at 7:26 PM, Minyue Jiang<EMAIL_ADDRESS>wrote:

I run this on 0.11 ,you can check what's changed on 0.12
https://github.com/tensorflow/tensorflow/releases
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/CharlesShang/TFFRCNN/issues/7#issuecomment-264760108,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAM4K8YlwJySyQHLcV0WwDdh6OqucJgOks5rE4RQgaJpZM4LBAdL
.

@miraclebiu I merge your contributions to resnet branch.
After a simple test, I'll finally merge it to the master.
Looking forward to your resnet101 models.
Thanks again.

@miraclebiu
Added your code. Notice there is psroi_pooling, have you tested this module?
I think we can build R-FCN together.

@CharlesShang  I have tested the forward, I haven't test the backward, I find it on the github and have contrasted it with  the caffe code  and made some modifications，I think it is right.
Besides, I think if someone want to build R-FCN, he should write some code about OHEM which I think is very important for R-FCN. It is just my opinion.
Google have submitted a paper on arXiv about many detectors ,"Speed/accuracy trade-offs for modern convolutional object detectors", I don't know if they use the OHEM for R-FCN. I wish they can release their codes.

@CharlesShang ,maybe you can use tf.nn.top_k or tf.nn.in_top_k to sort the loss and use the tf.gather .
Since the network has four loss and sum them , I don't know which should do the OHEM .

@miraclebiu, online hard example mining (OHEM), data augment, etc., are very important for detection. I'll add these functions in a few days. A sinple way is to check out the R-FCN code and find out how they use OHEM:-).
Thank you for your information about that google paper.

@CharlesShang  the current  map of  Resnet101 model is 70.4% , it has iterated 150k , maybe it need more iterations . The converted  Resnet50 and Resnet 101 model has a small gap about 0.6%  top5 error, so there is no big difference between using Resnet50 and Resnet101 ,but the Resnet101 model is about twice bigger than the Resnet50.

I think it's the best mAP we can get.
Maybe the bottleneck is training data, the origianl paper (FasterRCNN + ResNet101) reported the mAP on 07 was 0.738 using 07+12 for training.

Today I tried Resnet101 in CharlesShang/TFFRCNN (pulled latest codes as of
today from master branch), saw following error in training:
Traceback (most recent call last):
File "./faster_rcnn/train_net.py", line 109, in
restore=bool(int(args.restore)))

File "./faster_rcnn/../lib/fast_rcnn/train.py", line 396, in train_net
sw.train_model(sess, max_iters, restore=restore)

File "./faster_rcnn/../lib/fast_rcnn/train.py", line 108, in train_model
self.net.build_loss(ohem=cfg.TRAIN.OHEM)

File "./faster_rcnn/../lib/networks/network.py", line 446, in build_loss
cls_score = self.get_output('cls_score') # (R, C+1)

File "./faster_rcnn/../lib/networks/network.py", line 80, in get_output
raise KeyError('Unknown layer name fed: %s'%layer)

KeyError: 'Unknown layer name fed: cls_score'

This happens right after "Solving ...." and print of network layer names.
Went back to Minyue's fork, it works fine there.
Another question, I saw you are working on OHEM. Is it working in master
branch? Debug branch has more changes of OHEM.
Thanks
Dennis
On Mon, Dec 5, 2016 at 11:41 PM, Charles Shang<EMAIL_ADDRESS>wrote:

I think it's the best mAP we can get.
Maybe the bottleneck is training data, the origianl paper (FasterRCNN +
ResNet101) reported the mAP on 07 was 0.738 using 07+12 for training.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/CharlesShang/TFFRCNN/issues/7#issuecomment-265082208,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAM4K9spoXwBjlan54alpQbHq_NW3tbIks5rFRG2gaJpZM4LBAdL
.

Just FYI, found a way to unlock Resnet101 potential, in my own dataset, see
full 4.5 percentage improvement over Restnet50.
On Mon, Dec 5, 2016 at 11:08 PM, Minyue Jiang<EMAIL_ADDRESS>wrote:

@CharlesShang https://github.com/CharlesShang the current map of
Resnet101 model is 70.4% , it has iterated 150k , maybe it need more
iterations . The converted Resnet50 and Resnet 101 model has a small gap
about 0.6% top5 error, so there is no big difference between using Resnet50
and Resnet101 ,but the Resnet101 model is about twice bigger than the
Resnet50.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/CharlesShang/TFFRCNN/issues/7#issuecomment-265077293,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAM4KzeyJUTDsIFqQ0HTBJANU6VXpHWgks5rFQnggaJpZM4LBAdL
.


update simple Chinese

設計模式目錄 is traditional Chinese not simple.
the simple is 设计模式目录

Looks good! Thank you for the contribution 🎉


Implement removing category filter

Ideas:

c/!all
a static enum-like field in Category
a private constructor for Category

Implemented but I will group this with other PRs


Can we fine tune the FCN_8s network on our own dataset?

Hi, I was wondering if we can utilize the fcn_8s_train.py file to fine-tuned the FCN8s on our own dataset. It seems that when I do this, there is an incompatible shape problem because my dataset only has 7 classes. Besides, the fcn_8s_train.py file is trained on FCN_16s, am I right? So it might be improper for me to use that script. Do I have to use the tf_recored script to convert my dataset first? Any suggestion is welcome. Thanks.

Did you manage to tune the network with your dataset ? I also want to do that but can't find the right way.

@MrChristo59 I would like to suggest you check on this site: https://github.com/MarvinTeichmann/tensorflow-fcn, which I found out how to fine-tune on my own dataset. I noticed that for FCN, there is no need to have the deconv layer pre-trained, as long as we wanted to change the number of output neurons, we will need to train the deconv layer from stratch.


Why 20000 tasks are set, but only one batch task is used

Hello, I saw this paper and code recently. There are many puzzles in it. I hope someone can give me some advice.
The first question is:In data_ generator.py, why 20000 tasks are set, but only one batch task is used.

Have you figured out the reason?


pre-training time?

I wonder about how long it took to pre-train the model with the CC3M dataset.

Thank you for the interest on our work :)
As I remember, the elapsed time is roughly as follows depending on the number of GPUs available,

GPUs
Vision Encoder Status
Batch Size
Elapsed Time

8 x V100 (1 node)
Frozen
2048
~6 hours

8 x V100 (1 node)
Unfrozen
256
~4 days

64 x V100 (8 node)
Unfrozen
2048
~2 days

If you use the better network, multi-node training would be faster.

Thanks for answering my question!
So are the weights you posted the unfrozen version of Vision Encoder?

Exactly, we uploaded the unfrozen version (vision encoder trained) with 64 x v100 gpus (2048 batch).

thank you!!!


workaround for https://github.com/celery/kombu/issues/838

If we don't explicitly set the timeout we may inherit the temporary default.
This can cause failures when testing services that use AMQP and HTTP entrypoints with the builtin pytest helpers. The line at https://github.com/nameko/nameko/blob/master/nameko/testing/pytest.py#L120 can execute while a new service is starting, triggering the bug, and the listening socket only waits 0.1s for a connection.

There is probably a better workaround here.
I think the bug against kombu is valid, but the change added in https://github.com/nameko/nameko/pull/481 (which is the trigger for the hanging tests) has other problems too:
The kombu pool is shared at the process level and the vhost pipeline destroy runs in a background thread, not synchronously at the end of tests. So when tearing down from a previous test we may be closing publisher connections created by a currently running test. It is a little bit surprising that this hasn't caused other problems.
I think we need a different solution for the problem that #481 tries to address, as well as this workaround.

The alternative discussed in #481 was just to increase the number of file descriptors on the rabbit instance, which now seems much more reasonable than trying to accommodate a particular test suite's requirements with a change to the underlying library. I will revert ecf6c58.

@iky you're right, connection pools are per vhost. Not sure how I missed that. We use a different vhost per test, so the failure here is only a result of the kombu bug. I will revert the revert 😬


Having this issue on devnet

Anyone else having a similar issue?

I'm also having this issue

I'm also having this issue. Just from trying to use the manger at https://www.gemfarm.gg on devnet.

i figured it out, i was using the same rewards for A and B. They have to be different

Thanks @gilfs this was my problem too

i know this isn't specific to gem-farm but im getting this when trying to call InitFarmer.
next-dev.js?3515:32 Transaction simulation failed: Error processing Instruction 0: custom program error: 0xbc0
Program farmL4xeBFVXJqtfxCzU9b28QACM7E2W2ctT6epAjvE invoke [1]
Program log: Instruction: InitFarmer
Program log: AnchorError caused by account: gem_bank. Error Code: InvalidProgramId. Error Number: 3008. Error Message: Program ID was not as expected.
Program farmL4xeBFVXJqtfxCzU9b28QACM7E2W2ctT6epAjvE consumed 15115 of 200000 compute units
Program farmL4xeBFVXJqtfxCzU9b28QACM7E2W2ctT6epAjvE failed: custom program error: 0xbc0


Stellar Overlay Connection integration tests

Resolve #140
Add function to disconnect connection to overlay network

Once we call disconnect we still able call listen method to get message that was already received from stellar node before we called disconnect.
We can add extra logic to send method to prevent sending ConnectorActions::SendMessage(message) via mpsc
@ebma
kind of this options
pub async fn send(&self, message: StellarMessage) -> Result<(), Error> {
        if !self.is_disconected{
                self.actions_sender
		.send(ConnectorActions::SendMessage(message))
		.await
		.map_err(Error::from)
         }
         Ok(())
}

pub async fn send(&self, message: StellarMessage) -> Result<(), Error> {
         if self.is_disconected{
                return Error(...)
         }
         self.actions_sender
	 .send(ConnectorActions::SendMessage(message))
	.await
	.map_err(Error::from)
}

Yeah, sending should not be possible after disconnecting, right?
I think I would prefer the second option from your code proposals. Can you add it to the code?

Yeah, sending should not be possible after disconnecting, right?
I think I would prefer the second option from your code proposals. Can you add it to the code?

Done!

The formatting is off again, please fix that and I will approve afterwards.

Done!


feat: disable nakamoto modal

closes #1623

:tada: This PR is included in version 1.176.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket: