dotenv integration captures quotes in value/improperly stores json

Describe the bug
When using quoted json in .env the resulting environmental variable includes both the quotes and json. If something like "read" in Haskell is called on that environmental variable it will predictably fail.
The reason is because of this regex in the dotenv integration:
(.+) *= *(.+)

Which results in the single quotes being in the resulting value:
nix-repl> builtins.match "(.+) *= *(.+)" "a='{\"a\":\"b\"}'"
builtins.match "(.+) *= *(.+)" "a='{\"a\":\"b\"}'"
[ "a" "'{\"a\":\"b\"}'" ]

I can get the expected behavior by optionally matching ', but this might not be how you want to write it:
nix-repl> builtins.match "(.+) *= *'?(.+)?'" "a='{\"a\":\"b\"}'"
builtins.match "(.+) *= *'?(.+)?'" "a='{\"a\":\"b\"}'"
[ "a" "{\"a\":\"b\"}" ]

Some related issues:
https://github.com/theskumar/python-dotenv/issues/35
https://github.com/theskumar/python-dotenv/issues/285
https://github.com/theskumar/python-dotenv/issues/381
To reproduce
Please provide an Short, Self Contained, Correct (Compilable), Example by creating a gist using devenv.nix, devenv.yaml, and optionally devenv.lock.
Given .env:
data='{"a":"b"}'

Running echo $data with dotenv integration enabled results in:
$ echo $data
'{"a":"b"}'

I would expect only the json value and not the surrounding single quotes to be captured.
Version
Paste the output of $ devenv version here.

https://github.com/cachix/devenv/pull/718 can also fix this issue by offloading the whole parsing logic to the shell. It also means that config.env won't hold variables extracted from the .env file, but it's not like config.env was guaranteed to hold all the environment variables available in devenv shell, since any enterShell script can add its own environment variable.
I was looking into tweaking the regex used in parseLine to make it ignore comments, but I'm beginning to think #718 may be the smarter approach. (Or at least in that general direction - #718 doesn't handle comments well as-is)
@domenkozar What do you think?

I don't know if this needs a new issue or not, but the regex in question also seems to cause problems with values that have = in them. The "(.+) *= *(.+)" regex is greedy so everything up to the last = is captured as the variable name. For example, a typical database url environment variable such as DATABASE_URL=postgresql://postgres:postgres@localhost:5432/postgres?schema=public seems to be read in with a name of DATABASE_URL=postgresql://postgres:postgres@localhost:5432/postgres?schema instead of the expected name of DATABASE_URL.
I will second the ability to have the .env variables accessed in Nix is pretty useful. It lets me easily parameterize parts of an environment variable across environments, such as the port of the database url in the example above.

Yeah, on second thoughts, I can use direnv to source .env if I don't need the env vars to be available in config.env.
I have a regex that can handle comments and values with = somewhat, but the actual values don't get stripped of comments until after they've been exported as environment variables: https://gist.github.com/ento/765b47da99cc05d4fec508632e8d1f64 (and it doesn't handle quotes at all, as demonstrated by the test cases in that gist)

I will second the ability to have the .env variables accessed in Nix is pretty useful.

I am not sure if it is a good idea to access .env variables in nix expression. For example, if you create a script via pkgs.writeShellScript to use env.DATABASE_URL, then your pgsql password is leaked to the nix store, which is visible to all unix users. On the other hand, if your shell script reference it via $DATABASE_URL, your pgsql password would not be leaked to the nix store.

I just tried this again using devenv 1.0.8 with a .env that looks like:
data='{"a":"b"}'
data2="{\"a\":\"b\"}"
data3={"a":"b"}

And it looks like data3 variant works fine, so I also consider this fixed.


Audio Too large?

Hello @muaz-khan
When recording with 2 audio channels, the audio size is doubled, and when I reduced it to 1 audio channel, the audio is choppy when recording using laptop.
Any advice? Thank you!

I just opened issue #104 as I'd found that at least in Chrome I was able to compress audio to OGG when rendering the samples as a blob. Hopefully this will lead to an improvement for you.


Use GitHub Actions for continous integration

This PR replaces our current Travis CI workflow with GitHub Actions.
Features:

Runs on Linux and macOS, on py38 and py39
Uses mamba to install dependencies into a test conda environment
Checks out, builds, and installs the four BMI language examples, then runs tests on them
Lints once, on Linux with py38
Builds docs on Linux with py38

Windows is not tested--we already have a working solution with Appveyor. I'll add an issue recommending we fold the CI on Windows into Actions, as well.

For reference, our Travis CI workflow is preserved in the reference/ci-with-travis branch.


ref-objects component apply object to namespace different from app namespace causes CollectHealthStatus failed

Describe the bug
when i add ref-objects component to the application with an object which namespace different from application, the component apply success but health status failed.
i check the source code and find out that when CollectHealthStatus vela use application's namespace to get object instead of the ref-object's namespace. so the object is not found.
error message as follows:
    message:step deploy: run step(provider=multicluster,do=deploy): Found 1 errors. [(error encountered in cluster local: CollectHealthStatus: app=first-vela-app-pre-bug, comp=configmap, evaluate workload status message error: get template context: failed to get obj first-vela-app-pre with gvk /v1, Kind=ConfigMap : configmaps "first-vela-app-pre" not found)]

To Reproduce
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: first-vela-app-pre-bug
  namespace: dev-test
spec:
  components:
  - type: ref-objects
    name: configmap
    properties:
      objects:
        - resource: configmap
          name: first-vela-app-pre
          namespace: default
          cluster: local
  policies:
  - name: topology-local
    properties:
      clusters:
      - local
    type: topology
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: first-vela-app-pre
  namespace: default

Expected behavior
ref-objects Component could be used to apply objects which namespace is different from the application.
Screenshots

KubeVela Version
CLI Version: v1.4.3
Core Version: 1.4.3
GitRevision: git-5e3ab73
GolangVersion: go1.17.11
Cluster information
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:23:52Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:15:20Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Additional context

The bug you describe is solved in https://github.com/kubevela/kubevela/pull/4349. The next release version will fix them.
BTW, the topology policy you use should add namespace: "dev-test" because you are referencing an object from another namespace. If you do not add this flag, the application will try to dispatch the configmap into the original namespace ("default" in your case).

Feel free to reopen this issue if you find further bugs.

The bug you describe is solved in #4349. The next release version will fix them.
BTW, the topology policy you use should add namespace: "dev-test" because you are referencing an object from another namespace. If you do not add this flag, the application will try to dispatch the configmap into the original namespace ("default" in your case).

thanks for reply. what i want is to just apply one of the ref-objects to another namespace "dev-test" and others stay the same in "default" namespace. this fix may not help?  topology policy may cause all objects to apply to the specified namespace?

The bug you describe is solved in #4349. The next release version will fix them.
BTW, the topology policy you use should add namespace: "dev-test" because you are referencing an object from another namespace. If you do not add this flag, the application will try to dispatch the configmap into the original namespace ("default" in your case).

thanks for reply. what i want is to just apply one of the ref-objects to another namespace "default" and others stay the same in "dev-test" namespace. this fix may not help? topology policy may cause all objects to apply to the specified namespace?

i have try this fix, and i can help. i don't need to specify the namespace in topology policy. thank you.


Introduction chapter: Getting an error on assertTypeCmd in Lean4

I'm getting the following error in the Introduction#Building a command example.
My Lean version is "leanprover/lean4:nightly-2022-10-09".
Does this look familiar?

Oh, the API must have changed. What happens if you don't pass the  `assertTypeCmd argument?

It goes like this:

@lakesare I've bumped the toolchain and fixed that issue: https://github.com/arthurpaulino/lean4-metaprogramming-book/commit/ec355bba83e3d65db57952f13abc23a32cafd009
Make sure to check the indentation levels of your code because the try ... needs to be deeper than liftTermElabM

That worked! Thanks for the note about the nesting level, that was also important.


Mark repo unmaintained or call for maintainers

since there is a bug that make this repo unusable for more than 6 months.
their is many issues open for this bug #75 #72 #66 #55...
This bug make Android app unusable after the use of this package repos which requires manual action to fix it.
The package must be marked as not maintained and deprecated OR call for maintainers.
It is okay to have bug in open source packages. But they should be fixed or be referenced in the readme.
When people are looking for a package it is not to make their app unusable.

@ajcrites It would save a lot of developers their time if this repo is marked unmaintained / allow someone else to maintain this repo

@darshkpatel Hi! If you have some suggestions on how to improve or fix this library feel free a create a PRs.
Unfortunately, not all contributors have free time for working on the lib.


Backward compatibility

Trying to compile on Ubuntu 14.04 with protobuf 2.4.1 gives:
Using libprotoc 2.4.1 from /usr/bin/protoc
geobuf.proto:11:5: Expected "required", "optional", or "repeated".
geobuf.proto:11:21: Missing field number.
geobuf.proto:20:9: Expected "required", "optional", or "repeated".
geobuf.proto:20:23: Missing field number.
geobuf.proto:59:9: Expected "required", "optional", or "repeated".
geobuf.proto:59:26: Missing field number.

I suppose this version does not support the oneof feature. Is there any way I can modify the proto file to work with older versions of protobuf yet still generate the same buffers? Can I simply take out the oneof clause?

Yes, you would just take out the oneof block and add optional keywords to each of its fields.

Thanks that seems to work.


add --swift language option

add --swift CLI option
add swift option to library API (not documented at this point)
update iOS template to generate native module with Swift language implementation, if --swift option is enabled
update CLI test snapshots

resolves #221
only tested with React Native 0.62 at this point
I would like to give thanks to the following resources for some extremely helpful guidance for a Swift beginner like myself:

various comments in #221
https://reactnative.dev/docs/native-modules-ios.html#exporting-swift
https://medium.com/javascript-in-plain-english/react-native-native-modules-with-swift-6768ea03b3f
https://nshipster.com/swift-literals/
https://stackoverflow.com/questions/24074479/how-to-create-a-string-with-format
https://moduscreate.com/blog/swift-modules-for-react-native/

and some other interesting resources I found:

https://github.com/rhdeck/react-native-swift (does not seem to be really needed if we generate one -Bridging-Header.h per native module written in Swift as proposed here
https://teabreak.e-spres-oh.com/swift-in-react-native-the-ultimate-guide-part-1-modules-9bb8d054db03 (it looks like part 2 could potentially help with the native view part)
https://levelup.gitconnected.com/powering-react-native-with-kotlin-swift-support-e5c1210a4e25 (also useful for Kotlin as discussed in #352)

TODO items:

[ ] resolve an ugly "main queue setup" warning message from the Metro bundler
[ ] add test cases with swift option enabled
[ ] support native view with Swift language option
[ ] explicitly reject combination of Swift & Apple Networking options
[ ] test with tvOS
[ ] test with other supported React Native versions (0.60, 0.61, 0.63-rc)
[ ] use Stryker Mutator to check for any missing test cases
[ ] add documentation

/cc @dlowder-salesforce

Hi when are you planning to merge this changes?

I will try to finish and merge this in the near future.

ive been using this via npx brodybits/create-react-native-module#add-swift-option react-native-swift-module --swift --generate-example and works pretty well, i think you should merge it
adding
  static func requiresMainQueueSetup() -> Bool {
    return true
  }

to the Swift file
resolved the  requiresMainQueueSetup issue


Czech translation created

Any chance to get this PR merged?

Oh sorry, I didn't have a lot of time to work on TextFiction lately.
Am Sonntag, den 25.01.2015, 07:21 -0800 schrieb Jaroslav Lichtblau:

Any chance to get this PR merged?
—
Reply to this email directly or view it on GitHub.

No worries about the delay. And thanks for merging it!


chore: add eik setup to release workflow

Implemented eik setup based on our guide "Add support for automated Eik releases"
Also replaced npm with pnpm in the scripts and updated the readme.

 Automate releases of warp packages to NPM and EIK


Add scroll to top button  feature

Is your feature request related to a problem? Please describe.
Add scroll to top button  feature
Describe the solution you'd like
Add scroll to top button  feature
Describe alternatives you've considered
Add scroll to top button  feature
Additional context
Add any other context or screenshots about the feature request here.
i will work on this

Hey @notsoocool
is this issue still open?
i can work on it right way

@coderKrysio just complete the issue assigned to you first, raise a PR, then i will assign you this issue as well.

I would like to contribute to it

@KaroK365 i will assign you this issue.

thanks i will add this feature


FlipClock Console Log

I'm using FlipClock in angular 2 as a clock. How do I stop clock from console.log() current time out. The Source Line on log in the console is (unknown).

If you're using the minified version, Ctrl-F for (a=this.getDateObject()),console.log(a) and remove the ,console.log(a) part. If you're using the full source, just remove all calls to console.log (I think the one in the source is console.log(date). Granted, these should probably be removed in the release libraries...

Yep! already did that... I was trying not to edit the source code... ty anyways

I wouldn't mind seeing the incremented date/time in the console IF an debug option was set in order to output it and the default setting was false (ie, no output).

No longer an issue in the new js-es6 branch.


ImportError: No module named scrapyjs

Could you please help me with that error I am getting when running the script?
ImportError: No module named scrapyjs
I get some other errors too but I hope that are related to that one.
Thank you!

HI @joanpavlidou ,
Thanks for using my repo and asking this question.
My code was 2 years old and the scrapyjs module got renamed to scrapy_splash. Therefore the code was broken.
I've made the changes and updated the Readme.md accordingly. I've tested it and it works now!
Let me know if you run into other issues.

Thank you very much, Louis, for your timely response, your help and your very good work. Yes, it works!!!


Refactor Landing Page Micro-App

This issue outlines the development tasks for the Landing Page Micro-App, designed to serve as the primary user entry point for our application.
Requirements:

Feature Presentation: Showcase key features and functionalities of the application in an engaging and informative manner.
Pricing Plans: Present different subscription options with clear pricing details and potential comparison tables.
Language Selection: Allow users to switch the interface language between supported options.
User Sign-Up/Sign-In: Offer multiple sign-up/sign-in methods:

GitHub login
Google login
Email address with password creation

User Interaction: Enable user navigation through the landing page content and interaction with UI elements.
Conversational Interface (Optional): Explore integration of a conversational interface for user engagement (text-based or voice-enabled).
Microphone Permission Handling: If a conversational interface is implemented, request and manage microphone access permissions from the user.

Tasks:

[ ]  Design User Interface:

Develop a visually appealing and user-friendly landing page layout.
Design clear sections for showcasing features, pricing plans, and sign-up options.
Consider responsive design for optimal viewing on various devices.

[ ]  Develop Feature Presentation:**

Create compelling content to highlight application features and benefits.
Utilize engaging visuals like images, icons, or videos to showcase features.

[ ]  Implement Pricing Plans:**

Design clear and concise pricing tables or sections for different subscription options.
Ensure easy comparison of features and pricing across plans.

[ ]  Develop Language Selection:**

Integrate functionality to switch the landing page interface language.
Provide clear options for supported languages.

[ ]  Implement User Sign-Up/Sign-In:**

Develop user registration form for email-based sign-up with password creation.
Integrate login functionalities for GitHub and Google accounts.
Handle user authentication and session management securely.

[ ]  Develop User Interaction Features:

Implement navigation elements for users to explore different sections of the landing page.
Ensure smooth interactions with UI elements like buttons and links.

[ ]  (Optional) Explore Conversational Interface:**

Research potential text-based or voice-enabled conversational interfaces for user engagement.
Evaluate integration options and potential libraries for this functionality.

[ ]  (Optional) Implement Microphone Permission Handling:

If a conversational interface is implemented, develop logic to request and manage microphone access permissions from the user.
Ensure clear communication regarding microphone usage and respect user privacy.

[ ]  Content Creation:

Develop compelling copywriting for the landing page, including feature descriptions, pricing details, and call-to-action (CTA) elements.

[ ]  Responsiveness and Optimization:

Test and optimize the landing page for optimal performance across different devices and screen sizes.
Ensure fast loading times and a smooth user experience.

[ ]  Unit and Integration Testing:

Develop comprehensive unit and integration tests for various functionalities of the landing page micro-app.

[ ]  Documentation:

Document the landing page functionalities, user interface usage, and configuration options.

Additional Notes:

Consider using a micro-app framework to facilitate development and integration with the main application.
Define success metrics to track user engagement on the landing page (e.g., sign-up rates, time spent on the page).

Hi @mouimet-infinisoft , could u pls assign this to me?

hi could you pls assign this to me?

Hi @mouimet-infinisoft , could u pls assign this to me? and also, if possible, can we come on call or chat so that I can understand how exactly you want the UI to be and I can ask questions directly on the call itself? Could u also provide your availability timing?

hi could you pls assign this to me?

Hi @AryanKuAg and @Adonai-Technologies
Thanks for offering some help. I would be more then happy to take some time for a call with you both. I am pretty much open this week so, let me know when would be the best fit and let's have a call. I will introduce the project to you and we will discuss how we can collaborate together. In the mean time, you can join iBrain One Slack Channel as the call is going to be there. https://join.slack.com/t/ibrain-one/shared_invite/zt-2crym20x2-YyECbi9bK6JUfH6krPR0pw
I am curious, how did you find iBrain project?
Best Regards,
Mart

Yeah sure @mouimet-infinisoft , I've joined!!!!

@mouimet-infinisoft , I've joined.....  feel free to get in touch anytime within the week


update readme to specify python versions, remove unused template file

Pull request recommendations:

[ ] Name your pull request your-development-type/short-description. Ex: feature/read-tiff-files
[ ] Link to any relevant issue in the PR description. Ex: Resolves [gh-12], adds tiff file format support
[ ] Provide relevant tests for your feature or bug fix.
[ ] Provide or update documentation for any feature added by your pull request.

Thanks for contributing!

Codecov Report

Merging #12 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #12   +/-   ##
=======================================
  Coverage   83.28%   83.28%
=======================================
  Files          31       31
  Lines         999      999
=======================================
  Hits          832      832
  Misses        167      167

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c86e35f...f408299. Read the comment docs.


Thanks for s good day

That's the best (and only one lol) license agreement i ever read

Keep making cool stuff!


Feature/express server in main

Why is this change needed?
To add express server sends the video in bytes for video preview. Start express server in the main process.
What did you change?

Created an express server
Add an endpoint for get video in bytes with selected range

Actually just saw your commit moving nodemon to devDependencies. Why do we need it at all?

Actually just saw your commit moving nodemon to devDependencies. Why do we need it at all?

It can automatically restarting the node application when file changes in the directory are detected. I'm not the one who add this, I guess it's just for convenience

It can automatically restarting the node application when file changes in the directory are detected. I'm not the one who add this, I guess it's just for convenience

As far as I can tell it isn't linked to our start command, so I don't think it will do anything
Also, I only commented because it's an addition in the PR. It wasn't a dependency before. If we're not using it, no need to include


Make TypeScript clients consistent with other JavaScript event names

Our events have camel-cased names (onClosed, onDataReceived), but most JavaScript events in the browser have all-lower-case names (see https://www.w3schools.com/jsref/dom_obj_event.asp).
Unfortunately NodeJS is not consistent with this and does have camel-cased names, for example: https://nodejs.org/api/process.html#process_event_beforeexit. However, our events use the DOM style of having a property that is set to a function, rather than the NodeJS style of using EventEmitter. So it seems more appropriate to use the DOM event format.
We could also just decide to switch to EventEmitter and use that syntax, that's fine with me. We should just be consistent.

Our event names are camel cased to follow TS conventions.

My main concerns is discovery. All DOM events are strictly lowercase, and since JavaScript is case-sensitive, it can be difficult to realize why an event isn't firing. In theory we could support both, with one as an accessor aliasing to the other.
If we're fine with diverging from the DOM, that's OK with me, but it should be a conscious decision. I definitely agree that SignalR will be used outside the DOM but the primary environment in which the JavaScript client will run is the browser, so we should probably aim to support those conventions. What did SignalR 2.x do?
Either way, as long as we make a decision and stay consistent, I'm OK with it. I've just hit this a few times, and seen others hit it and wondered if we're aligning to the right convention :).

OK, let's discuss it during team meeting. I think, I am not concerned about diverging from the DOM too much since we want the client to be DOM/browser agnostic. If, however, camel case makes it difficult for the user to use the client (i.e. they have hard time figuring out why things don't work because they just have a typo in their code) then it would be the reason to make the names lowercase.
SignalR 2.x used camel case

The 2.x data makes me feel much more comfortable with camelCase. We can stick with this as-is and get some feedback on it.


Contrib Repo

@fchollet Since we're not adding much to the repo at this stage (in terms of layers, loss functions, callbacks, etc.), we've talked quite a bit about an external repo for user/additional contributions, that, based on utility and usage may eventually make it into the core.
I'm curious about how you would want/like to go about this. An external repo is not a big deal, but I'm wondering:

How do we keep the chain of commits and contributions/contributors intact if or when we start pushing items into the core Keras from the external repo?
How would we go about determining if something is used/useful enough to move to Keras?

Just looking for thoughts on this topic. I've seen a repo or two that are not forks (completely external) and overwrite certain Keras files then do a setup.py install, but that seems like a poor way to do it. A fork would keep the chain of contributions in tact and would be easy to merge, but having any reference to a fork would just be confusing in my opinion.

This is a good initiative.

How do we keep the chain of commits and contributions/contributors intact
if or when we start pushing items into the core Keras from the external
repo?

Git allows us to do pretty much anything. In particular we can reapply a
chain of commits with arbitrary authors. A contrib repo should live as an
independent repo from Keras (not a fork), and we can use custom git scripts
to transfer code from contrib to core Keras.

How would we go about determining if something is used/useful enough to
move to Keras?

Maybe by vote, as well as Github code searches.
Additionally: I would rather not have to manage a contrib repo myself,
since Keras keeps me busy enough already. We could have it under the
control of a small number of top Keras contributors.
On 6 January 2017 at 19:36, Pat York<EMAIL_ADDRESS>wrote:

@fchollet https://github.com/fchollet Since we're not adding much to
the repo at this stage (in terms of layers, loss functions, callbacks,
etc.), we've talked quite a bit about an external repo for user/additional
contributions, that, based on utility and usage may eventually make it into
the core.
I'm curious about how you would want/like to go about this. An external
repo is not a big deal, but I'm wondering:

How do we keep the chain of commits and contributions/contributors
intact if or when we start pushing items into the core Keras from the
external repo?
How would we go about determining if something is used/useful enough
to move to Keras?

Just looking for thoughts on this topic. I've seen a repo or two that are
not forks (completely external) and overwrite certain Keras files then do a
setup.py install, but that seems like a poor way to do it. A fork would
keep the chain of contributions in tact and would be easy to merge, but
having any reference to a fork would just be confusing in my opinion.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/4944, or mute the thread
https://github.com/notifications/unsubscribe-auth/AArWbyJoeEQr-9R5-FroUODNzUUu_fglks5rPomxgaJpZM4Lc-35
.

I anticipated you not wanting to manage that repo as well, given that you are the gatekeeper of this fairly massive one already. I was looking to get the contrib repo up and running, but I want to be sure it's setup correctly (e.g. in a useful way that allows future merges into Keras core) before starting anything.
I've seen a fair amount of decent code/PRs declined in the last couple of months alone that would fit well as contribs, so I think this would be good to figure out a framework for ASAP.
Perhaps we can do a proof of concept along the lines of a new repo, adding what amounts to come contrib code, and determining what works best in getting it back into Keras (or a temp branch).

This is a great idea. I have a few things I could contribute and would be willing to help moderate.

So would it be a kinda beta release of keras new components?

Cool  idea!

I wouldn't call it solely a beta for Keras, although that could definitely be part of it. I think of it as an extensions library as well; that is, some of the less used items (loss functions, layers, etc) will be merged and accessible via installation and importing of the contrib library, but they may not be used enough to make sense in the Keras core library and would never make it in there.
So, an extension library where some of the extension make it back into the Keras core.

I could definitely help moderate.

Would be great to have a contrib repo up and running. There are a lot of trig functions to add to keras backend and it might be cleaner to have them in a separate repo. I've also been wanting to add functionality for multiplayer models (GANs, BiGANs, AAEs, etc.) which would be better to put in a contrib repo. It would be a great place for stable code we just don't want polluting Keras. Not everyone is going to need arccosh or whatever.
Would the repo have separate CI, documentation hosting, etc? Separate Google/slack groups or no?

On that note, it would be nice for the contrib repo to have its own backend module (with the same logic and file structure as keras.backend) for any weird or esoteric backend functions.

Might be nice to hook all this up in a way that we can do pip install keras[contrib] and it would let us do something like from keras.contrib.layers import FancyUntestedLayer. We'd need some mods inside the keras setup for that, though. We can have just a keras_contrib PyPI package that can be standalone or patched into the main keras if installed as pip install keras[contrib].
I think we need separate CI / dox, though.

I think we should try to keep it entirely separate from Keras; simply a new repo that utilizes Keras for some things (backend functions, standard callbacks, etc), but that brings new functionality to the table.
An example would be
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras_contrib.layers import ForgetfulLSTM #not a real type of layer, I don't think
# but with a mirrored directory structure, the extensions should be self explanatory

import keras.backend as K
import keras_contrib.backend as C # perhaps we can call this "K" and if the function is not in the contrib library
#    we can just fall back directly to the Keras backend? e.g.
# import keras_contrib.backend as K

C.arccos(data) # extended backend functions
K.mean(data) # standard functions still available

model = Sequential()
model.add(ForgetfulLSTM()) #layers compatible with standard Keras models
model.add(Dense())

I've been playing around with it a bit today; I'm still stuck on wrapping my head around what mechanisms are available for preserving commit history across dissimilar repos. Making a bit of progress, but without being able to get the correct history of the extension back into Keras, if/when the changes are promoted to keras core, the contrib repo is kind of moot - copy/pasting code doesn't appeal to me.

Watch ForgetfulLSTM make it to NIPS 2018.
Something like this might be relevant. Involves a filtered view of one repository then pulling updates into the other.
http://gbayer.com/development/moving-files-from-one-git-repository-to-another-preserving-history/

If contrib is a large hodgepodge of things, maybe it makes sense to divide contrib up into extras.
Especially if we roll some specialized dependencies into contrib. For example, I wrote keras integration with tqdm which makes really nicely formatted progress bars in Jupyter. We could have a contrib extra that depends on progressbar, a contrib extra that depends on tqdm, etc.
We don't want to add any dependencies to keras to keep things lean but there could be contrib extras with various additional dependencies.

I thought a bit about submodules, subtrees, etc, but I think it would be better off to just be a separate repo (such that, when needed, we can merge stuff into Keras core). So far, it seems like just a mirrored structure and a bit of manual reworking and applying of patches will get us to:

Keras completely separate and independent from the contrib
Contrib just "sitting on top of" Keras providing new features
Commits available to push into Keras at @fchollet's and the contrib team's approval/discretion

History and contributor info intact

Separate documentation and CI build (but available to move to Keras as well)

I'd image the contrib repo being pretty lax in terms of requirements; just about anything can get merged, as long as it is valid code relating to deep learning, is "useful" to a wide audience (not super specific to a single model/dataset/person) and has tests written for it when called for. The really useful stuff would hopefully eventually move to Keras and be removed from the contrib repo, without much hassle.

Having played around with it a bit more, I think the separate repo is entirely viable. A quick test repo here (elephas, meaning ivory).
Some commits in the external repo (note: the repo is set up with similar structure/files as Keras, but this is not strictly necessary):

The highlighted commit above is put into a git patch via git format-patch. A little finagling of the patch file makes it compatible with Keras and available to be able to apply; these finagling changes are just to put the commit in the correct spot in the destination file. We take the patch to a new branch in Keras and apply with git am --signoff which keeps the commit message, original contributor, and has the sign-off note highlighted in blue:

At that point, the signee can reapply more commits in the same way; finally, just review and make any changes to the committed code, in a separate commit. These final fixes would be, perhaps, moving/removing/adding/editing import statements and their usage in the new code (e.g. elephas.layer.Name --> keras.layer.Name).
A lot of the manual stuff above could end up being handled automatically via simple search and replace; the patch finagling could be easier as well with a gui versus editing a diff file directly.

I don't think a contrib repo should mirror the code structure of core
Keras, and I don't understand why it should (it you feel strongly about it,
please explain what you reasons are). E.g. tf.contrib does not mirror core
tf, it's just a bunch of extras.
In the same way, I would see a Keras contrib repo as divided into modules
such as callbacks, layers, metrics, etc, with no redundancy between core
Keras code and contrib code.
On 9 January 2017 at 00:33, Pat York<EMAIL_ADDRESS>wrote:

Having played around with it a bit more, I think the separate repo is
entirely viable. A quick test repo here
https://github.com/patyork/elephas/commits/master(elephas, meaning ivory
https://github.com/fchollet/keras#why-this-name-keras).
Some commits in the external repo (note: the repo is set up with similar
structure/files as Keras, but this is not strictly necessary):
[image: image]
https://cloud.githubusercontent.com/assets/1304633/21754495/82d8fb48-d5b6-11e6-8617-9059e6cfaf3c.png
The highlighted commit above is put into a git patch via git format-patch.
A little finagling of the patch file makes it compatible with Keras and
available to be able to apply; these finagling changes are just to put the
commit in the correct spot in the destination file. We take the patch to a
new branch in Keras and apply with git am --signoff which keeps the
commit message, original contributor, and has the sign-off note highlighted
in blue:
[image: image]
https://cloud.githubusercontent.com/assets/1304633/21754525/1b8209d4-d5b7-11e6-82fc-f650049e6174.png
At that point, the signee can reapply more commits in the same way;
finally, just review and make any changes to the committed code, in a
separate commit. These final fixes would be, perhaps,
moving/removing/adding/editing import statements and their usage in the new
code (e.g. elephas.layer.Name --> keras.layer.Name).
A lot of the manual stuff above could end up being handled automatically
via simple search and replace; the patch finagling could be easier as well
with a gui versus editing a diff file directly.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/4944#issuecomment-271188622,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AArWb6ut1gDkDaBAqjV2PaPxJOX50XRvks5rQXI2gaJpZM4Lc-35
.

I agree it should be divided up into modules, such as callbacks, metrics, layers, backend additions. The best way to do that is via a directory and file structure that Keras has; e.g.:

- backend
 - theano_backend.py (additional ops for theano)
 - tensorflow_backend.py (additional ops for tensorflow)
- layers
 - .py
- metrics.py (additional metrics code)

The above files, while named the same, contain no Keras code; perhaps they will have a from keras import x statement, but no copies of Keras code. Following the Keras directory and file structure is simply the neatest way to organize everything. An example:
from keras.layers import Dense
from elephas.layers import ForgetfulLSTM

Import structure follows a reasonable pattern between Keras versus the contrib repo.

@patyork if there are going to be more things being added by a larger contributor base it is going to be easier to manage with separate files. So, maybe metrics.py, regularizers.py, backend, etc. should be upgraded to modules/directories for the contrib repo? Add in an import * so we don't have conflicting commits in the init.py and everyone can safely works on their own metrics, etc.
More PRs would be adding files instead of editing files, so hopefully fewer conflicts and cleaner histories.

Pat York: yes, that sounds perfectly reasonable.
On 9 January 2017 at 01:16, Ben<EMAIL_ADDRESS>wrote:

@patyork https://github.com/patyork if there are going to be more
things being added by a larger contributor base it is going to be easier to
manage with separate files. So, maybe metrics.py, regularizers.py, backend,
etc. should be upgraded to modules/directories for the contrib repo? Add in
an import * so we don't have conflicting commits in the init.py and
everyone can safely works on their own metrics, etc.
More PRs would be adding files instead of editing files, so hopefully
fewer conflicts and cleaner histories.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/4944#issuecomment-271192449,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AArWb9F7mRZD9Qmc5BMONQS3CTbYxjpsks5rQXxigaJpZM4Lc-35
.

It's probably a good idea to choose another name to avoid confusion, Elephas is taken by this keras related project:
https://github.com/maxpumperla/elephas

Elephas is already taken indeed, and it's a quite popular project. How
about just keras_contrib, or keras_extras?
On 10 January 2017 at 02:00, Michael Oliver<EMAIL_ADDRESS>wrote:

It's probably a good idea to choose another name to avoid confusion,
Elephas is taken by this keras related project:
https://github.com/maxpumperla/elephas
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/4944#issuecomment-271456231,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AArWb8bWvL0freVtLD8AYcErAAlexlDUks5rQthIgaJpZM4Lc-35
.

Ah bummer - I thought that name would have been clever for an extension repo.

For a detailed and mature design of the sort being discussed here look to boost, the obvious tensorflow contrib, and homebrew, and Qt. All are mature projects.
Also note splitting up will open up the latest successor to dll hell... versioning hell, as found in brew. OpenCV has done the same thing with their opencv_contrib repository, and as a few google searches can attest it seems to be very painful for users.
What about simply having a contrib folder in keras that is clearly marked experimental where things can be moved to the mainline folder as they are proven as done by tensorflow and some boost modules?

As @fchollet suggested, opening up commit permission to a few trusted top contributors is likely a good plan to reduce his workload. Making them the owner of that contrib directory could be a good way to go about it.

Is it possible to give commit rights specific to a directory on Github?

On 10 January 2017 at 03:33, Andrew Hundt<EMAIL_ADDRESS>wrote:

As @fchollet https://github.com/fchollet suggested, opening up commit
permission to a few trusted top contributors is likely a good plan to
reduce his workload. Making them the owner of that contrib directory could
be a good way to go about it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/4944#issuecomment-271469900,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AArWb90YMFPCquD4bSry4cmbjviPT4ZJks5rQu3kgaJpZM4Lc-35
.

I think it's all or nothing - I've heard that gitlab has something like this, but I'm not sure. If we were to have other people as the "owner" of the contrib folder, we could just have @fchollet tag PRs as contrib worthy but not core-ready, and it would have to be an honor system thing (perhaps with guidelines, such as the removal of moderator status if violated) such that moderators do not merge / push directly to core.

Yeah I guess this could mean trusting the people with access. Git hooks or the github status api could be another option.

I think it's all or nothing - I've heard that gitlab has something like this, but I'm not sure. If we were to have other people as the "owner" of the contrib folder, we could just have @fchollet tag PRs as contrib worthy but not core-ready, and it would have to be an honor system thing (perhaps with guidelines, such as the removal of moderator status if violated) such that moderators do not merge / push directly to core.

as @lukedeo says with the honor system, there's nothing like a lightweight code of conduct for contributors (pull requests) and collaborators (commit access) to keep things in order. If it doesn't work out, revoking collaborator access is just a few clicks.

That seems like a reasonable thing to do. Here's what we would need:

a document mapping contributor usernames to lists of folders they are
authorized to commit files to.
a code review guide to help collaborators perform consistent reviews and
vetting of contrib PRs.
a notification system were PRs made in contrib (or PRs tagged as
belonging in contrib) trigger an email to the contrib collaborators.

I can write 2). Any suggestions as to 3)? Also, anything I am missing?
Additionally: what is the current list of Keras contributors interested in
maintaining such a contrib folder?
On 10 January 2017 at 04:13, Andrew Hundt<EMAIL_ADDRESS>wrote:

as @lukedeo https://github.com/lukedeo says with the honor system,
there's nothing like a lightweight code of conduct
https://github.com/Homebrew/brew/blob/master/CODEOFCONDUCT.md for
contributors (pull requests) and collaborators (commit access) to keep
things in order. If it doesn't work out, revoking collaborator access is
just a few clicks.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/4944#issuecomment-271475167,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AArWbzoPk4n8cNpv7jmjIsWAtEOL4QUEks5rQvdQgaJpZM4Lc-35
.

As for 3), I can take a stab at a simple polling system with PyGithub that would cost O($0.01) to run per month - I can start taking a look tonight. As for list of contributors, we could open an issue and people can 👍 if they want to be considered?

Great, thanks!
Currently I am counting: @patyork, @lukedeo, @the-moliver. Anyone else, please 👍 this post to be considered.

I'm quite interested.

Just thinking ahead a bit, but if possible it would probably be good to have a separate TravisCI build for the contrib folder. Theoretically (and hopefully in practice) contrib items shouldn't break the Keras core, or even touch it, making those 15-20 minute tests redundant. I've never used TravisCI in depth, so I'm not sure if you have that fine of control over whether a certain PR calls test suite A versus suite B.

@patyork as tensorflow does, the pull requests can be compiled by travis before merges are permitted. For that reason I'd suggest merging via pull request with no direct commits be one element of the code of conduct. It gives the opportunity for code review and travis ci to run.

@fchollet I've set up a system using AWS Lambda => AWS SES using a web-hook from GitHub that should run at ~0.40$/mo that'll be ready by tomorrow. Only remaining questions are:

Who should maintain this? I'm more than happy to make sure the system is humming along.
How do we want to add users? I've just assumed it's going to be low enough velocity that we can add users in by hand. (The email has a link for users to unsubscribe, though)

I envision that we would have a COLLABORATORS.txt document (or similar) in the Keras root that would describe a mapping between Github usernames (maybe email addresses) and directories that collaborators can commit to. This would be updated by hand, and could be used as input to automatically update any infrastructure we have around collaboration.
Thinking further about the contrib repo, there are still a couple reasons why we might want to keep it separate from Keras:

When making changes to contrib, we don't want to trigger a Travis build for all of Keras, as Pat York points out. Will there be any way to prevent this if contrib is a Keras subdirectory?
A TF-only version of Keras will be moved to the TF repo in the near future. This version should maintain the same API as the external (multi-backend) version. Of course it cannot include the contrib repo,  so that would induce an API discrepancy: in external Keras you will be able to do from keras import contrib but this wouldn't work in TF-internal Keras.

I hope we can solve the first point. The second point may not be so important, but still something to keep in mind.

@lukedeo you could definitely be in charge of maintaining the notification system (in my experience it's possible to leave a similar setup running and essentially never have to touch it in years). It would just need to have open-source deployment instructions so that anyone could take it over in the future.

When making changes to contrib, we don't want to trigger a Travis build for all of Keras, as Pat York points out. Will there be any way to prevent this if contrib is a Keras subdirectory?

Sorry for missing this, but what is the downside to triggering a travis build for all of keras?

A TF-only version of Keras will be moved to the TF repo in the near future. This version should maintain the same API as the external (multi-backend) version.

Cool! Will it be in sync with the multi backend version? sorry for the digression

Of course it cannot include the contrib repo, so that would induce an API discrepancy: in external Keras you will be able to do from keras import contrib but this wouldn't work in TF-internal Keras.

Will the api be import tf.contrib.keras vs import keras.contrib or something similar to that? Would such a difference make for a reasonable indicator of what's available?

On the Travis build, the downsides are fairly minor of kicking off a full build each time: It's long running time and just running unnecessary builds on Travis (which is free for the end users).
On the TF integration/move, I think he just means that it wont include Keras' contrib folder in the TensorFlow repo? Meaning if someone is using from TensorFlow directly, they wont have access to these contributions/extensions without cloning/downloading Keras itself? Whereas a separate repo for the Keras contributions would sit on top both Keras standalone and the TF version (as an optional install). I could be wrong though, this is the first I've read about a TF-Keras integration.
-----Original Message-----
From: "Andrew Hundt"<EMAIL_ADDRESS>Sent: 1/10/2017 8:18 PM
To: "fchollet/keras"<EMAIL_ADDRESS>Cc: "Pat York"<EMAIL_ADDRESS>"Mention"<EMAIL_ADDRESS>Subject: Re: [fchollet/keras] Contrib Repo (#4944)
When making changes to contrib, we don't want to trigger a Travis build for all of Keras, as Pat York points out. Will there be any way to prevent this if contrib is a Keras subdirectory?
Sorry for missing this, but what is the downside to triggering a travis build for all of keras?
A TF-only version of Keras will be moved to the TF repo in the near future. This version should maintain the same API as the external (multi-backend) version.
Cool! Will it be in sync with the multi backend version? sorry for the digression
Of course it cannot include the contrib repo, so that would induce an API discrepancy: in external Keras you will be able to do from keras import contrib but this wouldn't work in TF-internal Keras.
Will the api be import tf.contrib.keras vs import keras.contrib or something similar to that? Would such a difference make for a reasonable indicator of what's available?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

On the Travis build, the downsides are fairly minor of kicking off a full build each time: It's long running time and just running unnecessary builds on Travis (which is free for the end users).

Travis there is a caching mechanism which could be used for incremental builds, or something like docker could be used to facilitate incremental builds and reduce build time. Even stuff in contrib should likely be verified, right?

On the TF integration/move, I think he just means that it wont include Keras' contrib folder in the TensorFlow repo?

I think he was saying that he has been working on a new TF only implementation of Keras that currently is internal at Google that will eventually be directly integrated into github.com/tensorflow/tensorflow.

I envision that we would have a COLLABORATORS.txt document (or similar) in the Keras root that would describe a mapping between Github usernames (maybe email addresses) and directories that collaborators can commit to.
What does "directories that collaborators can commit to"? Does it mean that we have several owners  who are respectful for different directories to merge the commit from other collaborators?

Here is a homebrew link I meant to post which has all the useful info for contributors. I thought it might make for a good example:
https://github.com/Homebrew/brew/tree/master/docs

I just realized contrib is going to break saving and loading and a PR I just made for a bugfix is a possible solution. https://github.com/fchollet/keras/pull/5012 (PR probably still needs some additional docs and unit tests).
I added a global dictionary of custom objects. I updated get_from_module to first check that dictionary. This can be a standard location to register custom objects so they load by name and work with save and load.
The idea for contrib is that I could write a file l3_regularizer.py
class L3Regularizer(Regularizer):
  ...

get_custom_objects().update({"L3Regularizer": L3Regularizer})

If I then import keras_contrib.regularizers.l3_regularizer, Dense(...,W_regularizer='L3Regularizer') will work as expected and saving and loading will work as well. This is a potential standard for how to add custom objects to custom_objects and have everything save and load correctly.
Any thoughts on how contrib is going to affect saving and loading?
Cheers,
Ben

Any movement on this?

This pull for custom objects just got merged yesterday: https://github.com/fchollet/keras/pull/5012
@fchollet suggested a scope operator which makes things much cleaner and helps prevent name collisions. Custom objects can be loaded within a with statement which prevents leaks.
We still need to enforce some standard for object names which would help reduce chances of collisions further, for example, using the module name in the name returned from get_config. Saving, loading, and instantiating by name will work as long as the string returned from get_config matches the string in custom_objects.
The trade-off is if we use fully qualified names, referencing the object by name gets a little wordy like Dense(512, ..., W_regularizer="l3_regularizer.regularizers.L3Regularizer") but you could still just create the instance yourself like Dense(512, ..., W_regularizer=L3Regularizer()).
#keras_contrib.regularizers.l3_regularizer.py
class L3Regularizer(Regularizer):
  ...
custom_objects = {"regularizers.l3_regularizer.L3Regularizer": L3Regularizer}

#using the module
from keras_contrib.regularizers import l3_regularizer
with custom_object_scope(l3_regularizer.custom_objects):
  # custom object will resolve within this scope by name
  layer = Dense(512, ..., W_regularizer="regularizers.l3_regularizer.L3Regularizer")

custom_objects is still a parameter to several functions and works as it did previously, but working through custom_object_scope prevents name leaking that was the existing functionality.

@bstriner Just to be clear, even in the case of a built in contrib folder (in the Keras repo), the above notation would be required as all contributions would be considered custom objects?

Could we start a contrib repo as an external repo (and potentially include
it into Keras later down the road if that sounds like a good strategy?)
On 22 January 2017 at 12:08, Pat York<EMAIL_ADDRESS>wrote:

@bstriner https://github.com/bstriner Just to be clear, even in the
case of a built in contrib folder (in the Keras repo), the above notation
would be required as all contributions would be considered custom objects?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/4944#issuecomment-274355399,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AArWb28yDbmZH3djqV2pMWdoZ6qPDr4Fks5rU7dZgaJpZM4Lc-35
.

I think that's a fine idea - I don't have the reservations against a (well managed) external repo that others have (such as versioning issues and whatnot).

@patyork such notation would be required for any objects you don't want to be part of the default namespace but want to add conditionally. This is also the notation you would use for custom objects in your own code if you want your models to save and load correctly.
If you want something to be included by default, you could add get_custom_objects.update(custom_objects) to the bottom of the module and make sure it is imported somewhere.
For objects in keras master, an alternative would be a further PR to get_from_module that can handle dotted objects. Then, contrib.L3Regularizer would resolve to keras.regularizers.contrib.L3Regularizer if run with module_params=keras.regularizers, unless overridden by custom_object_scope. I think this might be the cleanest final solution for handling a separate contrib repo, some included contrib repo, and custom objects in user code in a standardized way.
We could also add some custom handling to get_from_module to recognize relative and absolute paths. This would be my ideal logic for get_from_module:

get_from_module will first check custom_objects, which always takes precedence and can be used to override existing code
if the string is an identifier, get the identifier from the module
if the string is a dotted identifier, try to get the submodule from the module, possible importing modules if not already imported
if the string starts with /, try to import a fully qualified object

This way:

If the module is in contrib or some other module, return the fully qualified path in get_config and everything should work.
If the module is somewhere inside keras but not in one of the main modules, return the relative path in get_config.
If you want additional control or to enable some aliases, use custom_objects to provide clean, simple names for your objects.

Does that logic for get_from_module make sense? It would be back-compatible with existing code and would make everything nice and flexible. Wouldn't be too complicated to put together.
Cheers

"Kextras"

Separate repo sounds much cleaner. Keras can have a very high coding standard and we can enforce a bunch of quality checks and documentation standards and test coverage while keras-contrib can have a little more variety. Separate repo would also make permissions a lot safer.
@patyork Where are you going to start working on the contrib repo? I'd love to take a look and try it out.
contrib might also benefit from a more complicated branch structure. master could be the current stable code. PRs would be made to dev so the community can try things out and do some cleanup.
There could be one tier of developers pushing changes from dev to master after they seem stable, and a broader tier working on dev. You could submit some useful code to dev, then someone else could clean up the documentation and testing before it goes to master.

@bstriner I have no plans to start/own the contrib repo (if it is external); that'll be for someone else to start. AI/Keras is just a hobby, that I happen to have a bit of time for while I'm on a long vacation.
I started this thread was just to get a plan/repo in place - I've seen a lot of useful code just sitting in PRs and Issues without a good place to live, not to mention stuff in external repos. Plus, talk of starting a contribution repo had been floating around random PRs for a few months.

@bstriner I'd be happy to help with contrib. I use Keras ~daily for research and for production so I have aligned incentives, hah.

If it is a separate repo, I could start one, and add you guys as moderators. Let me know what you think.

@farizrahman4u sounds fine to me!

For an example of a reasonably active project with a main and contrib repository, see the static site generator, Pelican  https://github.com/getpelican/pelican/ and its plugin repository https://github.com/getpelican/pelican-plugins/
I'm a heavy user of that project and a sometimes contributor. Qualitatively, I'm not sure the plugins repo has added much to the project; rather, many of the actively maintained pelican plugins seem to have moved into their own repositories, and many of the ones that remain have long-hanging pull requests which the maintainers need to intervene in. The contrib repository has become partially a index of interesting git submodules; but practically you often ignore the plugins repo and install a specific plugin you want via pip install. It looks like a lot of work for the maintainers to keep that shared playground and the benefit is not 100% clear.
OTOH, there are already well-maintained projects which can use keras without a keras contrib repo, (e.g.  http://edwardlib.org/ ), and it's easy for your project to depend on keras through the standard python way of declaring dependencies in setup.py.
This is not necessarily to say we should not have a contrib repo (or one with the same system as pelican-plugins) - but it would be nice to clarify precisely what type of contrib repo we were looking for, and the precise costs and benefits of the model we are aiming for.
For example, I am writing a custom keras layer at the moment. Should I be writing it in some contrib repo, in my own repo, or in both? What do I gain from having the option? What are the benefits of going in the official contrib repo? Free publicity?

How should the "versioning hell" I described earlier be avoided in the separate repo?
Is there some way compatible commits could be synced between Keras and the separate contrib repo?
In other words, how do I make sure my contrib repo and keras repo are compatible, and if an update to one or the other causes a regression, how do I get back to the previous working state in an automated fashion?
Separate repos are fine if that's addressed, though I'm still more of a fan of the contrib folder idea which avoids that problem completely with some minor but manageable drawbacks as discussed earlier.

Here are my thoughts :
The keras-contrib (is the name confirmed?) repo will mirror the Keras project structure (layers : core, recurrent,..; optimizers, constraints, regularizers etc..)
The repo will be moderated pretty much like its done in Keras.. users PR their work, will be reviewed by the community and then merged. Additionally, write access will be provided for users with major contributions, under the condition that they make changes to their contributions only.. There will be a markdown file with user to directories mapping specifying the write access of each user... as discussed above.
Developers should consider contributing to keras-contrib rather than own repo, because the contributions are peer reviewed and therefore have higher quality and credibility, and will be trusted more by end users.. Also higher visibility and reputation.

K = Keras, KC = Keras-Contrib (got lazy)
The simplest versioning-hell-abating tactic is simply to only support bleeding Keras (e.g. Travis tests using the master branch of Keras). If the contrib repo tests fail against that branch, then the collaborator (and contributor) base should look into and rectify it. If at any time you wish to upgrade the contrib repo/keras, you should pull from git both the KC repo and the K repo.
If you want to support simple versioning (e.g. Keras v1.2.0 is supported, Keras 1.2.1 is not) then it is probably best handled as Keras versioning is; separate Github tags and PyPy releases done at the same time as Keras. So KC v1.2.0 should sit on K v1.2.0. What falls out of that is the same as above - the master branch of KC supports the master branch of K. If you only want to walk the releases, you install/upgrade to matching K and KC versions, as defined by the PyPY releases or the github tags.

Agree. Support only for bleeding edge.

+1 for only bleeding edge support

To clarify: KC.master <--supports--> K.master.
What falls out of that, then, is that when @fchollet preps a release, the KC team checks the KC Travis tests (against the current K.master, which is becoming a new release):

If the build is passing:

Tag as KC.
Push KC release
masters and equivalently versioned release support each other

If the build is not passing:

Pause any additional merging
Rectify build issues
Tag
release
masters and equivalently versioned release support each other

Therefore, it would also be true that KCv1.2.1 <--supports--> Kv1.2.1 as they are versioned in tags/branches/PyPy.

@ahundt It seems there would be 2 possible ways.

If I understand correctly, what @fchollet, @bstringer, and @patyork were discussing at the start is a KC which by disciplined community effort moves in lockstep with the development of the K main source tree, so the 'extras' in a given point release would be compatible with keras version at the same point release. (Edit: as I type this, @patyork has written more or less this) So, something like the tensorflow repository itself as regards compatibility, but with KC broken out into a second repository in the hope of fostering a good mutual code review community.

The KC model I was implicitly assuming, on reflection, seem to be different - A bunch of different packages, each with their own setup.py to specify their own version dependencies that happen to be in the same repo.

Number 2 (packages with explicit version requirements) is the standard python way to solve versioning problems, although it seems to defeat @farizrahman4u's suggestion that the purpose of a shared repo is exposure and (hopefully) peer review, since it essentially means having separate packages with loose dependencies.
On the other hand, number 2 feels dangerous to me; if someone drops off the maintenance bandwagon just before the release date for K 1.3.0, for their subtle and sophisticated KC sub package, I presume we either have to wait for someone else to step in to fix it, or just delete it so that KC can remain current. Or am i missing the point? Are we weeding out one-person-projects from KC to avoid this kind of risk?

One person projects are welcome. The assumption is that all contributions are properly documented, and the community will be able to pick up where the original contributor left off.

I do think Keras contrib master should be in sync with Keras master.

Therefore, it would also be true that KCv1.2.1 <--supports--> Kv1.2.1 as
they are versioned in tags/branches/PyPy.

Sounds good to me.
We will change the Keras API in the relatively near future with Keras 2
(changes will be made backwards compatible as much as possible, maybe
completely). Some updating of Keras contrib will be required when that
happens, but beyond that point I would expect the Keras 2 API to be
completely stable in the very long term. Keras 2 should essentially be
"Keras forever".
On 23 January 2017 at 12:49, Fariz Rahman<EMAIL_ADDRESS>wrote:

One person projects are welcome. The assumption is that all contributions
are properly documented, and the community will be able to pick up where
the original contributor left off.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/4944#issuecomment-274612451,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AArWb7v6IE0yoD1VbE-F2nyAq_RluPAuks5rVRJlgaJpZM4Lc-35
.

Keras 2 should essentially be "Keras forever".

Other potential candidates :

Keras returns
Keras rises
Revenge of the Keras

The thing with permanent bleeding edge is it inevitably breaks and you as a user need to reconcile code you've not modified before, and there's no trivial way to roll back to the last thing that worked, because the broken bleeding edge will still be used even if you revert.
Actually, if the build/test artifacts/log of KC could simply store/save/print the git hash of K it tested with, that would at least give something to key off of if you needed to go back in the future.
Anyway, don't let me block what is a very good thing, tweaks can always be made later once things are no longer hypothetical. :-)

@howthebodyworks
I think the reason for a separate contrib repo is that the research of Deep Learning goes too fast, there will be new architectures, regularization methods, activation functions everyday. It's best to set a separate official contrib repo so that 1). The keras users can have import and use those new methods at their will through an official repo, while at the same time 2). we keep the original keras repo intact and clean, we will only merge some of the new methods into the original keras repo after some kind of discussion.

I've discussed with some of the people responsible for tf.contrib, and they tend to agree that having a separate repo is much cleaner, and that having contrib part of the main repo is a source of issues. So that's what we should do.
So here are the specs for Keras contrib:

Repo name should be something like keras-contrib or keras-extras, nothing extravagant
Maintainers should be @patyork, @lukedeo, @the-moliver, @tboquet, @kemaswill, @bstriner, @farizrahman4u, @fchollet to start with (that's already a lot)
To become maintainer, you should be a recognized previous contributor to either contrib or core Keras
directory structure should mirror core Keras.
To make integration of specific objects into core easier, style guidelines should be the same as with Keras core (PEP8, docstring format) and that should be enforced via a Travis test.

Feel free to get started. Note that we would eventually move the repo to a Keras GitHub organization.

github.com/farizrahman4u/keras-contrib

Question from https://github.com/fchollet/keras/issues/4944:  Should a tf-slim style setup for "batteries included" vision based classification and semantic segmentation go in keras-contrib or deep-learning-models?
If the latter, can deep-learning models have maintainers and a more substantial structure as well?
What about an appropriate place for reinforcement learning algorithms like a3c, ddpg, cem and so on?

Please take me off this email list.  I am not even the right person.
On Feb 7, 2017 6:08 PM, "Andrew Hundt"<EMAIL_ADDRESS>wrote:

Question from #4944 https://github.com/fchollet/keras/issues/4944:
Should a tf-slim https://github.com/tensorflow/models/tree/master/slim
style setup for "batteries included" vision based classification and
semantic segmentation go in keras-contrib
http://www.github.com/farizrahman4u/keras-contrib or
deep-learning-models https://github.com/fchollet/deep-learning-models?
If the latter, can deep-learning models have maintainers and a more
substantial structure as well?
What about an appropriate place for reinforcement learning algorithms like
a3c, ddpg, cem and so on?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/4944#issuecomment-278207649,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ATd6Lo9gFLEHAtUskfqPHRX1ElIkO1Jqks5raSOagaJpZM4Lc-35
.

@bstringer your email provided an unsubscribe link https://github.com/notifications/unsubscribe-auth/ATd6Lo9gFLEHAtUskfqPHRX1ElIkO1Jqks5raSOagaJpZM4Lc-35 (this message will too)


Panic in versioncheck.go

After I installed namespace, most invocations of ns besides ns help crash with the following trace:
❯ ns
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x10204d783]

goroutine 70 [running]:
namespacelabs.dev/foundation/internal/cli/fncobra.checkRemoteStatus({0x103468c18, 0xc001009dd0}, 0xc0005a52c0)
	namespacelabs.dev/foundation/internal/cli/fncobra/versioncheck.go:55 +0x283
created by namespacelabs.dev/foundation/internal/cli/fncobra.DoMain.func1
	namespacelabs.dev/foundation/internal/cli/fncobra/main.go:168 +0x3a5

I suspect my network is blocking the URL the ns is trying to connect to.
I noticed that the installation script installs an older version (v0.0.73). This panic does not occur with tip of main branch.
https://github.com/namespacelabs/foundation/blob/84eb04f9f1c1bffa9dd5ae2a0bcf12695a3ea7b4/install/install.sh#L8
Can we upgrade the default version to a more recent one?

BTW, I also noticed that maybe here we should return early?
https://github.com/namespacelabs/foundation/blob/v0.0.73/internal/cli/fncobra/versioncheck.go#L52

Thanks for the report. Yes, we needed an early return there :).

@nichtverstehen do I understand correctly that the version in the script refers to nsboot?
Merging ns and nsboot would resolve this source of confusion then.


Always link to libc when compiling musl targets.

This fixes a regression when changing musl targets to always statically link to libstdc++, and the static version of libstdc++ can reference but be missing requiring symbols from libc. Due to the linking order of Rust, these symbols sometimes are not resolved. This ensures that all binaries with external C++ dependencies cross-compiled for musl targets, such as re2, will be properly linked.
Closes #1124.

bors try --target musl

This is going to link to a few issues with specific targets when compiling re2 (and will be updated periodically):

aarch64-unknown-linux-musl: missing soft-float intrinsics (I've seen this before)

          /usr/local/bin/../lib/gcc/aarch64-linux-musl/9.2.0/../../../../aarch64-linux-musl/bin/ld: /usr/local/bin/../lib/gcc/aarch64-linux-musl/9.2.0/../../../../aarch64-linux-musl/lib/libstdc++.
a(hashtable_c++0x.o): in function `std::__detail::_Prime_rehash_policy::_M_next_bkt(unsigned long) const':
          /tmp/tmp.Ll1kBV4fy6/build/local/aarch64-linux-musl/obj_gcc/aarch64-linux-musl/libstdc++-v3/src/c++11/../../../../../src_gcc/libstdc++-v3/src/c++11/hashtable_c++0x.cc:78: undefined refere
nce to `__floatunditf'
          /usr/local/bin/../lib/gcc/aarch64-linux-musl/9.2.0/../../../../aarch64-linux-musl/bin/ld: /tmp/tmp.Ll1kBV4fy6/build/local/aarch64-linux-musl/obj_gcc/aarch64-linux-musl/libstdc++-v3/src/c
++11/../../../../../src_gcc/libstdc++-v3/src/c++11/hashtable_c++0x.cc:78: undefined reference to `__extendsftf2'
          /usr/local/bin/../lib/gcc/aarch64-linux-musl/9.2.0/../../../../aarch64-linux-musl/bin/ld: /tmp/tmp.Ll1kBV4fy6/build/local/aarch64-linux-musl/obj_gcc/aarch64-linux-musl/libstdc++-v3/src/c
++11/../../../../../src_gcc/libstdc++-v3/src/c++11/hashtable_c++0x.cc:55: undefined reference to `__extendsftf2'
          /usr/local/bin/../lib/gcc/aarch64-linux-musl/9.2.0/../../../../aarch64-linux-musl/bin/ld: /usr/local/bin/../lib/gcc/aarch64-linux-musl/9.2.0/../../../../aarch64-linux-musl/lib/libstdc++.
a(hashtable_c++0x.o): in function `std::__detail::_Prime_rehash_policy::_M_need_rehash(unsigned long, unsigned long, unsigned long) const':
          /tmp/tmp.Ll1kBV4fy6/build/local/aarch64-linux-musl/obj_gcc/aarch64-linux-musl/libstdc++-v3/src/c++11/../../../../../src_gcc/libstdc++-v3/src/c++11/hashtable_c++0x.cc:100: undefined refer
ence to `__extendsftf2'
          /usr/local/bin/../lib/gcc/aarch64-linux-musl/9.2.0/../../../../aarch64-linux-musl/bin/ld: /tmp/tmp.Ll1kBV4fy6/build/local/aarch64-linux-musl/obj_gcc/aarch64-linux-musl/libstdc++-v3/src/c
++11/../../../../../src_gcc/libstdc++-v3/src/c++11/hashtable_c++0x.cc:100: undefined reference to `__floatunditf'
          /usr/local/bin/../lib/gcc/aarch64-linux-musl/9.2.0/../../../../aarch64-linux-musl/bin/ld: /tmp/tmp.Ll1kBV4fy6/build/local/aarch64-linux-musl/obj_gcc/aarch64-linux-musl/libstdc++-v3/src/c
++11/../../../../../src_gcc/libstdc++-v3/src/c++11/hashtable_c++0x.cc:101: undefined reference to `__floatunditf'
          collect2: error: ld returned 1 exit status

bors try --target musl
I've also tested aarch64-unknown-linux-musl against rust-cmake-, and all of the targets above passed, so as long as this passes the default tests, we should be fine (and CMake support is ready to be implemented and merged).

bors r=Emilgardis


Replace tendermint with cometbft

use commetbft types in rollkit in order to comply with cosmos-sdk 0.47.0 by replacing tendermint with cometbft, and then replacing cometbft with the rollkit fork of tendermint

Few things:

Consider re-importing latest version of mempool from cometbft (as separate, follow-up PR).
tm prefix for imports should be changed - maybe comet would do the trick? cometbytes? cometmath? or cb - cbbytes, cbmath, cbquery`, etc.
Do not use string([]byte("string")). Even in tests.

Could you elaborate on what you mean by re-importing the latest version of mempool as a follow up PR?
On it
On it

I wonder if this can be a quick "dependency hell" fix as long as the app doesn't migrate to CometBFT.
We should try depending on celestia-node after this PR, to see if it works.
If works, it would make Rollkit being able to migrate to RPC/API much earlier and even use BlobModule right after it's done.
cc @nashqueue

CometBFT is needed because of ABCI though

Yep, my point is that it can also bring additional benefit

I wonder if this can be a quick "dependency hell" fix as long as the app doesn't migrate to CometBFT.

Core is already on comet bft https://github.com/celestiaorg/celestia-core/pull/980

Core is already on comet bft https://github.com/celestiaorg/celestia-core/pull/980

It just pulls the changes; they didn't rename the module here. Node imports it as tendermint, not comet. And if you migrate to comet, then there will be no conflicts between comet and tendermint

need to resolve conflicts


Pinned Dependencies doc questions

I followed along with the example on the Pinned Dependencies page and had two questions.
When trying what is written in the last paragraphs:

Now, whenever we are running rescript build within our app package, the compiler would always rebuild any changes within its pinned dependencies as well.
Important: ReScript will not rebuild any pinned-dependencies in watch mode! This is due to the complexity of file watching, so you'd need to set up your own file-watcher process that runs rescript build on specific file changes.

After a rescript clean, rescript/rescript build worked fine. But if I made further changes in any of the pinned dependencies, running the normal build in the app package would not reflect those changes. In other words, with the normal build, only changes directly in the app workspace were compiled. However, if I ran rescript build -with-deps in the toplevel app workspace, changes in all pinned dependencies compiled fine. Is this the expected workflow and if so, should we add this point to these docs? After figuring out needing -with-deps, using my own watcher at the toplevel package to run this as needed is a great workflow for my project.
Second, in the codebase example snippet with Yarn Workspaces, I had to make a package.json in workspace as well before running yarn install, otherwise rescript would give me an error that the dependency is not installed. Would it be worth adding this to the snippet to make it easier to follow along with the example?

Good input. I guess --with-deps is necessary here...
@bobzhang what do you think?

Second, in the codebase example snippet with Yarn Workspaces, I had to make a package.json in workspace as well before running yarn install, otherwise rescript would give me an error that the dependency is not installed. Would it be worth adding this to the snippet to make it easier to follow along with the example?

You mean each subpackage should list its own package.json? Sounds like an adaption we want

See here for an example Yarn workspaces repository I created for https://github.com/rescript-lang/rescript-compiler/issues/5197 (which is somewhat related).

You mean each subpackage should list its own package.json?

Yeah, it's a detail about workspaces and not rescript, but still a gotcha when following the example


Fix d3-color vulnerability

There is currently a dependabot security alert about d3-color.
This is a quick fix by upgrading the d3 package. However dagre-d3 will need to be updated too, but appears to no longer be maintained. I recommend switching to a more recent fork, such as dagre-d3-es, instead.
I've tested this and it looks like the only required change to the code aside from the import statement will be to update
const zoom = d3.zoom().on('zoom', () => {
  inner.attr('transform', d3.event.transform);
});

to
const zoom = d3.zoom().on('zoom', (zoomEvent) => {
  inner.attr('transform', zoomEvent.transform);
});

@kd-ods I think that'd be best. It won't take long.


Install scylla-tools on Fedora box

Loader machine need a Scylla compatible version of cassandra-stress.
Till now loader machine use Scylla AMI, which runs Scylla service automatically, and than stops by the loader - an error prune process.
This request use a Fedora AMI instead, installing scylla-tools rpm for the cassandra-stress.

@glommer please let me know if this solve the issue you saw.


Facing styling issue when component is loaded

[X] The issue is present in the latest release.
[X] I have searched the issues of this repository and believe that this is not a duplicate.

Current Behavior 😯
When using material-ui/core-5.0.0-beta.2, some styles are behaving incorrectly.

https://codesandbox.io/s/material-ui-cron-unstable-fl1dh?file=/src/App.tsx
Another note: if the same component is not loaded initially and loaded only when some state changes (after component  mount) then the styles are preserved.

Expected Behavior 🤔
When using material-ui/core-4.11.2, styles are behaving as expected.

https://codesandbox.io/s/material-ui-cron-stable-hftnc?file=/src/App.tsx

Steps to Reproduce 🕹
See above sandbox

Context 🔦
I have built a library https://github.com/baymac/material-ui-cron. This issue is causing disruption in library usage after upgrading to material-ui/core-5.0.0-beta.2

Your Environment 🌎

`npx @material-ui/envinfo`
  System:
    OS: macOS 11.2.3
  Binaries:
    Node: 14.16.1 - ~/.nvm/versions/node/v14.16.1/bin/node
    Yarn: 1.22.5 - ~/.yarn/bin/yarn
    npm: 6.14.12 - ~/.nvm/versions/node/v14.16.1/bin/npm
  Browsers:
    Chrome: 92.0.4515.107
    Edge: Not Found
    Firefox: 89.0.2
    Safari: 14.0.3
  npmPackages:
    react: ^17.0.2 => 17.0.2
    react-dom: ^17.0.2 => 17.0.2

Thenkas for the feedback.
In order to debug this issue we need a repro that's just using @material-ui packages in react. We can't debug issues in component libraries built on top of Material-UI.

Closing since no reproduction was provided within 14 days.


net/upnp: Update plugin title/menu to include PCP as supported

Update plugin title and menu option to UPnP IGD & PCP/NAT-PMP as PCP is supported and also mention UPnP IGD as UPnP is probably not specific enough.
Updated title/menu option:
UPnP IGD & PCP/NAT-PMP
Current title/menu option:
Universal Plug and Play
Port Control Protocol (PCP) is the successor to NAT-PMP and has similar protocol concepts and packet formats, but adds IPv6 support.
PCP standard:
https://datatracker.ietf.org/doc/html/rfc6887
https://en.wikipedia.org/wiki/Port_Control_Protocol
NAT-PMP std. (see 9.1 Simplicity - 9.3 for some diff. to UPnP IGD):
https://datatracker.ietf.org/doc/html/rfc6886

I don't mind updating pkg-descr, but the rest is more suited for documentation.

Sure, fine. I simply don't want to bounce translations for no operational reason.

It's also translated, but not directly wrapped with gettext due to it being embedded in XML.

The main point of using simple language and avoiding acronyms is making this more accessible to beginners.

8 years project history and me personally do disagree on this.

Can you update your PR as suggested? That's easier than guessing which part you mean.
Thanks,
Franco

Short of adding extra knobs for missing features a GUI revamp shall be exclusive to a MVC migration, which is not planned at all. The risk of regressions is just too high for no apparent benefit to current users.
Cheers,
Franco

merged, thanks!


Stop scrolling editor background when hovering tip window

When i hover on a variable to get this information / tips i have to be really careful on how i scroll for example when checking for errors on that tip window they are at the bottom so when i scroll to it and if i reach the end of it it will start scrolling the editor window.
I don't think this issue was in the earlier versions.
Made a quick video demonstrating this.
https://imgur.com/a/lprnQTe

Dups of  https://github.com/microsoft/vscode/issues/113586 and https://github.com/microsoft/vscode/issues/112373


Tutorial Page btp-app-kyma-prepare-xsuaa.md Issue. PROD

Tutorial issue found: https://github.com/sap-tutorials/Tutorials/blob/master/tutorials/btp-app-kyma-prepare-xsuaa/btp-app-kyma-prepare-xsuaa.md contains invalid tags. Even though your tutorial was created, the invalid tags listed below were disregarded. Please double-check the following tags:

software-product>sap-btp-kyma-runtime

Affected server: PROD

fixed yesterday