TypeError: sequence index must be integer, not 'str'

This is my home lab, and I am trying to learn Splunk.  I am currently running Splunk version 7.2.0.  The Splunk instance is deployed as a Docker container.  The container image is obtained from the official Splunk's dockerhub repository.
splunk@f90961e255bf:~/.local/lib/python2.7/site-packages/splunk_eventgen$ splunk_eventgen --version Eventgen 6.3.3
Following the instruction from https://splunk.github.io/eventgen/SETUP.html#pypi-installation, I am trying to exercise the eventgen the first time using eventgen.conf.tutorial1
The only changes I made in the eventgen.conf.tutorial1 is to change the outputMode to splunkstream and set the splunk password (see below).
`
[sample.tutorial1]
mode = replay
sampletype = csv
timeMultiple = 2
#backfill = -15m
#backfillSearch = index=main sourcetype=splunkd
#outputMode = stdout
outputMode = splunkstream
splunkHost = localhost
splunkUser = admin
splunkPass = password123
`
When I try to run it by executing splunk@f90961e255bf:~/.local/lib/python2.7/site-packages/splunk_eventgen$ splunk_eventgen -v generate ./README/eventgen.conf.tutorial1, I got the following error.
plunk@f90961e255bf:~/.local/lib/python2.7/site-packages/splunk_eventgen$ splunk_eventgen -v generate ./README/eventgen.conf.tutorial1 2019-03-01 05:59:42 eventgen        INFO     MainProcess Logging Setup Complete. 2019-03-01 05:59:42 eventgen        INFO     MainProcess Retrieving eventgen configurations with ConfigParser() 2019-03-01 05:59:42 eventgen        INFO     MainProcess Key 'verbose' in stanza 'global' may not be a valid setting 2019-03-01 05:59:42 eventgen        INFO     MainProcess Key 'splunkHost' in stanza 'sample.tutorial1' may not be a valid setting 2019-03-01 05:59:42 eventgen        INFO     MainProcess Key 'splunkUser' in stanza 'sample.tutorial1' may not be a valid setting 2019-03-01 05:59:42 eventgen        INFO     MainProcess Key 'splunkPass' in stanza 'sample.tutorial1' may not be a valid setting 2019-03-01 05:59:43 eventgen        INFO     MainProcess Creating timer object for sample 'sample.tutorial1' in app 'splunk_eventgen' 2019-03-01 05:59:43 eventgen        INFO     MainProcess Start '1' generatorWorkers for sample 'sample.tutorial1' 2019-03-01 05:59:43 eventgen        INFO     MainProcess All timers started, joining queue until it's empty. 2019-03-01 05:59:43 eventgen        INFO     MainProcess Start '1' generatorWorkers for sample 'sample.tutorial1' 2019-03-01 05:59:43 eventgen        INFO     MainProcess Worker# 0: Put 0.0 MB of events in queue for sample 'sample.tutorial1' with et '2019-03-01 05:59:43.109299' and lt '2019-03-01 05:59:43.109363' 2019-03-01 05:59:43 eventgen        ERROR    MainProcess sequence index must be integer, not 'str' Traceback (most recent call last):   File "/home/splunk/.local/lib/python2.7/site-packages/splunk_eventgen/eventgen_core.py", line 352, in _generator_do_work     item.run(output_counter=output_counter)   File "/home/splunk/.local/lib/python2.7/site-packages/splunk_eventgen/lib/generatorplugin.py", line 213, in run     self.gen(count=self.count, earliest=self.start_time, latest=self.end_time, samplename=self._sample.name)   File "/home/splunk/.local/lib/python2.7/site-packages/splunk_eventgen/lib/plugins/generator/replay.py", line 128, in gen     self.set_time_and_send(rpevent, self.backfill_time, earliest, latest)   File "/home/splunk/.local/lib/python2.7/site-packages/splunk_eventgen/lib/plugins/generator/replay.py", line 57, in set_time_and_send     self._out.bulksend([rpevent])   File "/home/splunk/.local/lib/python2.7/site-packages/splunk_eventgen/lib/eventgenoutput.py", line 81, in bulksend     self.flush()   File "/home/splunk/.local/lib/python2.7/site-packages/splunk_eventgen/lib/eventgenoutput.py", line 130, in flush     outputer.run()   File "/home/splunk/.local/lib/python2.7/site-packages/splunk_eventgen/lib/outputplugin.py", line 51, in run     self.flush(q=self.events)   File "/home/splunk/.local/lib/python2.7/site-packages/splunk_eventgen/lib/plugins/output/splunkstream.py", line 64, in flush     queues[row['source']+'_'+row['sourcetype']] = deque([]) TypeError: sequence index must be integer, not 'str' Exception in thread Thread-1: Traceback (most recent call last):   File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner     self.run()   File "/usr/lib/python2.7/threading.py", line 754, in run     self.__target(*self.__args, **self.__kwargs)   File "/home/splunk/.local/lib/python2.7/site-packages/splunk_eventgen/eventgen_core.py", line 361, in _generator_do_work     raise e TypeError: sequence index must be integer, not 'str'
It seems to run fine once I set the outputMode back to stdout.
Please shed some light on what I did wrong.  Thanks!

@jmeixensperger and I spoked and we identified this as a bug. We will fix it. Thanks for creating the issue.

Is there a workaround on this?  Such as switching to an older version of eventgen or switching to an older version of Splunk?

I would assume that older version of Eventgen would still not work with this. Changing to an older version of Splunk will have no effect on this.
A wordaround would be changing your outputmode to something like tcp or httpevent. Those output methods still allow you to ingest data into your Splunk instance. Is there a particular reason why you need to Splunk stream?

No particular reason, I am trying to learn how to use eventgen to inject data into Splunk.
The output mode set in eventgen.conf.tutorial1 by default is stdout, and subsequent lines of the following were commented out so my first assumption is if I only need to uncomment those lines and it would automatically inject data into Splunk.
#outputMode = splunkstream
#splunkHost = localhost
#splunkUser = admin
#splunkPass = changeme

I am learning Splunk, and the way how I get data into Splunk is by installing agent on the monitoring machines.  When you suggested to set outputMode to TCP or HTTPEvent, do you also imply configuring Splunk to listening to a specific TCP port or HTTP events as well?
Thanks!

I see.
Yes. It would be configuring settings -> data input to add either HTTP Event endpoint or TCP endpoint. I would suggest using HTTPEvent, since it is a reliable recommended way of data ingestion for Splunk. Give it a try and let me know how that goes. We are always here to help you debug it.

#133

Seems like it does the trick now after updating the outputMode to httpevent with proper httpevent settings when running it in standalone mode by executing 'splunk_eventgen generate ./Playground/Splunk/sample.tutorial1'.
outputMode = httpevent
httpeventServers = {"servers":[{ "protocol":"http", "address":"<IP_ADDRESS>", "port":"8888", "key":"8dcf0623-279f-4afe-9
httpeventOutputMode = mirror

I stopped the splunk_eventgen, installed SA-Eventgen app on Splunk, enabled modinput_eventgen and then copied the sample.tutorial1 directory to /opt/splunk/etc/apps.  After that, I restarted the Splunk instance.  I do not see any events generated.  Can you please let me know what I did wrong?

When using the app, Eventgen searches other Splunk apps to find any occurrences of eventgen.conf. There's some relevant information at the top of the tutorial page in our docs. You will have to install a TA and place your conf and sample files inside it before starting Eventgen.

Oh did not see that you are trying to use it as an app.
Basically @jmeixensperger's response + i don't think you need an outputmode specified because Eventgen app will set it for you.

Thank you Jack and Tony for your explanation and tips.
Before I submit this issue, I originally trying to use the eventgen as a Splunk app but I was not able to get any events generated and the log files were not informative so I switched over to run the eventgen as a standalone Python application.
Anyhow, perhaps I misread the tutorial, my understanding is if you want to use the eventgen as a Splunk app, all you have to do is to create a directory structure like the following in /opt/splunk/etc/apps.
/opt/splunk/etc/apps
/opt/splunk/etc/apps/sample.tutorial1
/opt/splunk/etc/apps/sample.tutorial1/default
/opt/splunk/etc/apps/sample.tutorial1/default/eventgen.conf
/opt/splunk/etc/apps/sample.tutorial1/samples/sample.tutorial1

Upon restarting the Splunk instance, if eventgen is already installed as a Splunk app, it will automatically pickup the eventgen.conf that I placed in /opt/splunk/etc/apps/sample.tutorial1/default and start generating events using the /opt/splunk/etc/apps/sample.tutorial1/samples/sample.tutorial1
Is this not the case?

That is still the case :) This works because Eventgen App automatically triggers its embedded mode and starts looking for the conf files inside of /opt/splunk/etc/apps. When you do that, no need to supply the outputmode because Eventgen App will handle it.

Correct and that is where I am having issue with.
I created a directory structure like the following in /opt/splunk/etc/apps.
/opt/splunk/etc/apps
/opt/splunk/etc/apps/sample.tutorial1
/opt/splunk/etc/apps/sample.tutorial1/default
/opt/splunk/etc/apps/sample.tutorial1/default/eventgen.conf
/opt/splunk/etc/apps/sample.tutorial1/samples/sample.tutorial1

I updated eventgen.conf and didn't supply the outputMode.
[sample.tutorial1]
mode = replay
sampletype = csv
timeMultiple = 2
#backfill = -15m
#backfillSearch = index=main sourcetype=splunkd

#outputMode = stdout
#outputMode = splunkstream
#splunkHost = localhost
#splunkUser = admin
#splunkPass = secret

#outputMode = httpevent
#httpeventServers = {"servers":[{ "protocol":"http", "address":"<IP_ADDRESS>", "port":"8888", "key":"9e77bdb9-cff4-42be-bfc0-8e2902c5089c"}]}
#httpeventOutputMode = mirror

#outputMode = file
#fileName = /tmp/internal.log

token.0.token = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3,6}
token.0.replacementType = timestamp
token.0.replacement = %Y-%m-%d %H:%M:%S,%f

token.1.token = \d{2}-\d{2}-\d{4} \d{2}:\d{2}:\d{2}.\d{3,6}
token.1.replacementType = timestamp
token.1.replacement = %m-%d-%Y %H:%M:%S.%f

token.2.token = \d{2}/\w{3}/\d{4}:\d{2}:\d{2}:\d{2}.\d{3,6}
token.2.replacementType = timestamp
token.2.replacement = %d/%b/%Y:%H:%M:%S.%f

token.3.token = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}
token.3.replacementType = timestamp
token.3.replacement = %Y-%m-%d %H:%M:%S

token.4.token = \d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}
token.4.replacementType = timestamp
token.4.replacement = %Y-%m-%dT%H:%M:%S

SA-Eventgen is already installed as Splunk App but when I restart the Splunk instance, no new events were being generated.  I am not sure what I did wrong.

did you set your app "sample.tutorial1" setting to global? Eventgen App can't access it if it isn't global setting. You can change this in "manage apps". Then underneath the "sharing" column, please set that to Global.

That was it!  Thanks Tony
I don't think that criteria was mentioned in the tutorial document.  Perhaps it should be updated?

Thanks for the feedback. Will update it. Closing the issue now.


Identify which lessons/tutorials qualify as an MVP of the Distributed Web Primer

There are currently a handful of lessons in the decentralized web primer. The finished parts center around:
Tutorial 1: Install and Initialize IPFS
Tutorial 2: Files on IPFS
Tutorial 3: “Turn a File into a Merkle Tree”

What parts to we need to add in order to consider it a complete "First Draft"? Pinning & Deleting? etc..
What's the best way to handle release/branch management? -- the working branch will have unfinished stuff in the TOC and unfinished lessons while the "published" branch should have only stable, complete docs. How should we handle this?

Related discussion on discourse http://ipfs.trydiscourse.com/t/decentralized-web-primer-the-beginnings-of-an-ipfs-book/40


Deprecate more requirements.

These are ones that were either already deprecated due to audit rules or are just a simple which with a default_formula so should just be a dependency.
Additionally, audit more deprecated requirements and cleanup optional system dependencies, GPG code.
CC @ilovezfs @DomT4 as we've talked about this tangentially in the GPG requirement discussion.
Closes #3591

Any usage of Gpg. in formula seems to be blowing up now, FWIW:
==> Postinstalling rpm
Error: uninitialized constant Formulary::FormulaNamespace0687b9ea3596758a5cc31a58c3aca98d::Rpm::Gpg
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/rpm.rb:52:in `post_install'
/usr/local/Homebrew/Library/Homebrew/formula.rb:977:in `block (2 levels) in run_post_install'
/usr/local/Homebrew/Library/Homebrew/formula.rb:834:in `with_logging'
/usr/local/Homebrew/Library/Homebrew/formula.rb:976:in `block in run_post_install'
/usr/local/Homebrew/Library/Homebrew/utils.rb:549:in `with_env'
/usr/local/Homebrew/Library/Homebrew/formula.rb:968:in `run_post_install'
/usr/local/Homebrew/Library/Homebrew/postinstall.rb:16:in `<main>'

Presumably that can be resolved by a require "gpg" in formula.rb but this is a little more mysterious to me:
irb(main):002:0> require "gpg"
NameError: undefined local variable or method `module_function' for Gpg:Class
Did you mean?  module_exec
	from /usr/local/Homebrew/Library/Homebrew/gpg.rb:4:in `<class:Gpg>'
	from /usr/local/Homebrew/Library/Homebrew/gpg.rb:3:in `<top (required)>'
	from /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from (irb):2
	from /usr/local/Homebrew/Library/Homebrew/brew.rb:100:in `<main>'

@DomT4 should be fixed in https://github.com/Homebrew/brew/commit/51114139e1c6b442890adec620ffc42975e8fd30.

Aye, that resolves those two, thanks!
New one:
irb(main):002:0> Gpg.available?
=> false

Happy to take a look soon. Away from home at the moment but should have some time later tonight.

homebrew/core deprecation of :gpg https://github.com/Homebrew/homebrew-core/pull/22240

homebrew/core deprecation of mysql https://github.com/Homebrew/homebrew-core/pull/22241

homebrew/core deprecation of :postgresql https://github.com/Homebrew/homebrew-core/pull/22242

homebrew/core deprecation of :rbenv https://github.com/Homebrew/homebrew-core/pull/22243


Database content different with the one from LibChecker-Rules repo

After running diff between the rules.db from rules repo and the one included in this library, I wonder why these differences exist?

The database in the LibChecker-Rules repository is now relying on scripts to generate it, maybe the scripts are written a bit wrong, I'll keep an eye on it.

resolved in V32.


Kivy usage with 10" touchscreen on Raspberry Pi

Dear developers.
I have been developing several projects with Raspberry Pi connected with Official 7" Raspberry Pi touch screen.
Now, I need to expand my screen to 10" touch screen.
I have investigated about 10" touchscreen for several days and found one which seems like possible to use Kivy.
http://www.waveshare.com/wiki/10inch_HDMI_LCD_(with_case)
I wonder if you could confirm that this is capable with Kivy on Raspberry Pi.

I am using an 8 inch touchscreen that seems to use a similar controller.  The touchscreen on mine is the eGalax which is a common touch screen controller.
The touchscreen took a while to sort out but it works perfectly now. You will probably need to use the min/max_abs_x/y parameters in the [input] section of.kivy/config.ini and there is a bug you need to patch before kivy will use them. The eGalax reports absolute co-ordinates which is why you need them.
A very useful utility to install is evtest which is a command line program which will stream the touch screen values continuously and, by touching the corners etc, will indicate the values that your screen is generating.  The evtest output is then used to set the values in
You may need to update the tslib for eGalax as earlier code didn't recognise it.
The screen was also a PITA and I had to try a number of settings before I got the settings that matched the screen (800x600 in my case).  The various utilities that report the screen capabilities didn't help much as the screen didn't report correctly. In the end it was the HDMI settings in Rpi config.txt that nailed it for me.  After trying many combinations, I eventually set the hdmi_group and hdmi_mode specifically to the mode I wanted and voila!
I am using Jessie with Kivy 1.9.1.
In conclusion, I'm sure the screen you are considering should work with Rpi.

GitHub issues are meant to be used for bugs or feature requests, not general Q&A usage. Please use Kivy's Google group or IRC channel.
That said, Kivy can use any compatible screen connected to the Pi.

As mentioned by @hunternet93, this is not the right place for such questions.


Add contributors

Add @shumonsharif and @dufoli as contributors

@all-contributors add @shumonsharif and @dufoli for code and maintenance

@all-contributors add @dufoli for code and maintenance


WoL: AP Hatchery worker rally not working

The ability AP_RallyHatchery uses two validators from Swarm.SC2Mod (IsDroneOrOverlord and NotDroneAndNotOverlord), which causes drones spawned at hatcheries in WoL maps to move to the combat rally point instead of the worker rally point.
Example from Great Train Robbery with a Hive plopped down (and player race switched to Zerg for testing):

That got already fixed IIRC


Unable to run native XLA AOT tests

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): macOS Mojave 10.14.4
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
TensorFlow installed from (source or binary): source
TensorFlow version: r1.12
Python version: python 3.6.3
Bazel version (if compiling from source): 0.18.1
GCC/Compiler version (if compiling from source): Apple LLVM version 10.0.1 (clang-1<IP_ADDRESS>)
CUDA/cuDNN version: N/A
GPU model and memory: N/A

Describe the problem
There is a set of native tests in tensorflow/compiler/aot/tests. I am not able to run them using bazel because of a linking error while building the rule tfcompile_test in .../tests/BUILD. Specifically, tfcompile_test depends upon test_graph_tfmatmulandadd_with_profiling and test_graph_tfmatmulandadd which fail to link.
Code to reproduce the issue
In an untouched Tensorflow repo with r1.12 checkout and bazel 0.18.1 installed, run
bazel build //tensorflow/compiler/aot/tests:tfcompile_test
Describe the current behavior
Error message is as follows:
/Users/willxujun/code/tensorflow/tensorflow/compiler/aot/tests/BUILD:225:1: Linking of rule '//tensorflow/compiler/aot/tests:tfcompile_test' failed (Exit 1) Undefined symbols for architecture x86_64:  "___tfcompile_MatMulAndAddCompWithProfiling_HloProfilePrinterData_protobuf_array_contents", referenced from:       MatMulAndAddCompWithProfiling::StaticData()::'lambda'()::operator()() const in tfcompile_test.o   "___tfcompile_MatMulAndAddComp_ProgramShape_protobuf_array_contents", referenced from:       MatMulAndAddComp::StaticData()::'lambda'()::operator()() const in tfcompile_test.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) Target //tensorflow/compiler/aot/tests:tfcompile_test failed to build
The problem seems to stem from constructing MatMulAndAddCompWithProfiling or MatMulAndAddComp objects in tfcompile_test.cc. These two classes are graphs compiled by XLA.
Disabling the relevant tests in tfcompile_test.cc makes the issue go away.
Alternatively, in .../aot/tests/BUILD, removing tfcompile_flags from test_graph_tfmatmulandadd, and setting enable_xla_hlo_profiling to False in test_graph_tfmatmulandadd_with_profiling makes the issue go away.
This problem seems to happen to only macOS. I could run the same command without issues in Ubuntu 16.04.
This problem seems to be related to the linking of dependencies. I could successfully run the individual tf_librarys and get the compiled graph classes in bazel_genfiles/, and both undefined symbols are "extern C" variables found in the graphs' header files, and it seems the root definition is not found somehow. As a result, the graph objects cannot be constructed.
Describe the expected behavior
In Ubuntu 16.04, build succeeds with the following:
INFO: Build completed successfully, 3560 total actions
Then a tfcompile_test executable would be produced in bazel_bin/tensorflow/compiler/aot/tests, which can be run successfully.
Other info / logs
N/A

@willxujun Request you to please follow the link, as per the reference the bazel version for 1.12 is 0.15.0

Re-run using Bazel 0.15.0 exhibits identical behaviour

TensorFlow version: r1.12

Does this reproduce at git head?

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!


Enhancement: Supported multi-monitor screenshot.

It doesn't support multi-screen when I screenshot, So I make this.
I don't know if this has some conflicts with the project.
The new calling method of ScreenShot is below this:
screenCapturer = new ScreenCapture();
screenCapturer.SnapCompleted += ScreenCapturer_SnapCompleted;
screenCapturer.SnapCanceled += ScreenCapturer_SnapCanceled;
screenCapturer.Capture();

https://github.com/WPFDevelopersOrg/WPFDevelopers/pull/19


[bug] Stale connection retries

Bug
If the internal notifier connection was not used for approx. 1 minute it goes stale and needs to be retried on the next occurrence, this results in an "Internal Notifier Error" error.
To fix this we should implement 2 measures:

Add retry / backoff logic for random transport errors in tonic / h2
Open a new backend connection for each user request

This will be mitigated by: https://github.com/ArunaStorage/ArunaServer/commit/6ffb29a2d70952ab8eab4be8a0e2b8fcef9bd605
For a proper retry logic we should wait for: https://github.com/hyperium/tonic/issues/733
and: https://github.com/tower-rs/tower/issues/682


Identifier (e.g. className) uglifying or hashing

This may be either a question or a feature request.
I used to develop React application with Webpack, and one of the most useful gimmicks of Webpack was its css-loader's localIdentName. What localIdentName does was to give the reference to the css selector with a hashed name.
For example, if there is a style definition as following,
.foo {
  color: blue;
}

I could reference foo in my JavaScript file, but the name could be hashed or 'uglified' as in 12lk1j.
var styles = require('./file.css');
styles.foo // 12lk1j

I now use styled-components, refraining from defining styles in external css files. I, however, still need the feature (uglified css identifier) and currently have to define my own custom function (css name resolver) to make it happen. But I'm asking, would there be any chance you add this functionality (in whatever way) into the library? Or is there anyone in a similar situation who uses his own tricks?

Could you clarify a bit more? Our classes are normally hashed and so exactly what you want?

I also don't understand this question. If you want human readable names, use https://github.com/styled-components/babel-plugin-styled-components!
If you don't want human readable names, then don't use it.

The issue is closed but I late explain what I have asked, just to make it clear.
I was trying to do something like this.
styled.div`
   // someStyle
   .child {
      display: inline-block;
   }
   .child2 {
      display: block;
   }
`

So when writing render(), for example, I don't have to wrap all the html elements with styled-component.
return (

)

When I used css-loader in a similar situation, the names of selectors child1, child2 could be hashed if I wanted. I particularly liked the feature with regards to security, since I could intentionally remove semantic information about the html elements. styled-components do also hash the classnames but not what is written inside the template literal.
However, I realized that I'd better use this library while having possibly every html element that I would like to style wrapped with styled-component (especially if I would like to hash all those elements). At least it is what I understand. It just took some time for me to get used to it. (If I'm wrong here, I'm glad to be corrected)

The correct usage of styled-components in your example would be:
const Parent = styled.div`
     // styles
`

const Child1 = styled.div`
    display: inline-block;
`;

const Child2 = styled.div`
    display: block;
`

const Comp = () => (
    <Parent>
        <Child1>
               ...
        </Child1>
        <Child2>
               ...
        </Child2>
    <Parent>
)

If I understand you correctly that's the thing you are moving to which is good :)

I want to know what condition className is hashed based on(the first className). can someone help me?


Fix NCTL For Fast-Sync

NCTL needs to be fixed for the feat-fast-sync branch, as well as compiling without the casper-mainnet flag.

Closed by #1895


add pod distruption budjet to kubemcapool

add pod disruption budjet to kubemacpool to set
the minimum number of available kubemacpool pods
on one node to 1 (on volentary distruption)

I made a mistake with the previous PR and not included namespace...

I closed this PR (which is an override on an original PR to add podDisruptionBudget)  because in the PR I added a namespace metadata  which is not necessary because kustomization already adds the namespace


add wechaty document

#73 #252

Thanks! Please get at least three votes(approved) from our contributor team first, because the document needs to be improved heavily and continuously.

here is the auto generate doc: https://github.com/lijiarui/wechaty/blob/new-doc/docs/index.md
not sure why it has double wechaty intro....

maybe we should add all return promise type with notice "await"

I saw lots of ISC License in the generated document, it's not right.
The license of Wechaty project is: Apache-2.0
There might have some comments say "ISC license", should be replaced.

ISC License should be changed in whchaty.ts @line#68

Thanks, @kis87988, I had fixed that by removing the LICENSE info.

Coverage remained the same at 54.688% when pulling 5a7802b85c69becebf44312cae75b02ad36248a1 on lijiarui:master into d3aa2516c423b62dd2c7a2601695533ad344ed67 on Chatie:master.

look forward to merge

@binsee Could you help to approve this pull request? then @zixia can merge this, thanks!

Great! Thanks for all approving this PR, appreciate that!


event attribute value for sdk.AttributeKeyModule is incorrectly determined with ibc-go Msg types

          I think it seems to have some problem if ibc case.

ex) ibc.applications.fee.v1.MsgRegisterPayee
Originally posted by @zemyblue in https://github.com/Finschia/finschia-sdk/pull/1063#discussion_r1287908725

          I will add logic not to append attribute `module` in case of `ibc`. It would be addressed by another PR.

Originally posted by @0Tech in https://github.com/Finschia/finschia-sdk/pull/1075#discussion_r1289935807


Ace.createEditSession method signature

The Ace.createEditSession method is defined as following:
Document createEditSession(Ace.CreateEditSessionTextUnionType text, Object mode);

But the mode param is not simply an object but it's referring to the Mode object used for instance to syntax highlight  the code. The TextMode definition should be here into ACE project.
In Closure extern file the method is defined as following:
/**
 * Creates a new EditSession, and returns the associated Document.
 * @param {(!ace.Document|string)} text
 * @param {?} mode // TODO: define !ace.Mode.
 * @return {!ace.Document}
 */
ace.createEditSession = function(text, mode) {};

but in the ACE source, is following:
/**
 * Creates a new [[EditSession]], and returns the associated [[Document]].
 * @param {Document | String} text {:textParam}
 * @param {TextMode} mode {:modeParam}
 *
 **/
exports.createEditSession = function(text, mode) {
    var doc = new EditSession(text, mode);
    doc.setUndoManager(new UndoManager());
    return doc;
}

The Mode type was a bit weird - I didn't find an obvious way to create the type declaration so that it made sense in JsInterop, but I didn't try for very long. As I understand it, it depends on how you are loading the various Modes that you are working with - if they are included via a script tag, or the require() call that ace can optionally provide, etc.
From the work you did before with Ace, how did you use modes: how did you include your text modes, etc?

@niloc132 in the prototype I did, the mode files were injected at runtime, when required. I included all js files into the project, I mapped them into a ClientBundle, create a singleton for every MODE and injected, when the mode will be requested.
The ACE createEditSession was as following:
EditSession createEditSession(Document text, String mode);

because (as coded here) the mode param could be the OBJECT (but as you wrote, almost not possible to retrieve if if now I'm curious if there is a way) or the name of mode itself, as string.

@niloc132 Ace.require sounds the right way to have that mode object. Unfortunately that method is used to get other kind of modules (not only modes). I'm testing some implementations but not simple to find out a generic way to use it.

For ease of use, it might make sense to provide some handmade java classes which abstract over that require method to provide the ability to download another mode at runtime.
We could also just use the String overload and remove the others as you suggest, and then ace will do the download for us? Or do you see an advantage to providing Mode instances, letting the developer interact directly with them?

From ACE doc:

The mode can then be used like this:
    var JavaScriptMode = ace.require("ace/mode/javascript").Mode;
    editor.session.setMode(new JavaScriptMode());

Sounds difficult to me to be able to implement it by using only JsInterop.

Okay, let's plan on that for the time being. I'm not entirely certain how that arequire().Mode works, but I'll take a look - it looks like it would need to do blocking IO, which isn't really ... a good idea.
We'll resolve this for now by providing the string only - I'll get CI fixed and then make this change. Will probably diff in a TODO also so that it is clear that we don't want to leave it this way.

Fully agree to go to string only!

Going to string implementtion, the EditSession constructors should be updated as well


Add a default favicon

Added a small default favicon into the header section.

Hi @cremich, thanks for using the theme and thanks for taking the effort to make this pull request! The favicon you created makes it look like the theme is aimed at developers, while I would like anyone to be able to use the theme. I'm in favor of including a favicon, but I would like it to be a more neutral one. Sorry! :pray:

No problem 😉 Thanks for your review. In my opinion an interesting feature would be:
Let the hello-theme user define what favicon he wants to use. So there should be a chance to override the favicon.
What do you think about the following emoji as a default:
👋

I love that suggestion! And since there's no favicon defined in the <head>, there is currently no option to use a favicon at all. That's a very good point. I would welcome a new PR if you have the time, otherwise I'll look into this myself. 🙂

Hi @cremich, just to let you know: I just added the favicon you suggested.


Fixed #34171 -- Fixed QuerySet.bulk_create() on fields with db_column in unique_fields/update_fields.

Ticket 34171
I think we don't need separate test for this, just adding a db_column to one of the models in bulk_create should do the work.

@DevilsAutumn Thanks :+1: I rebased after 7d5329852f19c6ae78c6f6f3d3e41835377bf295, and changed naming because all fields with db_column are affected, not just fields with mixed case column names.

Cool 👍


Optical: There is a button called "RT function" obscuring the button "activation map"

Corrected by commit 1e0a609c005cc9db9309810bdf20d63ace393e73
Patch will be normally applied in v1.1-alpha-4


n-wise list splitting

:+1: for the function -1 for the name. I would definitely call this chunk

i think of chunk as different, viz:
chunk(2, [1, 2, 3, 4, 5]); //=> [[1, 2], [3, 4], [5]]
nwise(2, [1, 2, 3, 4, 5]); //=> [1, 2], [2, 3], [3, 4], [4, 5]]

I immediately think of a moving average that could be implemented as a mean aggregate of each of these chunks... so maybe movingWindow or something like that?  There has to be a name for this...  maybe it's nwise :)

i am willing to rename. nwise is probably better thought of as all n-tuples, rather than just consecutive ones. so open to pitches for new names.

consecutive?
sequent?

I like something like (moving|sliding|rolling)(Window|Buffer)
It's almost the outer loop of a convolution (sliding a 1d kernel), I'm surprised I can't find a name for this...

overlap?

Whats a use case for this - I thought it was chunk when I saw it

this was inspired by the discussion on #872

aperture?

pairwise seems so obvious:
R.pairwise(['a', 'b', 'c', 'd']); //=> [['a', 'b'], ['b', 'c'], ['c', 'd']]

that it really would be nice if the general name was an abstraction of that.  But I can't think of one.  apeture would be the best so far except that, like most suggestions here, it's a noun.  If we can't have a verb, I'd really rather have an adjective.  :smile:

dilate, regurgitate, exfoliate? ;)
You could drop the ing to make a verb  (move|slide|roll)(Window|Buffer) but that's not nearly as fun.
Eh. Whatever works.

pairwise should really be [1, 2, 3, 4] => [[1,2], [2,3], [3,4], [1,3], [1,4], [2,4]] so anything-wise ia not accurate.
lens also not a verb. let's not overconstrain ourselves. I'm for aperture.

:+1: for aperture

ping

:ram:


bindgen doesn't seem to find lmdb.h

Crate name
mmtkvdb
Build failure link
https://docs.rs/crate/mmtkvdb/0.0.1/builds/539741
Additional details
With version 0.0.1 of mmtkvdb I get:
/opt/rustwide/workdir/lmdb-bindgen.h:1:10: fatal error: 'lmdb.h' file not found

If you want to add native C dependencies to the build environment, you can do so as long as they're packaged in Ubuntu LTS: https://forge.rust-lang.org/docs-rs/add-dependencies.html

The missing Ubuntu package would be liblmdb-dev:
diff --git a/linux/packages.txt b/linux/packages.txt
index ce6dda4..2850b70 100644
--- a/linux/packages.txt
+++ b/linux/packages.txt
@@ -507,6 +507,7 @@ libleptonica-dev
 libllvm6.0
 libllvm7
 libllvm8
+liblmdb-dev
 liblocale-gettext-perl
 liblsan0
 libltdl-dev

However, I get the same error message when I try to follow the instructions at https://forge.rust-lang.org/docs-rs/add-dependencies.html
2022/04/10 22:37:40 [INFO] rustwide::cmd: running `Command { std: "docker" "start" "-a" "0551a4a5fa1e1c549026d6265497014d1c6bffc6a754316ca2e9010b9e0a8df6", kill_on_drop: false }`
2022/04/10 22:37:41 [INFO] rustwide::cmd: [stderr]    Compiling mmtkvdb v0.0.1 (/opt/rustwide/workdir)
2022/04/10 22:37:41 [INFO] rustwide::cmd: [stderr] error: failed to run custom build command for `mmtkvdb v0.0.1 (/opt/rustwide/workdir)`
2022/04/10 22:37:41 [INFO] rustwide::cmd: [stderr]
2022/04/10 22:37:41 [INFO] rustwide::cmd: [stderr] Caused by:
2022/04/10 22:37:41 [INFO] rustwide::cmd: [stderr]   process didn't exit successfully: `/opt/rustwide/target/debug/build/mmtkvdb-b9dbc448d595b166/build-script-build` (exit status: 101)
2022/04/10 22:37:41 [INFO] rustwide::cmd: [stderr]   --- stdout
2022/04/10 22:37:41 [INFO] rustwide::cmd: [stderr]   cargo:rerun-if-env-changed=LMDB_INCLUDE
2022/04/10 22:37:41 [INFO] rustwide::cmd: [stderr]   cargo:rerun-if-env-changed=LMDB_LIB
2022/04/10 22:37:41 [INFO] rustwide::cmd: [stderr]   cargo:rerun-if-env-changed=LMDB_LIBNAME
2022/04/10 22:37:41 [INFO] rustwide::cmd: [stderr]
2022/04/10 22:37:41 [INFO] rustwide::cmd: [stderr]   --- stderr
2022/04/10 22:37:41 [INFO] rustwide::cmd: [stderr]   /opt/rustwide/workdir/lmdb-bindgen.h:1:10: fatal error: 'lmdb.h' file not found
2022/04/10 22:37:41 [INFO] rustwide::cmd: [stderr]   /opt/rustwide/workdir/lmdb-bindgen.h:1:10: fatal error: 'lmdb.h' file not found, err: true
2022/04/10 22:37:41 [INFO] rustwide::cmd: [stderr]   thread 'main' panicked at 'unable to generate bindings: ()', build.rs:34:14
2022/04/10 22:37:41 [INFO] rustwide::cmd: [stderr]   note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Note that setting the LMDB_ environment options shouldn't be necessary if lmdb.h is in the default include directory (as installed by liblmdb-dev).
The documentation seems to be outdated and/or incomplete; there were some issues:

docs.rs needs to be cloned before you can change to /path/to/docs.rs
in docs.rs no .env file is included, but it is needed for docker-compose (touch .env seems to help?)
environment variable DOCS_RS_LOCAL_DOCKER_IMAGE is reported as outdated and was suggested to be replaced with DOCSRS_DOCKER_IMAGE

Manually compiling a small C-file like
#include <lmdb.h>

int main() {
    return 0;
}

seemed to work, but I'm not sure if the right docker image has been used for the crate building. Perhaps someone with more knowledge could look into this problem and add liblmdb-dev to the build environment and see why/if mmtkvdb doesn't compile?

in docs.rs no .env file is included, but it is needed for docker-compose (touch .env seems to help?)

Apparently the .env file needs to be created by copying .env.sample to .env (according to the docs.rs README).
This seems to work (but still trying).

Works now; waiting for pull-request to be merged.
Maybe someone can trigger re-building the docs for mmtkvdb version 0.0.1 afterwards?

Two more issues I had with the documentation:

docker build —tag build-env . must be run in directory linux inside crates-build-env
the lint.sh file is in the root directory but not in ci/ as mentioned in the manual

Feel free to make a PR fixing the docs :) I'll try to get to it in the next few weeks if you can't find time. https://github.com/rust-lang/rust-forge/blob/master/src/docs-rs/add-dependencies.md
Thanks for testing the instructions out in such detail! I think you may be the first person to do so 😆

I can try to do that with my colleague in the next days. Can you trigger re-building the docs for mmtkvdb or do I need to contact someone else for that?

Docs.rs won't use the new build environment until the next night we deploy it, but I'll make sure your crate gets rebuilt at that time.

Okay thanks, I'll leave this ticket open until I (or you) looked into the doc issues, so it doesn't get forgotten.

Building works fine now. The documentation has been fixed in the pull request by @bswrczk (see here), but that PR has not been merged yet.

Glad to hear it!


Feature/dashboard page

Since there is a V1 that needs to be merge into this feature.
I will close this PR.


Team 4 Sprint 2: Creating a basic functional countdown

Description
Task: Creating a basic functional countdown
Feature: Countdown Clock(#3)
The task is to implement a functional countdown feature. The basic functionality - the countdown - should be implemented for now. Adding time, starting, pausing can come later.
Completion Deadline: Sept. 8
Member

Rio Tanabe (@riotanabe ) (riotanabe)

Hey guys, is this at a service level that we can interact with yet? Trying to make consumeable items add time to it

Hey guys, just checking in, how do you plan on making the timer accessible to other components?

@riotanabe hey man, I've added in some basic countdown logic time increase work. Unsure if you were looking into this. Just on this commit here 8d98cbd

@riotanabe hey man, I've added in some basic countdown logic time increase work. Unsure if you were looking into this. Just on this commit here 8d98cbd

Thank you. I was looking into the time adding component but were struggling a bit. Is the time increase working, or should I look into it a bit further? Also has it been merged with main?
Thanks.

Thank you, we were thinking of adding it in.
Is the time adding feature working? If not I’ll implement it by the deadline.
Kind Regards,
Rio Tanabe
10/9/2022 15:31、Connor Ryan @.***>のメール:

@riotanabehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Friotanabe&data=05|01||dcff187683e3429e37fb08da92f60939|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637983882685263530|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=bwG6gXBH%2BXgnU8iR1ptYN8jRnWKPimW14d4JGJwkndQ%3D&reserved=0 hey man, I've added in some basic countdown logic time increase work. Unsure if you were looking into this. Just on this commit here 8d98cbdhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FUQdeco2800%2F2022-ext-studio-1%2Fcommit%2F8d98cbd2b0c36fa831ad6ed05b53c7185f1a46fe&data=05|01||dcff187683e3429e37fb08da92f60939|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637983882685263530|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=AepCmJPaKRkjC4yvQj5r2bMPM%2FBPEmj6gs%2BBq6qWgTA%3D&reserved=0
—
Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FUQdeco2800%2F2022-ext-studio-1%2Fissues%2F78%23issuecomment-1242645422&data=05|01||dcff187683e3429e37fb08da92f60939|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637983882685263530|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=R3Kd84X5BoQ2FS9raSETiHtGy3hrLrj%2Fb3TrtMSullU%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAVXBXYD5ZPYBV6V6DXL66WLV5QTKPANCNFSM6AAAAAAQHR3P5U&data=05|01||dcff187683e3429e37fb08da92f60939|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637983882685263530|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=xlIg4Op36MNzYCI%2BQObApxt%2FavcaSMN5%2Fa10ZpMmVPQ%3D&reserved=0.
You are receiving this because you were mentioned.Message ID: @.***>


How much drift is expected and what contributes to more drift vs less?

I've got a few different AR demos we're working on. The majority of the 3DObjects/Videos/Models have some amount of drift. I'm wondering how much is to be expected and what are the various contributors to drift?
In this video, I'm using a Image Marker to displaying a ViroVideo component ontop of the recognized image. When I move around, the video doesn't stay consistently on top of the image.
Here's the code for that example:
<ViroARImageMarker
          onAnchorFound={this._onAnchorFound}
          target={"targetOne"}
          pauseUpdates={this.state.pauseImageMarkerUpdates}>

          <ViroNode ref="objectNode" position={[0, 0, 0]} rotation={[-90, 0, 0]} dragType="FixedToWorld" onDrag={()=>{}}>

            <ViroVideo
              source={require('../../assets/videos/mural.mp4')}
              width={0.215}
              height={0.2794}
              />

         </ViroNode>

        </ViroARImageMarker>

In this example, when a load a 3DObject into the view, as I move around the object, slight bumps and twitches can be seen. Again subtle, but obviously it would be great to mitigate.
Here's the code for that example:
<ViroARImageMarker
          onAnchorFound={this._onAnchorFound}
          target={"targetOne"}
          pauseUpdates={this.state.pauseImageMarkerUpdates}>

          <ViroNode
            ref="objectNode"
            >

            <Viro3DObject
             source={require('../../assets/objects/vianneyhouse.glb')}
             onLoadStart={this._onLoadStart}
             onLoadEnd={this._onLoadEnd}
             onError={this._onError}
             onClick={this.props.onObjectClick}
             scale={this._getAdjustedScale()}
             position={this._getAdjustedPosition()}
             type="GLB" />

         </ViroNode>

        </ViroARImageMarker>

In this thread, the pauseUpdates attribute is mentioned to help with drift, which I'm using, but obviously doesn't totally fix the problem.
I also noticed you were adjusting the position and scale of the objects being added. How do those properties relate to drift?

Hi @bryantAXS i'm having the same problem. Sometimes the Node is also rotated in strange way. I think it depends on how the marker is found.
Hope to find a solution because it's annoying

Hi @bryantAXS,
I took a look at your first video, and it looks like what's actually happening is that your virtual component is being rendered "behind" your physical component. As such, this is causing the virtual component to move less relative to the camera than your physical one. This usually happens to image markers whose targets have a slightly off physicalWidth (refer to ViroARTrackingTargets).
In your second video, the fidgeting in the model seems to be a result of poor camera tracking (the room looks a bit dim and the table is featureless) as opposed to poor image recognition (to test this, place another object in the scene not tied to a marker and see if it too exhibits the fidgeting).
As for the pauseUpdates property, this is advised if you plan on using ViroARPlane components because both their rotation property get updated as you move around, so if you were to place an object within a ViroARPlane, it would continue to rotate as the extent of the plane is being discovered. For a ViroARImageMarker however, you would want at least a few updates in case the first "find" of the marker isn't a very accurate one, or simply leave pauseUpdates to false as it will recalibrate the position of the marker even if you experience camera drift.
In that thread you were looking at, the person didn't properly scale and position their object so the portal they were using looked right until they started moving because the object was positioned about 1-2 feet into the ground. This is probably what is happening with your first video. If you walk all the way to the "side" of the image, you might notice that your virtual image being positioned "further" from your original position than the physical one.
Let me know if you still are seeing issues, thanks!

@achuvm as you can see I'm having problems with stability too
https://www.dropbox.com/s/4w3an64xfrr4ctx/UKCT5304.MP4?dl=0
how can i make the image to stand attached to the marker?
<ViroARImageMarker
              target={arVideoMarker}
              key={arVideoMarker}
              onAnchorFound={anchor => _this.arVideoFound(anchor)}
              onAnchorUpdated={anchor => _this.arVideoUpdated(anchor)}
              pauseUpdates={false}
              // pauseUpdates={_this.state.pauseArVideoUpdates}
            >
              {_this.state.foundAnchor && (
                <ViroImage
                  width={0.09}
                  height={0.09}
                  position={[0, 0, 0]}
                  rotation={[-90, 0, 0]}
                  source={require('../../../../assets/images/ar_targets/ar_video.png')}
                />
              )}
            </ViroARImageMarker>

Hi @jonnyscholes,
Thanks for the heads up, and yes, since we do use ARCore/ARKit, this tool should help with Image Recognition on Viro.
I'll go add it to our documentation!


Asynchronous refactor || Project Modernisation

Description
Simplified many code flows in this library for better readability.
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

Checklist:

[ ] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules
[x] Changes have been manually tested (please provide information on test platform using the fields below)

Coverage increased (+29.07%) to 75.758% when pulling 13bf2f8aba26e0f66e421cff4ca7eb854df18ba1 on remove-async-dep into 4c88107415fec46eacabf81fb2fb8a6f271b7828 on master.


Neuroglancer blocked image adapter

This PR holds the Neuroglancer blocked image adapter, to be continued in the future.

@dickscheid Perhaps you could create a branch as a container for this work, and accept the pull request there? This is helpful as it allows readily showing interested folks (e.g., the blockedImage developers!)  the integrated code, rather than navigating code changes.

@vijayiyer05 we created the branch feat_blocked_image for this, and merged this PR. Will pickup work on the blocked images later again then.


how to use threadpool with latest version disruptor

The support for executors + thread pools has been explicitly dropped.
Looking at an older version
 @deprecated Use a {@link ThreadFactory} instead of an {@link Executor} as a the ThreadFactory
  is able to report errors when it is unable to construct a thread to run a producer.`

I'd recommend migrating to a thread factory.

@isolgpus is there an example of this in the documentation? If not, could be useful to add it?


Adding G-Reg 5.2.0 Scripts

Please review and merge.

Hi Thushara,
I see that your publisher and store profiles for default platform have Kubernetes related configuration data. You can have a look at Greg 5.1.0 hieradata to get an idea of how default and kubernetes platforms are separated.


[BUG] Failed to drain nodes because IM pods are unable to evict due to timeout

Describe the bug
After adding/removing new nodes the cluster is stuck in a deadlock where the control plane cannot be updated, since the system tries to evict a pod that is not existant anymore in the cluster and is stuck since the task can't be completed.
[controlPlane] Failed to upgrade Control Plane: [[error draining node s1: [error when evicting pods/"instance-manager-e-689a397e" -n "longhorn-system": global timeout reached: 2m0s, error when evicting pods/"instance-manager-r-68277463" -n "longhorn-system": global timeout reached: 2m0s]]]
--
To Reproduce
it randomly occured after removing two nodes and adding 2 nodes right after
Expected behavior
longhorn automatically detect if manager instances not present anymore and resolved the deadlock
Environment

Longhorn version: 1.3.1
Installation method (e.g. Rancher Catalog App/Helm/Kubectl): rancher catalog
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version: 1.23.7

Number of management node in the cluster: 1
Number of worker node in the cluster: 3

Node config

OS type and version: ubuntu

Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal): hetzner
Number of Longhorn volumes in the cluster: ~60

Additional context
this is quite critical since we cannot add any further nodes to the cluster as long as it is in error state. Is there any way to remove the error state manually, since the pods that are to be evicted are not existing anymore anyways.
Thanks!

cc @PhanLe1010 @shuo-wu @innobead  any idea?

@michafn It seems like you use rke distro. Do you have logs just before hitting global timeout? We would like to see the exact error returned from the Kube API server. ref: https://kubernetes.io/docs/concepts/scheduling-eviction/api-eviction/#how-api-initiated-eviction-works.

[controlPlane] Failed to upgrade Control Plane: [[error draining node s1: [error when evicting pods/"instance-manager-e-689a397e" -n "longhorn-system": global timeout reached: 2m0s, error when evicting pods/"instance-manager-r-68277463" -n "longhorn-system": global timeout reached: 2m0s]]]

The possible reason should be related to PDB protection of volume, so ensure there is at least another healthy replica in other nodes before the last replica of a volume on the node is being evictd, otherwise, the eviction will not succeed. ref: https://longhorn.io/docs/1.3.1/volumes-and-nodes/maintenance/#updating-the-node-os-or-container-runtime
P.S. Kube API eviction should always get the active pods from the evicting node, then evict them. But, what you mentioned is the Kueb API server tried to evicit absent IM pods to stay in a stuck situation/deadlock, but this seems not quite clear and uncertain.

I guess the last replica of some volumes blocks the instance manager eviction.
By default, there are 3 replicas for each volume. In your 3-worker-node cluster, evicting 2 nodes in a short time means each volume will lose 2 replicas and it's too short to build new replicas. In order to prevent the last replica from being unavailable, Longhorn will retain the PDB for the last set of instance managers then the replicas inside them can be retained.

it's quite hard to get anything meaningful from the api logs, since it is not entirely clear which log messages are connected with the problem. I grepped one log sequence associated to the mentioned manager instance below.
I understand the topic with the missing replicas, but if i interpret the above error message correctly, the problem is on node s1, which is not even scheduled for deletion, where as node l1 (which is scheduled for being removed) still runs and acocmodates around 40 replicas. So could it be that l1 is the problem rather than s1 ?

Now the key question: what can i do to resolve this issue? I mean the error still indicates that k8s is trying to evicts pod instances that are not present anywhere, so how can this be cured? Obviously deleting a non-existant pod will fail at all times, which would make this cluster worthless since i cannot add/change anything in the cluster.
Appreciate any help!
I0820 10:11:46.542934       1 trace.go:205] Trace[661332898]: "Create" url:/api/v1/namespaces/longhorn-system/pods/instance-manager-e-689a397e/eviction,user-agent:Go-http-client/2.0,audit-id:1f7792bc-9bbb-4de2-a296-af8cb69c097b,client:<IP_ADDRESS>,accept:application/json, */*,protocol:HTTP/2.0 (20-Aug-2022 10:11:45.826) (total time: 716ms):
Trace[661332898]: [716.081416ms] [716.081416ms] END
I0820 10:11:46.610184       1 trace.go:205] Trace[1557110471]: "Call conversion webhook" custom-resource-definition:volumes.longhorn.io,desired-api-version:longhorn.io/v1beta1,object-count:1,UID:b464ea82-f9ac-49a3-af63-0c678717aeb5 (20-Aug-2022 10:11:46.432) (total time: 177ms):
Trace[1557110471]: ---"Request completed" 176ms (10:11:46.609)
Trace[1557110471]: [177.165426ms] [177.165426ms] END
W0820 10:11:46.629722       1 clientconn.go:1331] [core] grpc: addrConn.createTransport failed to connect to {<IP_ADDRESS>:2379 <IP_ADDRESS> <nil> 0 <nil>}. Err: connection error: desc = "transport: Error while dialing dial tcp <IP_ADDRESS>:2379: connect: connection refused". Reconnecting...
I0820 10:11:46.668615       1 trace.go:205] Trace[1106385201]: "Create" url:/api/v1/namespaces/longhorn-system/serviceaccounts/longhorn-service-account/token,user-agent:kubelet/v1.23.7 (linux/amd64) kubernetes/42c05a5,audit-id:0c58cdbd-d5d9-4665-95ca-435efc198eed,client:<IP_ADDRESS>,accept:application/vnd.kubernetes.protobuf,application/json,protocol:HTTP/2.0 (20-Aug-2022 10:11:46.136) (total time: 532ms):
Trace[1106385201]: ---"Object stored in database" 532ms (10:11:46.668)
Trace[1106385201]: [532.357363ms] [532.357363ms] END

meanwhile i drained node l1 entirely, no further volumes/replicas or pods running on that node. Yet the error message remains. Could that be that the CP just need a kick to check the status again?

Can you provide the support bundle?

What do you mean by that?

Please refer to https://www.suse.com/support/kb/doc/?id=000020145.
BTW, one workaround is to delete the PDB by manual. cc @PhanLe1010

the support bundle hast around 150MB which i cannot upload here. Any logs you are particularly interested? I'd certainly would delete the pdb manually if those would exist, but if i list all pdbs with
kubectl get pdb -n longhorn-system
i get this output

i do not find the pdbs in doubt anywhere, which are

i'd be happy to delete whatever neccessary to recover the cluster, but still seems to cluster is looking for something which doesn't exist anymore...

https://we.tl/t-0z3AYkuAir this is the support bundle @derekbit

https://we.tl/t-0z3AYkuAir this is the support bundle @derekbit

is there any idea on how to manually resolve this issue? This is severly impacting our cluster setup by now

Can you check the pdb by kubectl -n longhorn-system get pdb?
Then, delete the stuck ones.
cc @PhanLe1010 @shuo-wu

the stuck ones are not listed there...that is the root cause of the problem

https://we.tl/t-0z3AYkuAir this is the support bundle @derekbit

@shuo-wu @PhanLe1010 Any idea...?

But I found that instance-manager-e-90313f94 and instance-manager-r-206dec4f on node s1 are still running and they contain some running processes. Not sure if Longhorn can evict them correctly.

just ran the longhorn upgrade again without any improvement. The cluster remains in the same state, trying to evict those non existing pdb's . I can delete the pdbs on s1, but i don't think those will cure the cluster issue, since the problem are the pdbs that are not existing, yet still somehere the queue to be deleted.
Any chance to manually delete those evict inquiries manually to restore operational mode? I just can't find where those eviction instructions are stored in the cluster.
Also those this error message has not been updated since 4 days so i assume, that if we could trigger some kind of upgrade on the CP to check the state it might cure itself?

I was thinking if somehow Longhorn manager or something else keeps creating and deleting the PDB, which can lead to an illusion that the PDB does not exist at all. Based on this, the workaround may be:

Stop all engines or replicas running on the draining node
Delete or somehow stop the longhorn manager DaemonSet for a while
Try to delete the PDBs that blocks the node draining.
Wait for the drain (and the upgrade) complete
Resume the longhorn manager DaemonSet.

Notice that Stopping the longhorn manager won't lead to the current volume IO interruption. But other operations like snapshot or backup would be unavailable.
The workaround is based on guessing anyway. I cannot guarantee if it works or not.

we cannot delete the pdbs since those are not existent. The error message refers to pdbs that are not present on the node, otherwise it would be straight forward. My problem is that the pdbs do not exist anymore, therefore cannot be deleted, yet the CP is convinced to delete those pdbs nevertheless, which naturally fails. The CP is drained, there are no replicas no nothin running on it.

The logs [controlPlane] Failed to upgrade Control Plane: [[error draining node s1: [error when evicting pods/"instance-manager-e-689a397e" -n "longhorn-system": global timeout reached: 2m0s, error when evicting pods/"instance-manager-r-68277463" -n "longhorn-system": global timeout reached: 2m0s]]] is complaining that it cannot delete 2 pods. This error doesn't seems related to PDB because PDB blocking would have something like: cannot evict pod xxx because it violates PDB y
On the other hand, you are saying that you cannot see the pods using kubectl. This leads me to the theory about the possible inconsistency in etcd.
This cluster has 2 etcd nodes (s1, l1)  and 1 CP node (s1).  This a dangerous situation because if you lose l1, the etcd quorum will be permanently lost. And in deed, you are trying to delete l1 https://etcd.io/docs/v3.3/faq/#why-an-odd-number-of-cluster-members. https://etcd.io/docs/v3.3/faq/#why-an-odd-number-of-cluster-members
To verify this theory, can you increase the number of etcd node to 3?


Workflows GUI

This is the initial changes to allow (use?) the DefaultWorkflows in Qiita. To start the easiest is to reuse and create a GUI to visualize the current DefaultWorkflows. A few changes here:

We don't need to have command_id in DefaultWorkflowNode because we can get the command from the DefaultParameters
Adding an active flag to default_workflow
During the development of the GUI, I realized that we are missing 2 map files (so the GUI wasn't working as expected)

The new GUI looks like this:

This new version looks like this:

Thanks @antgonza


Change stonesoup url to application-pipeline

Fixes
Fixes HAC-3584
Description
Changes the stonesoup segment of the URL to application-pipeline. Redirects any URLs with stonesoup to the new URL.
Type of change

[x] Refactoring (no functional changes, no api changes)

/cc @rohitkrai03 @karthikjeeyar

/retest

/retest

/retest

@jeff-phillips-18 looks like the latest run failed in E2E:
11:55:30   1) Create Component from Private Git Using Login Form
11:55:30        Creating Component From Private Github Repo
11:55:30          Works After Signing in to Github:
11:55:30      AssertionError: Timed out retrying after 40000ms: Expected to find element: `[data-test="details__tabItem components"]`, but never found it.
11:55:30       at ./utils/Applications.ts.Applications.goToComponentsTab (webpack:///./utils/Applications.ts:113:7)
11:55:30       at Context.eval (webpack:///./tests/create-application-from-private-git-form.spec.ts:89:21)
11:55:30
11:55:30   2) Create Component from Private Git Using Login Form
11:55:30        Creating Component From Private Github Repo
11:55:30          "after all" hook for "Works After Signing in to Github":
11:55:30      TypeError: $el.css is not a function
11:55:30
11:55:30 Because this error occurred during a `after all` hook we are skipping the remaining tests in the current suite: `Create Component from Priva...`
11:55:30       at elHasDisplayNone (https://env-ephemeral-dovjpb-6mlz646c.apps.c-rh-c-eph.8p0c.p1.openshiftapps.com/__cypress/runner/cypress_runner.js:163133:14)
11:55:30       at Object.getReasonIsHidden (https://env-ephemeral-dovjpb-6mlz646c.apps.c-rh-c-eph.8p0c.p1.openshiftapps.com/__cypress/runner/cypress_runner.js:163380:7)
11:55:30       at Object.onError (https://env-ephemeral-dovjpb-6mlz646c.apps.c-rh-c-eph.8p0c.p1.openshiftapps.com/__cypress/runner/cypress_runner.js:131258:81)

/retest

@jeff-phillips-18 no redirect from stonesoup is happening.

@christianvogt Fixed, change got lost in the rebase.

/lgtm
/approve

/retest

/retest


Docs for --format option are missing in Readme

As above.
It is only mentioned slightly in the --verbose section.

Current release is 1.11.
For next minor release re already added it: https://github.com/FriendsOfPHP/PHP-CS-Fixer/blob/1.12/README.rst#usage
Thanks and have a good day !


can ng2-smart-table Export to Excel or PDF

as the title, can ng2-smart-table Export to Excel or PDF? if can, how?
thanks.

I recommend leveraging this library where you can trigger a download of CSV from well structured data object.   https://www.npmjs.com/package/downloadjs

It would be interesting if a method were implemented to export the data into PDF.

Hello, do we have any update if it's possible to download data from ng2-smart-table in excel format?

it's a shame that exporting to excel is not supported.


Filtering with DateTime not working properly

{
  "errors": [
    {
      "message": "Argument \"filters\" has invalid value
 {content: {op: LIKE, v: \"image\"}, datecreated: {op: GT, v: \"2020-05-01 00:00:00\"}}.\n
In field \"datecreated\": In field \"v\":
 Expected type \"DateTime\", found \"2020-05-01 00:00:00\".",
      "locations": [
        {
          "line": 4,
          "column": 11
        }
      ]
    }
  ]
}

we need to create special handling for datetimes since they will always be represented as ISO8601 strings on GraphQL side.


Should latest release be v3.3.1?

Not sure where v3.3.0 came from since it's not included in the repo, but v3.3.0 was the latest release version before the v3.2.1 from a few days ago according to npm:

Should the latest version be bumped to v3.3.1 to account for this?

Looks like v3.3.0 contains https://github.com/pmndrs/meshline/pull/15, and the version number was just bumped in the repo.

Good catch. Fixed.


Compilation crashes with CUDA_ERROR_MISALIGNED_ADDRESS: misaligned address error.

We're experiencing crashes when compiling certain TTGIR files with a CUDA_ERROR_MISALIGNED_ADDRESS: misaligned address error. This is what we retrieved from compute-sanitizer:
========= Invalid __shared__ write of size 8 bytes
=========     at 0x3130 in a_
=========     by thread (1,0,0) in block (0,0,0)
=========     Address 0x84 is misaligned
=========     Saved host backtrace up to driver entry point at kernel launch time
=========     Host Frame: [0x3050c2]
=========                in /usr/lib/x86_64-linux-gnu/nvidia/current/libcuda.so.1

We've tracked down the issue to this commit: https://github.com/openai/triton/commit/855ac8f02ed98c9ad5c4f4e270c7ef2062062d95 - the same TTGIR was working before, and is crashing since then.
Here's an example of a TTGIR dump that results in a crash (I'm working on a smaller reproducer):
misaligned_address_error.ttgir.zip
So far we investigated that the issue is likely coming from:
%71 = triton_gpu.convert_layout %70 {allocation.offset = 0 : i32} : (tensor<32x32xf32, #mma>) -> tensor<32x32xf32, #blocked>
FWIW, this was caught by internal tests, and we can so far only reproduce it on head when compiling from TTGIR (instead of TTIR). Our pipeline seems to generate slightly different TTGIR, so the issue may be that we're violating some assumptions of TTGIR, but it's curious that this worked before the commit above.

Good news is we've been able to reproduce this internally on Volta.

It also reproduces on an Ampere for me, as long as I use the Volta TTGIR (i.e. pass a ttgir file to triton.compile and then try to run it).

Probably fixed by https://github.com/openai/triton/pull/3079.  I'll close this, but feel free to reopen if it still reproduces.

Thanks for the quick response, that fixed the issue!


track_links redirecting to /undefined

I was trying to integrate the track_links functionality with a Wordpress site, but sometimes, after clicking in a tracked button/link, I was redirected to /undefined.
mixpanel.track_links(".class_name", "Event", {
    referrer: document.referrer
});

Somebody can explain if I'm doing something wrong?

Is it a real HTML link (<a> tag)? The code looks for the element's href property: https://github.com/mixpanel/mixpanel-js/blob/16e431855d3d4f6abfadd12506edf4f42e81500f/src/dom-trackers.js#L118

@jeduardo824 Did you figure this out? I'm running into the same issue.
@tdumitrescu I'm seeing real  tags in my implementation (I'm also using Wordpress):

any chance you could put together a minimal repro for us to investigate?

hey @tdumitrescu, after some debugging I think the problem is that I'm using Divi, a WP builder, and I wrongfully assumed that my ids in cases where I didn't hardcode them in myself were being placed on the a tag but in fact, they were being placed on parent divs. When I saw that the aforementioned (<a href="/process"...>) wasn't working, it was because I'd already come across a broken instance.
I'm good here. Thanks for responding!

In case this helps anyone else, I ran into this when I used track_links() on an element that isn't a real link (i.e., not an <a> tag). I was iterating over a list of objects, some of which were linking to other pages via onClick, and some which were standard <a> links. The solution was to detect which type of link I was working with, and then call track_links() on the real links and track() on the onClick links.


Changing branches (beta-versions)

Thanks for your valuable work!
I play heavily modded ETS2 with my friend, and thus, it takes a long time to update to the latest version. There is an official way to keep ETS2 at the specific version.
We can use this feature in SteamCMD by supplying -beta argument.

This is a nice way to handle that. I have not had to use older versions so I was not aware this was possible.
Thank you for your work.
Looks good.
I see you are cooking timezone setting support as well. Do you want to create a single PR with your timezones+branch branch or should I merge this one?

You could merge this one first


Refactor type names to improve Kapacitor documentation.

As discussed in #433 the type names used in the online Kapacitor documentation are unhelpful since the correspondence between Tick script keywords and type names is not as clear as it could be.
Since this documentation is auto-generated we really need to fix the type names in the source code.
Since this would be tedious and error prone if done manually, the refactoring is done automatically
with a bash script which:

applies one refactoring at a time with grep and sed
compiles the resulting code to check for compile errors
commits the result to git
finally, removes itself from the code base.

Once this is ready to be merged, the first commit in the series should be rebased onto the current origin/master and refactor-doc.sh executed which will produce a refactoring that exactly matches the current origin/master.
The other commits in this series are provided so that the refactorings the script is doing can be reviewed.
/cc @nathanielc

@jonseymour Thanks! The original names have drifted in meaning, I really like these new names simple and clear.
There are a few spots that should be changed to reflect this name change.

https://github.com/influxdata/kapacitor/blob/master/task_master.go#L509 change srcstream0 to stream0.
Two other places in the diff that I commented.

@jonseymour I would like to get this change in before too much changes and makes this PR invalid. Do you think you will have time to make a few small changes and rebase over the next few days?

closing in favor of #476

@nathanielc apologies for not getting back to you earlier about this change - I have deeply absorbed in some other tasks that reduced my time for working on these changes. Thanks for making the changes in #476

@jonseymour No problem, Kapacitor is cutting a release and I wanted these good changes to make it in, so I finished up. Thanks for getting it started!


Height of right frame elements unstable

Visible in Firefox and also Edge under Windows and Linux
The engine boxes switching horizonatl scroll bar on/off with changing length of the pv. That makes the whole frame "unstable".

Removed scrolling so this is no longer an issue.


Understand and document the steps to safely expand prometheus's disk size, and do so for Pangeo

Context
We sometimes need to increase the size of the prometheus disk to accommodate the amount of data we are generating. We should document the steps of how to do this safely so any engineer can perform the task without worrying about running into errors such as below and needing to debug them.
Error: UPGRADE FAILED: cannot patch "support-prometheus-server" with kind PersistentVolumeClaim: persistentvolumeclaims "support-prometheus-server" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize

When we have these steps, we should use them to merge PR #4808
Task list
- [ ] Figure out the steps needed to safely expand prometheus's disk size
- [ ] Document the steps
- [ ] Use the steps to merge #4808

Definition of Done

[ ] #4808 is merged and running as expected
[ ] There is documentation for expanding prometheus' disk size that any engineer can follow

Pre-defined Definition of Done

[ ] All the tasks have been completed
[ ] It has been tested to work and to not break any existing functionality
[ ] Documentation was added so other engineers can implement this in the future if needed

Thi sis dependent on #4810

The reason that the steps for expanding the persistentVolume size were not required for pangeo-hubs, is because that cluster is GCP, and all the storage classes there have autoexpansion set to true by default.


Make sure KMS key is decrypted based on Lambda encryption code examples

I've been trying to use the KMS encrypted ciphertext for webhook URL, generated via CLI and via Lambda encryption helpers, however none of them appear to work - when decrypting a "InvalidCiphertextException" is thrown.
Based on the Lambda encryption helper example code I've adjusted the decryption process which appears to have resolved the issue.
If this was indeed working as intended it would be great if @homeyjd could provide instructions on generating the encrypted string.

Sorry, this was a late-and-fast feature addition.  When I tested, I must have either not committed the code or had the wrong local branch.
Thank you for the fix!


Getting the client IP address

Is there any way to get the client IP address, maybe something similar to req.ip in NodeJS? In my case, there is no reverse-proxy involved, so there is nothing like x-forwarded-for in the header.

I think I actually just found it - req.socket:getpeername()

Yes, that would be the best way to get the peer name, it might be worth adding some kind of helper to the request type to make this a little more obvious


In your instructions for cgpt add of the new partition, the partition type should be specified

(co) # cgpt add -i 13 -b 4014080 -s 18874368 -l ARCH  /dev/sda -t data
I guess (haven't tried it out yet).
Otherwise, thanks for these instructions Guillaume. Very helpful.

I’m not sure if this is needed, I think this is the default. At least in this tutorial (that I’ve used a lot, I don’t know why I haven’t linked to it yet), they do not specify the partition type.
But of course, it will not harm if you do add the partition type :-)

Yes, it's needed when creating a new partition (-i 13 which didn't exist) on my chromebook, otherwise I got a message telling that "-s -b and -t should be specified when creating a new partition"

Ok, thanks, I will update it :-)


GIve a thumbnail to the tags

It would be possible to have a window with the tags and apply a thumbnail to each of them, so as to facilitate the search in the tags?

Where would a thumbnail come from? Would this be something a user has to manually add (drag & drop from their OS, for example)?

Where would a thumbnail come from? Would this be something a user has to manually add (drag & drop from their OS, for example)?

I think it would be useful if I could add a thumbnail taken from a user chosen image and manually add it to each tag.
For example, as in windows explorer, you can change the icon in a folder.
In our case to a "Video Hub App" tag.
Thank you for your answer.

I think it would be useful if I could add a thumbnail taken from a user chosen image and manually add it to each tag.
For example, as in windows explorer, you can change the icon in a folder.
In our case to a "Video Hub App" tag.
Thank you for your answer.

Thank you for the idea. It seems relatively straight-forward to have an alternative view of tags - with thumbnails inside each. There is enough supporting code in the app that should make adding this feature not very challenging.
What aspect ratio would you recommend for the tag thumbnails? A square seems natural. Or should it be the standard 16:9 ratio that all the thumbnails already have?
Cheers!

Thank you for the idea. It seems relatively straight-forward to have an alternative view of tags - with thumbnails inside each. There is enough supporting code in the app that should make adding this feature not very challenging.
What aspect ratio would you recommend for the tag thumbnails? A square seems natural. Or should it be the standard 16:9 ratio that all the thumbnails already have?
Cheers!

Well, I think keeping the 16: 9 standard is fair.  This way once you have chosen your thumbnails, it will be easier to navigate through multiple tags.  Regards

Well, I think keeping the 16: 9 standard is fair.  This way once you have chosen your thumbnails, it will be easier to navigate through multiple tags.  Regards


Update select.js

change reult==value to angular.equals(result,value)
it very important!
without that init objects that came with the model not match objects in option list

@brianfeister @dimirc @amcdnl this does fix issue #540 and probably a couple others when using 0.9.5.

Same as #410


Feature Request: Implement state_dict() / load_state_dict()

thanks for this code release, I wish more papers did this! It really makes it effortless to try out this new optimizer :)
It'd be great if YellowFin supported the state_dict() and load_state_dict() functions, to maintain a consistent serialization API with the other PyTorch optimizers.

I'm totally agree with @nelson-liu 's idea.
I think automatic(adaptive) lr and momentum scheduling is much more crucial in large-scale dataset(e.g. imagenet). The imagenet1K dataset needs at least a week to get best accuracy. So, It's very time consuming if we manually tune our parameters. The state_dict() and load_state_dict() operations can reduce our time and efforts dramatically.
Thanks in advance,

@taey16
Thanks for trying out the optimizer. We have already merged a pull request on this. Please try this feature out and let us know whether there are issues.


Add support for caffe-style ResNet and more flexible pretrained model loading

Add a config.MODEL.STYLE field in the config to support caffe-style ResNet (by setting config.MODEL.STYLE='caffe')
Support more flexible pretrained model loading

now support directly loading models trained with PyTorch ImageNet example
pretrained model file can be a dictionary with the key: state_dict
pretrained model can be directly saved by torch.nn.DataParallel (keys begined with module.)

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: bowenc0221 sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

@leoxiaobin

add caffe-style ResNet results in README
pretrain models are uploaded in Google Drive, you can download these models and put in your shared folder


Pull out seasonality as YearlyFourier and MonthlyFourier

Description

Related Issue

[x] Closes #679 closes #796
[ ] Related to #

Checklist

[x] Checked that the pre-commit linting/style checks pass
[x] Included tests that prove the fix is effective or that the new feature works
[x] Added necessary documentation (docstrings and/or example notebooks)
[x] If you are a pro: each commit corresponds to a relevant logical change

Modules affected

[x] MMM
[ ] CLV

Type of change

[x] New feature / enhancement
[ ] Bug fix
[ ] Documentation
[ ] Maintenance
[ ] Other (please specify):

📚 Documentation preview 📚: https://pymc-marketing--802.org.readthedocs.build/en/802/

Dont know what is up with the codecov

Great @wd60622 ! Before we merge this one, can you please double-check running the example notebook (no need to add it to the PR, just check) that we get the expected plots showing seasonal components

and

🙏

Great @wd60622 ! Before we merge this one, can you please double-check running the example notebook (no need to add it to the PR, just check) that we get the expected plots showing seasonal components

and

🙏

Yup. Reran and these outputs were the same. Notebook was like before

Fantastic! Thanks @wd60622 !


Return Jobs/Operations directly from the method that starts them

RE: https://github.com/GoogleCloudPlatform/gcloud-node/pull/1294#discussion_r62872247
Before:
vm.delete(function(err, operation, apiResponse) {
  operation
    .on('error', function(err) {})
    .on('running', function(operationMetadata) {})
    .on('complete', function(operationMetadata) {});
});

After:
vm.delete()
  .on('error', function(err) {})
  .on('running', function(apiResponse) {}) // is this what you had in mind, or is it the operationMetadata?
  .on('complete', function(operationMetadata) {});

@callmehiphop What should we do in these situations (regarding the second argument, vm):
compute.createVM('vm-name', function(err, vm, operation, apiResponse) {});

Emit vm with running?
compute.createVM('vm-name')
  .on('error', function(err) {})
  .on('running', function(vm, apiResponse) {})
  .on('complete', function(operationMetadata) {});

... or is that getting weird?

I think if the operation/job was responsible for creating something (like a VM) that the created VM would be emitted via complete.
I forgot about the previously existing running event, perhaps created (or similar) would be better suited.
compute.createVM('vm-name')
  .on('error', function(err) {})
  .on('created', function(apiResponse) {})
  .on('running', function(operationMetadata) {})
  .on('complete', function(vm, operationMetadata) {});

created and running feel a little too similar, maybe we should just not return the original apiResponse. If an error occurred with the createVM API call, then the apiResponse is already included there. The initial state of the operation shouldn't be vital... but it theoretically should be available from operation.metadata at some point:
var operation = compute.createVM('vm-name')
  .on('error', function(err) {}) // the "POST /vm-name" call failed OR the operation failed
  .on('running', function() {}) // arg removed. `operation.metadata` can be checked for the status
  .on('complete', function(vm) {}); // took out `operationMetadata`. they can check `operation.metadata` for the status

I agree, created was a bad choice, I think the point I'm trying to convey is that we could easily just emit an event specifically to capture the apiResponse. Some other possible options could be queued, operation-created, started, etc..

I know, I think it's just unnecessary. Please let me know what you think about the example in the last post. I've removed all metadata like arguments from the events themselves.

That could work, I'm not a huge fan of making the initial apiResponse completely inaccessible though.
Unrelated: Does this have any effect on autoCreate apis?

That could work, I'm not a huge fan of making the initial apiResponse completely inaccessible though.

If it's an error, apiResponse is accessible from the ApiError. If it succeeds, then the response is the initial Job/Operation metadata, which is available from the returned object:
var operation = zone.createVM('vm-name');
operation.metadata === apiResponse; // obviously not immediately, we emit `running` but no potentially stale "status" of the operation

Unrelated: Does this have any effect on autoCreate apis?

Hmm, good thought. Let's see...
var operation = vm.get({ autoCreate: true });

// the "GET /vm-name" call failed OR the "POST /vm-name" call failed OR the operation failed
operation.on('error', function(err) {});

// The VM is being created -- not emitted unless we have to create it
operation.on('running', function() {});

// The VM was either created or it already existed. Return the object here
operation.on('complete', function(vm) {});

Wdyt?

It does seem like we're stepping on the toes of the future support of Promises... should we wait to tackle this then?

Probably a good idea

@callmehiphop do you think we should support this in 1.0 or wait until after, at the cost of breaking the API and forcing a 2.0?

I like the idea of supporting it in 1.0 personally, we're already going to be breaking quite a bit with promises, so it might be a good idea to do it now and not make users have to refactor their code a second time when we do get around to it.

I agree that this is a pretty broad pattern, so I'd vote we keep this for 1.0 rather than 2.0.
I'm also OK with error, running, and complete. The Operation itself only has a boolean done state, so this seems reasonable.
To translate this to promises, we'd only handle error and complete (as then, and catch) right?

To translate this to promises, we'd only handle error and complete (as then, and catch) right?

Yep!

:+1:

I'm fine with doing this, as long as there's still a way for the user to get back the operation (and not wait for it to complete). I can see scenarios where one would want to start an operation in one place, and wait for it to complete elsewhere (or not wait for it to complete at all).

Very true. I think that gives us two options...

A new method (i.e., createWithOperation())
Return a decorated Promise with an event emitter (the Operation)

A new method (i.e., createWithOperation())

This could work, we'd need to come up with a consistent naming scheme.

Return a decorated Promise with an event emitter (the Operation)

I don't think this would work as the promise would still have the timer that's waiting for the operation to complete. The user needs a way to create the operation without google-cloud-node adding a listener for the complete event of the operation.
e.g.
// Current behavior
speech.startRecognition(...)
  .then((operation) => { ... });

// Behavior proposed by this PR
speech.startRecognition(...)
  .then((transcription) => { ... });

// Option 1: Create operation without waiting, via option
speech.startRecognition(..., { wait: false })
  .then((operation) => { ... });

// Option 2: Create operation without waiting, via different method
speech.startRecognitionNoWait(...)
  .then((operation) => { ... });

Given that req, I think one new method that returns an operation that doesn't start an interval is the cleanest and clearest separation.

Hmm, this is interesting. For the Speech API, the current method name is startRecognition. That name implies starting something, but not necessarily finishing. It's an apt name for a method that returns the operation, no timer started. So, a new method that calls startRecognition listens for the complete event under the hood?

Using the speech example... how about.
// I'd just like to do this the regular way...
speech.recognize(...)
  .then((transcript) => { ... });

// I don't want to wait. Give me the LRO please...
speech.recognize(...)
  .on('start', (operation) => { ... });

// I want the final result, but I also want the operation....
speech.recognize(...)
  .on('start', (operation) => { ... })
  .then((transcript) => { ... });

// The same thing written slightly differently...
speech.recognize(...)
  .on('start', (operation) => { ... })
  .on('success', (operation) => { ... });

I feel like "beginning the work" is an event tied to the technical implementation. Resolving the promise is about completing my "business-level goal" of getting back a transcript of the audio I just shipped up "to the cloud".
In other words... we return a promise (which is an event emitter that happens to spit out either 'success' or 'error'), but this promise also emits other events that are relevant to the underlying implementation details (in this case, "your work has started" via the 'start' event).

Source: https://gist.github.com/dmvaldman/12a7e46be6c3097aae31

@callmehiphop is this feasible to implement? I stumbled on this, which might help: https://github.com/59naga/carrack

I don't see how it's feasible to offer both behaviors without doing it via separate synchronous code paths. The user wants to pick one or the other—start the timer for me (convenience google-cloud-node wants to offer), or don't start the timer for me (how the AsyncRecognize method actually works). @jjg's examples start the timer every time*.
https://github.com/59naga/carrack won't help, as it solves a problem where the timer is always started. If the user wants the operation, then google-cloud-node cannot call .on('start', ...) under the hood at all. Maybe the user doesn't even want to call .on('start', ...), but just wanted to queue the Speech Recognize LRO. Always creating the timer is irresponsible with the user's CPU resources, and unnecessarily queues timers into the event loop, keeping a callstack around in a request/response cycle even if the user's code has already finished executing.
I also don't think we should tell the user to drop to a lower level of abstraction (gapic) to get to the API's default behavior (no timer). We should preserve existing behavior while also adding the new behavior, achieving it via different synchronous code paths (put it behind an option or a new method).
*The method has to know synchronously whether to start the timer or not (and when it should resolve the promise), it cannot wait for the user to call .then or .on on the method's return value to decide whether to start the timer. Hence the need for an option or new method—these are synchronous signals.

Personally I'm not huge into mixing interfaces like that, I would vote on offering two different methods. I also like the way we handle AsyncRecognize.
My naming could use some work but here's a simple example.
// user doesn't care when the op ends
compute.startCreateVM();

// user cares about the op
compute.startCreateVM()
  .on('error', console.error)
  .on('whatever', console.log)
  .on('complete', function(vm) {});

// convenience method that sits on top of op
compute.createVM = function() {
  return new Promise(function(resolve, reject) {
    compute
      .startCreateVM()
      .on('error', reject)
      .on('complete', resolve);
  });
};

// user just cares about when op is over
compute.createVM().then(function(vm) {});

@jjgeewax's examples start the timer every time

That doesn't sound right, but let me know if I'm mistaken:
Zone.prototype.createVM = function() {
  console.log('making the request...')
  return this.request(...)
  // ...
}

var operation = zone.createVM();
// => "making the request..."

So if the request can be made when someone calls zone.createVM(), we just need to know when events are registered, so we know what to do next (not sure if this is possible):
// Calling this method makes the API request to create an instance
var operation = zone.createVM();

// Returns the API response from the VM being created
// The operation hasn't started an interval
operation.on('response', function(apiResponse) {});

// Now the operation will start an interval
// This callback executes when the operation is complete
operation.then(function() {
  // VM created
});

Basically, if it's possible to distinguish between when a user registers an event handler on response vs a then handler, we should be able to make this work... maybe?
If not, separate methods are fine with me.

On trying to watch for .then: the promise creator is decoupled from the promise consumer(s)—it doesn't know anything about when or how the promise will be consumed. We can't have the promise creator trying to couple itself to the consumer. We'd have to re-implement Promise#then.
On trying to watch for .on: Similar argument as with promise, We'd have to change the implementation of EventEmitter#on in order to hook into it and couple the creator to the consumer. My insides hurt thinking about this.

// Calling this method makes the API request to create an instance
var operation = zone.createVM();

I don't see how Zone#createVM (or Speech#startRecognition) can be synchronous, as they're making an API call. We have to wait for the API to complete to get the operation id in order to even be able to do .on('complete', ...).
@callmehiphop's examples, slightly amended:
// user just wants to enqueue the operation
compute.startCreateVM();

// user cares about the operation
compute.startCreateVM().then((operation) => {
  console.log(operation.id); // id was provided by response from initial API call

  operation
    .on('error', (err) => {
      // error during operation, i.e. server error
    })
    .on('complete', (vm) => {});
}, (err) => {
  // failed to start operation, i.e. bad argument
});

// user just cares about the result of the operation
compute.createVM().then((vm) => { ... }, (err) => {
  // err could be from initial API call, e.g. failed to create operation
  // or err could have happened during the operation itself
});

I don't see how Zone#createVM (or Speech#startRecognition) can be synchronous, as they're making an API call.

It wouldn't be synchronous, but using another example, what happens with a method that doesn't use an operation:
Zone.prototype.getVMs = function() {
  return this.request(...);
}

var getVMsPromise = zone.getVMs();
// was a request made, even though I didn't register a `then`?

Regarding your first two points, I agree that offering dual-functionality from this single method doesn't sound great, but wanted to see if a solution is in fact available, given @jgeewax's original proposal. From a technical standpoint, it might not be as sickening as overriding .then and .on, if there is an underlying event emitter on a Promise, i.e.:
Zone.prototype.createVM = function() {
  var requestPromise = this.request(...)
  requestPromise.on('newListener', function (name) {
    if (name === 'then') // resolve with completed operation
    else if (name === 'response') // resolve with api response
  })
  return requestPromise
}

Just putting this out there in case you guys know if that's how it works. If not, I definitely don't want to override then, on, etc.

After some thinking, a more simple and obvious solution presented itself, implemented in #1689, which fits in nicely with the chained nature of promises. Feel free to re-open this issue if more discussion is needed.


fix: Fix FromStr not found error on AccountId20

This PR fixes the following issue.

When building fp-account without default features with the command:
cargo build -p fp-account --no-default-features --target wasm32-unknown-unknown

an error will occur because core::fmt::FromStr is not implemented for H160 without default features.
    Checking fp-account v1.0.0-dev (/Users/conr2d/Projects/frontier/primitives/account)
error[E0599]: no function or associated item named `from_str` found for struct `H160` in the current scope
  --> primitives/account/src/lib.rs:51:9
   |
51 |         H160::from_str(s)
   |               ^^^^^^^^ function or associated item not found in `H160`
   |

Normally, we don't compile the primitive crate using the command params that you pasted. Is there any specific reason behind this? I personally think this hack is not worth it.

I am writing a crate to extend Substrate features including fp-account, but this prevents using fp-account in a no_std environment. Is the fp-account crate intended to be used exclusively with Frontier? According to #1218, there are plans to publish Frontier crates to crates.io. In my opinion, a reusable component should build successfully without default features.
I added the --target wasm32-unknown-unknown flag because the underlying Substrate dependency requires a pointer size of 4 bytes in a no_std environment. The use of core::fmt::FromStr instead of std::fmt::FromStr suggests that this implementation is intended for no_std use, but it currently doesn’t support that case correctly. The implementation should either be changed to use std::fmt::FromStr with a feature guard, such as #[cfg(feature = "std")] or corrected to properly handle no_std environments like this PR.
Moreover, I don't believe my change qualifies as a hack since it doesn't introduce any new dependencies or use unsafe or non-idiomatic syntax. :)


Trace-X uses its own BPDM lookup to identity Customer data for BPNL

As user
I want that BPDM lookup is managed in trace-x
so that I receive detail information attributes from golden record for a bpnl (address, customer contact, etc )
Background
Currently, Trace-X uses BPDM lookup feature of irs to determine manufacturer name for BPNL. This limits the information attributes of Golden record to the manufacturer name.
This ineffective bpnl lookup should be removed from irs. Trace-X has to develop its own bpnl lookup in order to receive more detail information for a bpnl.
Hints / Details

...

Outcome / Acceptance Criteria

Outcome

...

Acceptance Criteria

[ ] Manufacture name lookup  for bpnl is made by trace-x app

Out of Scope

...

@jzbmw Idea is to descope bpn lookup feature from irs. trace-x should provide its own implementation.


Added critical sections around management of the BackupFiles list

Ensure that BackupFile collection is fully populated prior to being read and is not modified while it is being read.

Fixes #8

Closes #8


Fix opam dev-repo URL typo

Signed-off-by: Ian Campbell<EMAIL_ADDRESS>

LGTM (but not a maintainer here), thanks!

LGTM


Password in clear text if PASSWORD_HASH=auto and no password exists

The option PASSWORD_HASH=auto stores the password in clear text if no password is set before, because there is no Hash mechanism used if no password exists.
I think it would be a better way to use a standard Hash mechanism like SHA instead of using clear text. So the password is hashed, even if no password exists before.

This is an issue with upstream, recommend you change PASSWORD_HASH manually from Auto to something of your liking.

I changed it.
But I think for better security there should be no way, that the password is stored in plain text if it is not explicitly requested. Could be a security issue for someone who don't think about this behavior and PASSWORD_HASH is set to auto.

I'll add a warning in upcoming release when someone has the var set to auto.


Print command line options if run_command() failed.

It helps to reproduce failed command execution during training.

@eighttails, please consider using tesstrain.py (part of the tesstrain repository) instead of the old tesstrain.sh.
@Shreeshrii, do you think that we can remove the old shell scripts for training?

do you think that we can remove the old shell scripts for training?

Since 5.0 is quite different from 4.0.x, I think it is fine to remove them.
I suggest you do it along with a new tag/release so that it is easy to identify.
It will also require changes in documentation.  We should probably move Ray's tesstutorial to be as applicable for Tesseract 4.0.x and have new instructions for training/tutorial for 5.0.

I have made some changes in tessdoc, moved training related info to tess4 subdirectory. Will Add new pages for tess5.