[Snyk] Upgrade dotenv from 16.3.1 to 16.3.2

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to upgrade dotenv from 16.3.1 to 16.3.2.
:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

The recommended version is 1 version ahead of your current version.
The recommended version was released 21 days ago, on 2024-01-19.

Release notes

Package name: dotenv

16.3.2 - 2024-01-1916.3.2

16.3.1 - 2023-06-1716.3.1

    from dotenv GitHub release notes

Commit messages

Package name: dotenv

883d6a6 16.3.2
0489c7d Merge pull request #793 from zshnb/fix-buffer-slice
8609b84 update CHANGELOG
54d54e9 update CHANGELOG
630d2a0 Merge pull request #735 from anthonyshibitov/encoding-fix
823d0b8 Merge branch 'master' of github.com:motdotla/dotenv
2cffe84 update CHANGELOG
27e4c2b Merge pull request #792 from phuvinhbmt/fix/populate-return-type
1548c27 Update README
2eed66b Update README
6979f94 fix: use subarray instead of slice
c5b7592 refactor: remove redundant params
060db4f refactor doc of populate() signature
5d00dd9 Merge branch 'master' of github.com:motdotla/dotenv
01000f0 Remove retool
fc60b1b Merge pull request #779 from AnirudhKaranth/master
fab349b Add note in README
677b113 Change order of steps
101e1a7 Updated a link in README.md
cf4c569 Adjust example links to /usage
5924934 Merge pull request #762 from hoangdesu/patch-2
c8085ae Fix link to 12factor methodology
aa03dca Fix broken link
fa3cb05 Remove redundant key

Compare

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:
🧐 View latest project report
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (8ab34cc) 77.83% compared to head (97dcc87) 77.83%.

:exclamation: Current head 97dcc87 differs from pull request most recent head ac58c7f. Consider uploading reports for the commit ac58c7f to get more accurate results

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2096   +/-   ##
=======================================
  Coverage   77.83%   77.83%
=======================================
  Files          40       40
  Lines        3090     3090
  Branches      629      629
=======================================
  Hits         2405     2405
  Misses        479      479
  Partials      206      206

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Bug: concurrent_index complaining when creating/dropping an index on a partitioned table in PostgreSQL

I got the cop RuboCop(Sequel/ConcurrentIndex) complaining when I'm trying to run a migration to create/drop an index on a partitioned table in PostgreSQL when concurrently: true is not set, while it is currently not possible in PostgreSQL.
Steps to reproduce

Create a migration that adds an index to a partitioned table
Don't add concurrently: true
Check that the cop RuboCop(Sequel/ConcurrentIndex) is complaining
Add concurrently: true
Run the migration
Get an exception

Sequel::DatabaseError: PG::FeatureNotSupported: ERROR:  cannot create index on partitioned table "my_table" concurrently

ROM::SQL.migration do
  no_transaction

  change do
    alter_table :my_table do
      add_index :my_index, concurrently: true
    end
  end
end

Expected behavior

The cop should not complain, as the proposed change is not working for partitioned table in PostgreSQL
Actual behavior

The cop is complaining, forcing me to change to a non-working solution where a Sequel::DatabaseError: PG::FeatureNotSupported is raised.
Proposed change
If this situation is still happening with up-to-date gems and ruby versions, I believe we could go in 2 directions:

Remove the cop: fast and simple, but might decrease experience for people that don't fact this issue.
Make the cop less restrictive or more clear: we could enforce to specify concurrently, even if it is set to false, and change the message from 'Prefer creating or dropping new index concurrently' to 'Specify concurrently option when creating or dropping an index.
I'm of course open for other solutions!

I'm more in favor of the second option of keeping the cop but forcing it to say we should specify the concurrently option, instead of setting it to true, but I'm open for suggestions!
I've opened a PR for this if it makes sense.
System configuration
rubocop-sequel version:
0.3.4
sequel version:
5.76.0
pg version:
1.5.4
Ruby version:
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux]

You could also disable the cop locally using comments, see: https://docs.rubocop.org/rubocop/1.62/configuration.html#disabling-cops-within-source-code

You could also disable the cop locally using comments, see: https://docs.rubocop.org/rubocop/1.62/configuration.html#disabling-cops-within-source-code

Yes, that was also a possibility! Thanks for pointing that out!
I think updating the message is still a good idea, as it is more explicit about what's expected and doesn't enforce "wrong" implementation in some situation (like the one in this PR).


Fix up --clean for MySQL

Added a function to drop guests as there was an issue in MySQL where we couldn't delete the samples table due to a FK constraint with tasks, which had a FK contraint to the guest table. We also drop the machines-related tables; should probably drop errors and tags tables in a future commit to be truly clean. We were also missing commits to the drop statements for the samples and tasks tables. These errors were invisible due to the missing log handler, which I have logged under issue #453.

Thank you very much for your contribution, although I tried a more sqlalchemyish fix (db46352). Let me now if it's ok with mysql (postgres is fine).

Works in my env with MySQL -- thanks!


Fixes Ruby 2.6 BigDecimal deprecation

We recently added exonio to a Ruby 2.6 project, and we're getting hundreds of console line spam messages about the BigDecimal depreciation. This affected a lot of gems, and I think is an easy change.
/Users/bbugh/.rvm/gems/ruby-2.6.2/gems/exonio-0.5.3/lib/exonio/helpers/irr_helper.rb:14: warning: BigDecimal.new is deprecated; use BigDecimal() method instead.
/Users/bbugh/.rvm/gems/ruby-2.6.2/gems/exonio-0.5.3/lib/exonio/helpers/irr_helper.rb:14: warning: BigDecimal.new is deprecated; use BigDecimal() method instead.
/Users/bbugh/.rvm/gems/ruby-2.6.2/gems/exonio-0.5.3/lib/exonio/helpers/irr_helper.rb:14: warning: BigDecimal.new is deprecated; use BigDecimal() method instead.
/Users/bbugh/.rvm/gems/ruby-2.6.2/gems/exonio-0.5.3/lib/exonio/helpers/irr_helper.rb:14: warning: BigDecimal.new is deprecated; use BigDecimal() method instead.
/Users/bbugh/.rvm/gems/ruby-2.6.2/gems/exonio-0.5.3/lib/exonio/helpers/irr_helper.rb:14: warning: BigDecimal.new is deprecated; use BigDecimal() method instead.
/Users/bbugh/.rvm/gems/ruby-2.6.2/gems/exonio-0.5.3/lib/exonio/helpers/irr_helper.rb:14: warning: BigDecimal.new is deprecated; use BigDecimal() method instead.
/Users/bbugh/.rvm/gems/ruby-2.6.2/gems/exonio-0.5.3/lib/exonio/helpers/irr_helper.rb:14: warning: BigDecimal.new is deprecated; use BigDecimal() method instead.

I see that Travis is set up, but it doesn't appear to be running.

@rizidoro anything I can do to help get this merged? It's a very minimal change that is backwards compatible. Thanks!

I would like to see this merged as well.
PS: Good to see a familiar face @bbugh 👋

Hey @bbugh , thanks for fixing it =)
@PauloHerrera could you review and merge it?

@rizidoro what's the status of this ?

Published at https://rubygems.org/gems/exonio/versions/0.5.4
Thank =)

Amazing! Wishes really can come true. Thank you both for shipping this!


TickTick hangs when initialized in an interactive shell

Initializing ticktick in an interactive bash shell causes the shell to hang
# Start a new bash shell without any extra environment variables
env - bash
# Try to initialize ticktick
. ticktick.sh

The source command doesn't complete.  It just hangs.

@kristopolous wrote:

because including ticktick basically hijacks the shell, makes a copy of your script in memory, rewrites it into valid bash, executes the rewritten in-memory version of your program and then exits.

Ahem in memory, you say?
I think it only does it in memory for the main script that sourced ticktick.sh.
Anything that script subsequently sources, seems to get written to a file in /tmp, with a name like /tmp/ymrjibble.skibble (though usually without the rhyming). Whether or not TickTick has to make any changes to that file.
At least, that's my understanding from the increasing number of files hanging out under /tmp, and the periodic errors like Missing ] on line 52 of /tmp/blahbloh.blih.
That said, nice post, and brilliant project!


pykafka raises an exception about BalancedConsumer

when I use pykafka, it raise an exception which I don't know the problem, anyone know?:
No handlers could be found for logger "pykafka.simpleconsumer"
Process Process-12:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "pyConsumerKafka.py", line 96, in consumer
    for message in kafka_consumer:
  File "/home/idm/.local/lib/python2.7/site-packages/pykafka/balancedconsumer.py", line 742, in __iter__
    message = self.consume(block=True)
  File "/home/idm/.local/lib/python2.7/site-packages/pykafka/balancedconsumer.py", line 728, in consume
    message = self._consumer.consume(block=block)
  File "/home/idm/.local/lib/python2.7/site-packages/pykafka/simpleconsumer.py", line 415, in consume
    self._raise_worker_exceptions()
  File "/home/idm/.local/lib/python2.7/site-packages/pykafka/simpleconsumer.py", line 227, in _raise_worker_exceptions
    raise ex
KeyError: 20
Exception KeyError: (19,) in <bound method BalancedConsumer.__del__ of <pykafka.balancedconsumer.BalancedConsumer at 0x16d4e10 (consumer_group=kafkarec)>> ignored

and the codes is:
def consumer(queueName):

    global pid
    pid = os.getpid()
    LOGGERS[pid] = logging.getLogger('MyLogger' + str(pid))
    LOGGERS[pid].setLevel(logging.DEBUG)
    LOG_PATH = 'log/hh.redis.out' + str(pid)
    HANDLER = logging.handlers.TimedRotatingFileHandler(LOG_PATH,'midnight',1)
    formatter = logging.Formatter('%(asctime)s - %(message)s')
    HANDLER.setFormatter(formatter)
    LOGGERS[pid].addHandler(HANDLER)

    client = KafkaClient(hosts = kafka_hosts)
    topic = client.topics[kafka_topics]

    kafka_consumer = topic.get_balanced_consumer(
        consumer_group = kafka_group,
        auto_commit_enable = True,
        auto_offset_reset = OffsetType.LATEST,
        ##-+ reset_offset_on_start = True
        ## | reset_offset_on_start = False
        #auto_commit_interval_ms = 1000,
        #reset_offset_on_start = False,
        zookeeper_connect = kafka_zk_connect
    )
    for message in kafka_consumer:
        if message is not None:
            callback(message.value)

def callback(value):

    global pid
    try:
        lineInfo = json.loads(value.strip())
        eventId = lineInfo['eventId'] if 'eventId' in lineInfo else ''
        account = lineInfo['userId']
        nuId = lineInfo['sessionUuid']
        deviceId = lineInfo['deviceUdid']
        actionTime = int(lineInfo['occurTime'])
        appVersion = lineInfo['appVersion']
        category = lineInfo['category']
        if account in testUsers: return
        if category == 'pageJump':
            LOGGERS[pid].info("[%s,%s,%s]\t%s\t%s\t%s\t%s\t%s" % \
                (account, deviceId, nuId, appVersion, eventId, category, actionTime, timeStamp2Date(actionTime)))
    except Exception as e:
        LOGGERS[pid].info("Exception with %s in log:%s" % (e, value))

if __name__ == '__main__':

    while(1):
        try:
            print 'start python worker ', datetime.datetime.now()
            processList = [ Process(target=consumer, args=("kafka",)) for i in xrange(PROCESS_NUM)]
            for i in xrange(PROCESS_NUM):
                processList[i].start()
            for i in xrange(PROCESS_NUM):
               processList[i].join()
            #consumer("kafka")
        except Exception as e:
            print 'python workder error occur: ', e
            error = 'kafka error: %s' % str(e)
            alert_sms(error)
            time.sleep(3)
    print 'end python worker ', datetime.datetime.now()

@armavrdsp Thanks for filing this. It looks like a real error, but it's hard to see exactly where it's originating. In your log, are there any additional tracebacks prior to the one you posted? This one is the  wrapper exception that the main thread raises to indicate a failure on a worker thread. If you can find the original traceback from the worker itself (which should immediately precede yours in the log), we will have a much better chance of fixing this quickly. Thanks!

I tried to reproduce this error and yeah it throws an exception .
File "/home/.local/lib/python2.7/site-packages/pykafka/balancedconsumer.py", line 728, in consume
    message = self._consumer.consume(block=block)
  File "/home/.local/lib/python2.7/site-packages/pykafka/simpleconsumer.py", line 415, in consume
    self._raise_worker_exceptions()
  File "/home/local/lib/python2.7/site-packages/pykafka/simpleconsumer.py", line 227, in _raise_worker_exceptions
    raise ex
KeyError: 20
Exception KeyError: (19,) in <bound method BalancedConsumer.__del__ of <pykafka.balancedconsumer.BalancedConsumer at 0x16d4e10 (consumer_group=kafkarec)>> ignored

:smile_cat:

@arpit1997 Looks like that traceback is the same as the original - notice the self._raise_worker_exceptions call indicating that this is the wrapper exception, not the root one. When you reproduce the issue, you should see another traceback before this one. If I can see that root exception (the one not containing self._raise_worker_exceptions), I may be able to give more information.

Does this have anything to do with an exception that looks like this?
Exception RuntimeError: RuntimeError('cannot join current thread',) in <bound method BalancedConsumer.__del__ of <pykafka.balancedconsumer.BalancedConsumer at 0x2316290 (consumer_group=group one)>> ignored

It is happening in about the same situation, using balanced consumers in a single program (integration test file), but I can't find any other issues/pr's as close as this one to the problem.

@dskoda1 It looks to me like what you're seeing is another side effect of the root cause - we don't really gain much information by knowing that the thread crashed if we don't know what caused it to crash. Thanks for the help solving this one!

@emmett9001 Some more background on the exception I am getting:
Instead of creating actual processes like OP is doing, in a test case I am creating two instances of balanced consumer in the main process, and they both belong to the same group (important to note, because test cases where I have two consumers in different groups, this exception is not being raised).
Also since it is in the __del__ method of the BalancedConsumer:
<bound method BalancedConsumer.__del__ of <pykafka.balancedconsumer.BalancedConsumer at 0x2316290 (consumer_group=group one)>>
I was able to avoid the exception by explicitly calling the __del__ method on the BalancedConsumer, which makes it seem to me like some threading resources potentially are not being handled correctly.

@arpit1997 It looks like you may be getting error code 20 (NOT_ENOUGH_REPLICAS_AFTER_APPEND) from the server. What does your topic's replication look like, and do you think this might indeed be happening?

@armavrdsp @dskoda1 @arpit1997 I recommend changing your consumer instantiation to with topic.get_balanced_consumer(...) as kafka_consumer: and performing message consumption inside of that with block. This will ensure that the consumers internal threading resources are cleaned up properly when your child process completes, and I'd expect to see these errors go away.

Hey @emmett9001 :  I am trying to write python code for calculating Kafka Consumer Lag, after executing the code, I am getting no handlers could be found for logger "pykafka.simpleconsumer".
Here is my code:
import pykafka
def fetch_consumer_lag(topic, consumer_group):
"""Get raw lag data for a topic/consumer group.
:param client: KafkaClient connected to the cluster.
:type client:  :class:pykafka.KafkaClient
:param topic:  Name of the topic.
:type topic:  :class:pykafka.topic.Topic
:param consumer_group: Name of the consumer group to fetch lag for.
:type consumer_groups: :class:str
:returns: dict of {partition_id: (latest_offset, consumer_offset)}
"""
latest_offsets = fetch_offsets(topic, 'latest')
consumer = topic.get_simple_consumer(consumer_group=consumer_group,
auto_offset_reset=OffsetType.LATEST,
auto_commit_enable=False,
auto_commit_interval_ms=1000)
consumer = topic.get_simple_consumer(consumer_group=consumer_group,
auto_start=False,
auto_offset_reset=OffsetType.LATEST,
consumer_timeout_ms=5000)
current_offsets = consumer.fetch_offsets()
def kafkaLagCalculator(client):
"""Calculate lag for a topic/consumer group.
:param client: KafkaClient connected to the cluster.
:type client:  :class:pykafka.KafkaClient
"""
topic_list = get_all_topics(client)
consumer_group_topics, consumer_groups = get_consumer_group_for_topic(client, topic_list)
Don't auto-create topics.
for consumer_group_topic in consumer_group_topics:
if consumer_group_topic not in client.topics:
raise ValueError('Topic {} does not exist.'.format(consumer_group_topic))
topic = client.topics[consumer_group_topic]
threads = [None] * len(consumer_groups)
i = 0
try:
    for consumer_group in consumer_groups:
        threads[i] = Thread(target=fetch_consumer_lag, args=(topic, consumer_group))
        threads[i].start()
        i += 1

    for i in range(len(threads)):
        threads[i].join()
except Exception as e:
    print "ERROR: ", e.message

Any suggestion would be appreciated.

@rohitpandey5491 It's not clear from your post what error you're getting. It's easier to pinpoint the issue if you post the minimum amount of code you can use to replicate it.


fix polls demo run application

I'm adding a bug fix, but there's no branch for current version of aiohttp - 1.2. Set master as a target branch.
What do these changes do?
Fixed run demo polls application:
(polls) MacBook-Pro:polls e.iskandarov$ python -m aiohttpdemo_polls
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/e.iskandarov/sandbox/aiohttp/demos/polls/aiohttpdemo_polls/__main__.py", line 3, in <module>
    main()
TypeError: main() missing 1 required positional argument: 'argv'

Checklist

[√] I think the code is well written
[-] Unit tests for the changes exist
[-] Documentation reflects the changes
[√] Add yourself to CONTRIBUTORS.txt

The format is <Name> <Surname>.
Please keep alphabetical order, the file is sorted by names.

[√] Add a new entry to CHANGES.rst

Current coverage is 98.85% (diff: 100%)

Merging #1487 into master will not change coverage

@@             master      #1487   diff @@
==========================================
  Files            30         30
  Lines          6984       6984
  Methods           0          0
  Messages          0          0
  Branches       1163       1163
==========================================
  Hits           6904       6904
  Misses           40         40
  Partials         40         40

Powered by Codecov. Last update 4eef48a...34c2d18

Cool! Thank you.
P.S.
I've created missing 1.2 branch


Fix a s/as/has/ typo in basic-auth-with-2fa

Ref #10836

(noticed this during translation)

Also needs done to https://github.com/pypa/warehouse/blob/main/warehouse/templates/email/basic-auth-with-2fa/body.txt

@di done!


[Feature] Smart speed selection

Hi,
I thought of an interesting feature: a kind of "intelligent" management of game speed.
Overall, each player could choose the speed at which they want to go, and the game will go at the speed closest to everyone's choices.
For example:
Player 1 chooses speed 3 (max)
Player 2 chooses speed 2
Player 3 chooses speed 2
[example added for more clarity]
Then the game goes to speed 2.
Player 1 chooses speed 2
Player 2 chooses speed 2
Player 3 chooses speed 2
Player 4 chooses speed 1
Then the game goes to speed 1.
I think it would make games with many players more enjoyable, and it would prevent accidents during the fights of some players!
If I can participate in the development of this feature, I would be happy to do so!
Thank you in advance for reading, sorry for the lack of formalism

Hey !
Good idea, but it may be better if the game choose the lowest selected game speed.
For exemple :
Player 1 : Speed x3
Player 2 : Speed x3
Player 3 : Speed x1
Then the game goes to x1
With your method, if 7 players choose x3 and the eighth want x1, then it will still go x3 (if I understood correctly).

Hey !
Good idea, but it may be better if the game choose the lowest selected game speed.
For exemple :
Player 1 : Speed x3
Player 2 : Speed x3
Player 3 : Speed x1
Then the game goes to x1
With your method, if 7 players choose x3 and the eighth want x1, then it will still go x3 (if I understood correctly).

I might have explained it badly, because we're thinking of the same system!
The lowest set speed will always be the server's speed!
That's the best I could think about to protect players being attacked while allowing easier speed management

I was going to suggest "lowest speed" feature too, until i saw this discussion.
Looks like a lot of people want this!

Sadly I haven't seen much feedback from the devs, and the number of issues is starting to grow! Looks like the devs are busy


Throw exception when circular template reference

angular.module('abcApp')
        .when('/inventory', {
//circular reference
          templateUrl: 'js/inventory/inventoryDirective.html'
        })

(function () {
  'use strict';

  function inventoryDirective() {
    return {
      restrict: 'EA',
      scope: {},
      controller: 'inventoryController',
      controllerAs: 'vm',
//circular reference
      templateUrl: 'js/inventory/inventoryDirective.html'
    };
  }

Not sure how we could go about this, technically speaking, nor if it is worth the overhead. But you should be getting already an error from a browser, no?
For now I'm leaning towards "won't fix" - while I'm all for having great error messages we need to keep in mind that each error message has cost (bytes added to the framework + CPU). So we need to make sure that we are creating overhead for rare cases (and I believe this one isn't very common)

Agree with @pkozlowski-opensource. The fact that templateUrl can be a function and can change the value it returns does not make it any easier.
Closing this as wont fix

@pkozlowski-opensource
Chrome was stuck, I have to kill that tab


Default value of attributesForFaceting

The default value of attributesForFaceting is currently null but should be [] because it expects an array of attributes.
(On master)

closed by #747


Hiding stacked windows for transparency

Hi, I recently set my terminal's background to be semi-transparent, because I like the aesthetic of being able to see my desktop background.
However, when I stack, fullscreen, or zoom-parent, I run into the issue that stacked windows in the back are not hidden, simply stacked over, leaving them partially visible, like so:

Is there a way to do have stacked-over windows hidden to prevent this?

Just write a chained command for changing stacks... where active_window_opacity=0 is before the actual stack window change, and active_window_opacity=1 after it.

@Bellavene this won't fix zoom-fullscreen or zoom-parent, which are the main ones I actually care about.

For those, you can write a script.

@Bellavene do you have an example you can link me please? I'm not really sure what that would look like.

Sorry, but I have things to do, try to search here about scripts with jq parsing. With the help of jq, you can pick states for a window, including  zoom-fullscreen and zoom-parent


AttributeError: 'int' object has no attribute 'dtype'

[ ] Check that you are up-to-date with the master branch of keras-vis. You can update with:
pip install git+git://github.com/raghakot/keras-vis.git --upgrade --no-deps

[ ] If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found here.

[ ] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps

[ ] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

Hello, I am trying to run through the Attention on MNIST (Saliency and grad-CAM) example and I am running into the following issue.

AttributeError                            Traceback (most recent call last)
 in
11 model = utils.apply_modifications(model)
12
---> 13 grads = visualize_saliency(model, layer_idx, filter_indices=class_idx, seed_input=x_test[idx])
14 # Plot with 'jet' colormap to visualize as a heatmap.
15 plt.imshow(grads, cmap='jet')
~/jessica/anaconda3/envs/tf_gpu/lib/python3.7/site-packages/vis/visualization/saliency.py in visualize_saliency(model, layer_idx, filter_indices, seed_input, wrt_tensor, backprop_modifier, grad_modifier, keepdims)
132         (ActivationMaximization(model.layers[layer_idx], filter_indices), -1)
133     ]
--> 134     return visualize_saliency_with_losses(model.input, losses, seed_input, wrt_tensor, grad_modifier, keepdims)
135
136
~/jessica/anaconda3/envs/tf_gpu/lib/python3.7/site-packages/vis/visualization/saliency.py in visualize_saliency_with_losses(input_tensor, losses, seed_input, wrt_tensor, grad_modifier, keepdims)
77     """
78     opt = Optimizer(input_tensor, losses, wrt_tensor=wrt_tensor, norm_grads=False)
---> 79     grads = opt.minimize(seed_input=seed_input, max_iter=1, grad_modifier=grad_modifier, verbose=False)[1]
80
81     if not keepdims:
~/jessica/anaconda3/envs/tf_gpu/lib/python3.7/site-packages/vis/optimizer.py in minimize(self, seed_input, max_iter, input_modifiers, grad_modifier, callbacks, verbose)
151
152             # 0 learning phase for 'test'
--> 153             computed_values = self.compute_fn([seed_input, 0])
154             losses = computed_values[:len(self.loss_names)]
155             named_losses = list(zip(self.loss_names, losses))
~/.local/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py in call(self, inputs)
2713                 return self._legacy_call(inputs)
2714
-> 2715             return self._call(inputs)
2716         else:
2717             if py_any(is_tensor(x) for x in inputs):
~/.local/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py in _call(self, inputs)
2653                 array_vals.append(
2654                     np.asarray(value,
-> 2655                                dtype=tf.as_dtype(tensor.dtype).as_numpy_dtype))
2656         if self.feed_dict:
2657             for key in sorted(self.feed_dict.keys()):
AttributeError: 'int' object has no attribute 'dtype'

I've run the example notebook, but could NOT reproduce the issue.
Did you modify the example notebook ? If not, there may be the environmental issue.
Please let me know the result of pip freeze and keras-vis version(or git-commit-id) of example notebook you use.


Fix db error reporting

This is an apparent fix of the bug mentioned here. This bug obfuscates any error response stored by self._async_table.get_artifact_in_task(...) in DBResponse.body.
Basically, when there is a generic DB error, self._async_table.get_artifact_in_task(...) returns a DBResponse with .body attribute of type string (containing the error message) and a .response_code of 500, indicating an error. The string-type body is set in db_utils.aiopg_exception_handling.
ArtifactsApi._filter_artifacts_by_attempt_id(...) expects artifacts.body to be an iterable, which is what get_artifact_in_task returns when no error has occurred (and fetch_single==False, which is the case in get_artifact_in_task). This fix avoids the call to ArtifactsApi._filter_artifacts_by_attempt_id(...) when DBResponse.response_code is not 200.

Here is the full comment on gitter for thoroughness:

I'm trying to take a look at the artifacts associated with one of our SFN executions but when trying to call:
Step(...).task.data

We run into the following error:
ServiceException: Metadata request (/flows/{Flow}/runs/{Run}/steps/start/tasks/{Task}/artifacts) failed (code 500): 500 Internal Server Error

Looking at the logs from our metadata service, I see the following error:
Traceback (most recent call last):
    File "/opt/latest/lib/python3.7/site-packages/aiohttp/web_protocol.py", line 418, in start
        resp = await task
    File "/opt/latest/lib/python3.7/site-packages/aiohttp/web_app.py", line 458, in _handle
        resp = await handler(request)
    File "/opt/latest/lib/python3.7/site-packages/services/metadata_service/api/artifact.py", line 140, in get_artifacts_by_task
        artifacts.body)
    File "/opt/latest/lib/python3.7/site-packages/services/metadata_service/api/artifact.py", line 355, in _filter_artifacts_by_attempt_id
        attempt_id = ArtificatsApi._get_latest_attempt_id(artifacts)
    File "/opt/latest/lib/python3.7/site-packages/services/metadata_service/api/artifact.py", line 349, in _get_latest_attempt_id
        if artifact['attempt_id'] > attempt_id:
TypeError: string indices must be integers

From what I can tell, this can only occur when the result of this call returns a DBResponse object with a body of type string. And it looks like this would only happen in the case of error handling. Unfortunately it looks like the underlying error message is obfuscated by this error.

Here is the full comment on gitter for thoroughness:

I'm trying to take a look at the artifacts associated with one of our SFN executions but when trying to call:
Step(...).task.data

We run into the following error:
ServiceException: Metadata request (/flows/{Flow}/runs/{Run}/steps/start/tasks/{Task}/artifacts) failed (code 500): 500 Internal Server Error

Looking at the logs from our metadata service, I see the following error:
Traceback (most recent call last):
    File "/opt/latest/lib/python3.7/site-packages/aiohttp/web_protocol.py", line 418, in start
        resp = await task
    File "/opt/latest/lib/python3.7/site-packages/aiohttp/web_app.py", line 458, in _handle
        resp = await handler(request)
    File "/opt/latest/lib/python3.7/site-packages/services/metadata_service/api/artifact.py", line 140, in get_artifacts_by_task
        artifacts.body)
    File "/opt/latest/lib/python3.7/site-packages/services/metadata_service/api/artifact.py", line 355, in _filter_artifacts_by_attempt_id
        attempt_id = ArtificatsApi._get_latest_attempt_id(artifacts)
    File "/opt/latest/lib/python3.7/site-packages/services/metadata_service/api/artifact.py", line 349, in _get_latest_attempt_id
        if artifact['attempt_id'] > attempt_id:
TypeError: string indices must be integers

From what I can tell, this can only occur when the result of this call returns a DBResponse object with a body of type string. And it looks like this would only happen in the case of error handling. Unfortunately it looks like the underlying error message is obfuscated by this error.


Reading text

Hi @paulocoutinhox ,
On your wasm example i have added the following to the intializefpdf
FPDF.Text_GetText = Module.cwrap('FPDFText_GetText', 'number', ['number', 'number', 'number', 'number']);
FPDF.Text_CountChars = Module.cwrap('FPDFText_CountChars', 'number', ['number']);
Text_CountChars & Text_GetText  always returns 0 even for pages that have plenty of text. In your compiled wasm example is there a settings/flag which disables the text getting generated?
Appreciate any help with this.

Hi @gamblor999,
No, i don't select the methods. All methods are exported, as you can see here:
https://github.com/paulocoutinhox/pdfium-lib/blob/master/modules/wasm.py#L697-L726
The methods that you say is used here:
https://github.com/paulocoutinhox/pdfium-lib/blob/d1afeb21b3b414c15a8841efc38e39c11ce35517/sample-apple/Sample/PDFium/PDFiumPage.m#L77-L90
You need check only if you are passing the correct parameters to FPDFText_CountChars.
Or you test your methods creating a test method inside the custom.cpp file:
https://github.com/paulocoutinhox/pdfium-lib/blob/d1afeb21b3b414c15a8841efc38e39c11ce35517/extras/wasm/utils/custom.cpp

@paulocoutinhox thanks a lot for your detailed reply which has resolved my issue:
The IOS example uses both
FPDF_LoadPage
FPDFText_LoadPage  Not used in WASM example
I needed to call FPDFText_LoadPage to be able to get FPDFText_CountChars .
Thanks again for your help.


SparkR statement support in azure-synapse-spark

Package Name: azure-synapse-spark
Package Version: 0.7.0
Operating System: any
Python Version: py3

Describe the bug
Now that Synapse notebooks support SparkR, please add support for sparkR in azure-synapse-spark so that we can submit sparkR statements using this package.
To Reproduce
Steps to reproduce the behavior:

Add SparkR in kind parameter of SparkStatementOptions

Expected behavior
Submit sparkR statement on Spark Pool
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Currently this package only support spark, pyspark, dotnetspark, sql statement kinds.

Thank you for your feedback @shakeelansari63 . We will get back to you asap

Thank you for the updates.
These enums come from https://github.com/Azure/azure-rest-api-specs/blob/main/specification/synapse/data-plane/Microsoft.Synapse/stable/2020-12-01/sparkJob.json#L1542.
Please open an issue in that repo.


Custom directive's inserted hook gets called when is not inserted

Version
2.5.16
Reproduction link
https://codesandbox.io/s/7ko5jno700?expanddevtools=1
Steps to reproduce
Please take a look at this code:

The key things here are :key attribute on the element and DOM manipulations in the inserted hook.
What is expected?
Expected to have the element mounted (or at least have a parent node)
What is actually happening?
No parent node

The documentation says the parent node is guaranteed: https://vuejs.org/v2/guide/custom-directive.html#Hook-Functions.

You shouln't manually move the content the dom that is handled by Vue because it relies on it to perform updates (Moving the el with  wrapper.appendChild(el);)


Unable to convert tensorrt SAM model

Hi, I am unable to export SAM decoder model to tensorrt. It failes with the following error -
[02/28/2024-12:37:37] [V] [TRT] Registering layer: /Tile for ONNX node: /Tile
[02/28/2024-12:37:37] [E] Error[4]: [graph.cpp::symbolicExecute::539] Error Code 4: Internal Error (/OneHot: an IIOneHotLayer cannot be used to compute a shape tensor)
[02/28/2024-12:37:37] [E] [TRT] ModelImporter.cpp:771: While parsing node number 108 [Tile -> "/Tile_output_0"]:
[02/28/2024-12:37:37] [E] [TRT] ModelImporter.cpp:772: --- Begin node ---
[02/28/2024-12:37:37] [E] [TRT] ModelImporter.cpp:773: input: "/Unsqueeze_3_output_0"
input: "/Reshape_2_output_0"
output: "/Tile_output_0"
name: "/Tile"
op_type: "Tile"

[02/28/2024-12:37:37] [E] [TRT] ModelImporter.cpp:774: --- End node ---
[02/28/2024-12:37:37] [E] [TRT] ModelImporter.cpp:777: ERROR: ModelImporter.cpp:195 In function parseGraph:
[6] Invalid Node - /Tile
[graph.cpp::symbolicExecute::539] Error Code 4: Internal Error (/OneHot: an IIOneHotLayer cannot be used to compute a shape tensor)
[02/28/2024-12:37:37] [E] Failed to parse onnx file
[02/28/2024-12:37:37] [I] Finished parsing network model. Parse time: 0.13611
[02/28/2024-12:37:37] [E] Parsing model failed
[02/28/2024-12:37:37] [E] Failed to create engine from model or file.
[02/28/2024-12:37:37] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8601] # /usr/src/tensorrt/bin/trtexec --onnx=assets/export_models/sam/onnx/xl1_decoder.onnx --minShapes=point_coords:1x1x2,point_labels:1x1 --optShapes=point_coords:16x2x2,point_labels:16x2 --maxShapes=point_coords:16x2x2,point_labels:16x2 --fp16 --saveEngine=assets/export_models/sam/tensorrt/xl1_decoder.engine --verbose

Environment:
torch                     2.2.1
torchaudio                2.2.1
torchelastic              0.2.2
torchpack                 0.3.1
torchprofile              0.0.4
torchvision               0.17.1
onnx                      1.15.0
onnxruntime               1.17.1
onnxsim                   0.4.35

ii  libnvinfer-bin                                    <IP_ADDRESS>-1+cuda12.0                      amd64        TensorRT binaries
ii  libnvinfer-dev                                    <IP_ADDRESS>-1+cuda12.0                      amd64        TensorRT development libraries
ii  libnvinfer-dispatch-dev                           <IP_ADDRESS>-1+cuda12.0                      amd64        TensorRT development dispatch runtime libraries
ii  libnvinfer-dispatch8                              <IP_ADDRESS>-1+cuda12.0                      amd64        TensorRT dispatch runtime library
ii  libnvinfer-headers-dev                            <IP_ADDRESS>-1+cuda12.0                      amd64        TensorRT development headers
ii  libnvinfer-headers-plugin-dev                     <IP_ADDRESS>-1+cuda12.0                      amd64        TensorRT plugin headers
ii  libnvinfer-lean-dev                               <IP_ADDRESS>-1+cuda12.0                      amd64        TensorRT lean runtime libraries
ii  libnvinfer-lean8                                  <IP_ADDRESS>-1+cuda12.0                      amd64        TensorRT lean runtime library
ii  libnvinfer-plugin-dev                             <IP_ADDRESS>-1+cuda12.0                      amd64        TensorRT plugin libraries
ii  libnvinfer-plugin8                                <IP_ADDRESS>-1+cuda12.0                      amd64        TensorRT plugin libraries
ii  libnvinfer-samples                                <IP_ADDRESS>-1+cuda12.0                      all          TensorRT samples
ii  libnvinfer-vc-plugin-dev                          <IP_ADDRESS>-1+cuda12.0                      amd64        TensorRT vc-plugin library
ii  libnvinfer-vc-plugin8                             <IP_ADDRESS>-1+cuda12.0                      amd64        TensorRT vc-plugin library
ii  libnvinfer8                                       <IP_ADDRESS>-1+cuda12.0                      amd64        TensorRT runtime libraries

I encountered such a mistake
[02/29/2024-10:13:33] [E] [TRT] ModelImporter.cpp:751: --- End node --- [02/29/2024-10:13:33] [E] [TRT] ModelImporter.cpp:754: ERROR: ModelImporter.cpp:179 In function parseGraph: [6] Invalid Node - /image_encoder/backbone/stages.4/op_list.1/context_module/main/Pad [shuffleNode.cpp::symbolicExecute::391] Error Code 4: Internal Error (/image_encoder/backbone/stages.4/op_list.1/context_module/main/Reshape_1: IShuffleLayer applied to shape tensor must have 0 or 1 reshape dimensions: dimensions were [-1,2]) [02/29/2024-10:13:33] [E] Failed to parse onnx file [02/29/2024-10:13:33] [I] Finish parsing network model [02/29/2024-10:13:33] [E] Parsing model failed [02/29/2024-10:13:33] [E] Failed to create engine from model. [02/29/2024-10:13:33] [E] Engine set up failed &&&& FAILED TensorRT.trtexec [TensorRT v8400] # trtexec --onnx=assets/export_models/sam/onnx/xl1_encoder.onnx --minShapes=input_image:1x3x1024x1024 --optShapes=input_image:4x3x1024x1024 --maxShapes=input_image:4x3x1024x1024 --saveEngine=assets/export_models/sam/tensorrt/xl1_encoder.engine

I encountered such a mistake
[02/29/2024-10:13:33] [E] [TRT] ModelImporter.cpp:751: --- End node ---
[02/29/2024-10:13:33] [E] [TRT] ModelImporter.cpp:754: ERROR: ModelImporter.cpp:179 In function parseGraph:
[6] Invalid Node - /image_encoder/backbone/stages.4/op_list.1/context_module/main/Pad
[shuffleNode.cpp::symbolicExecute::391] Error Code 4: Internal Error (/image_encoder/backbone/stages.4/op_list.1/context_module/main/Reshape_1: IShuffleLayer applied to shape tensor must have 0 or 1 reshape dimensions: dimensions were [-1,2])
[02/29/2024-10:13:33] [E] Failed to parse onnx file
[02/29/2024-10:13:33] [I] Finish parsing network model
[02/29/2024-10:13:33] [E] Parsing model failed
[02/29/2024-10:13:33] [E] Failed to create engine from model.
[02/29/2024-10:13:33] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8400] # trtexec --onnx=assets/export_models/sam/onnx/xl1_encoder.onnx --minShapes=input_image:1x3x1024x1024 --optShapes=input_image:4x3x1024x1024 --maxShapes=input_image:4x3x1024x1024 --saveEngine=assets/export_models/sam/tensorrt/xl1_encoder.engine

I encountered such a mistake
[02/29/2024-10:13:33] [E] [TRT] ModelImporter.cpp:751: --- End node ---
[02/29/2024-10:13:33] [E] [TRT] ModelImporter.cpp:754: ERROR: ModelImporter.cpp:179 In function parseGraph:
[6] Invalid Node - /image_encoder/backbone/stages.4/op_list.1/context_module/main/Pad
[shuffleNode.cpp::symbolicExecute::391] Error Code 4: Internal Error (/image_encoder/backbone/stages.4/op_list.1/context_module/main/Reshape_1: IShuffleLayer applied to shape tensor must have 0 or 1 reshape dimensions: dimensions were [-1,2])
[02/29/2024-10:13:33] [E] Failed to parse onnx file
[02/29/2024-10:13:33] [I] Finish parsing network model
[02/29/2024-10:13:33] [E] Parsing model failed
[02/29/2024-10:13:33] [E] Failed to create engine from model.
[02/29/2024-10:13:33] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8400] # trtexec --onnx=assets/export_models/sam/onnx/xl1_encoder.onnx --minShapes=input_image:1x3x1024x1024 --optShapes=input_image:4x3x1024x1024 --maxShapes=input_image:4x3x1024x1024 --saveEngine=assets/export_models/sam/tensorrt/xl1_encoder.engine

I had a similar issue and I solved it by replacing the problematic torch.repeat_interleave operation in the original SAM repo with monkey patch, see #78

Thanks for your solution!
I encountered the same problem under TensorRT <IP_ADDRESS> / 8.6.3.
It's worth noting that I tried on TensorRT 8.6.1, and trt did not report any conflict.
I am not sure whether they changed the way of the parsing of some nodes.


Add support to give input as JSON

We have to add support to the user to give the input file as JSON.

https://github.com/microplan-xyz/microplan/pull/59


Retrieval metrics : improvements

[ ] Add guardrails to metrics to check for the correct format.  Score NaN if returned otherwise
[ ] Differentiate between context relevance and precision
Relevance -> reference free ( can be used to optimize chunking/ ranking )
Precision -> works with ground truth

Hi @shahules786, thank you for these improvements, we were also a little bit confused by the different definition of context recall and precision, glad to know that it's going to be changed.
BTW, we have a question concerning the Relevance->reference free improments, what do you mean by reference free in this case?
Thanks for your response in advance.


Will need bom-2.249.x

Description
Plan to have a https://github.com/jenkinsci/bom line for 2.249.x, so will need to deploy to a new artifact id.

Submitter checklist for adding or changing permissions

Always

[X] Add link to plugin/component Git repository in description above

For a newly hosted plugin only

[ ] Add link to resolved HOSTING issue in description above

For a new permissions file only

[ ] Make sure the file is created in permissions/ directory
[ ] artifactId (pom.xml) is used for name (permissions YAML file).
[ ] groupId / artifactId (pom.xml) are correctly represented in path (permissions YAML file)
[ ] Check that the file is named plugin-${artifactId}.yml for plugins

When adding new uploaders (this includes newly created permissions files)

[ ] Make sure to @mention an existing maintainer to confirm the permissions request, if applicable
[ ] Use the Jenkins community (LDAP) account name in the YAML file, not the GitHub account name
[ ] Make sure to @mention the users being added so their GitHub account names are known if they require GitHub merge access (see below).
[ ] All newly added users have logged in to Artifactory at least once

Reviewer checklist (not for requesters!)

[ ] Check this if newly added person also needs to be given merge permission to the GitHub repo (please @ the people/person with their GitHub username in this issue as well). If needed, it can be done using an IRC Bot command
[ ] Check that the $pluginId Developers team has Admin permissions while granting the access.
[ ] In the case of plugin adoption, ensure that the Jenkins Jira default assignee is either removed or changed to the new maintainer.
[ ] If security contacts are changed (this includes add/remove), ping the security officer (currently @daniel-beck) in this pull request. If an email contact is changed, wait for approval from the security officer.

There are IRC Bot commands for it

see https://github.com/jenkinsci/bom/pull/301


Feat: enhance the Trace widget for batch consuming spans

Video
https://github.com/apache/skywalking-booster-ui/assets/20871783/6a94cba3-979c-47a0-b917-a8c6f09501ac
Signed-off-by: Qiuxia Fan<EMAIL_ADDRESS>

Please fix CI.

Please sync the UI submodule.


Version 2.5 background data just dissapear. No OSM neither background are loaded

Description
When you open V.2.5 in power use mode, no background image neither existing OSM data is visualized
Screenshots
https://rapideditor.org/rapid#map=17.21/-11.00341/-66.04040&background=EsriWayback_2024-12-12&datasets=fbRoads,msBuildings&disable_features=boundaries&overlays=osm-gps&poweruser=true
Version
2.5
What browser are you seeing the problem on? What version are you running?
EDGE Version 120.0.2210.91 (Official build) (64-bit)
The OS you're using
No response
Steps to reproduce
Open Rapid 2.5
go to -11.00341-66.04040
black screen for the area
The browser URL at the time you encountered the bug
https://rapideditor.org/rapid#map=17.21/-11.00341/-66.04040&background=EsriWayback_2024-12-12&datasets=fbRoads,msBuildings&disable_features=boundaries&overlays=osm-gps&poweruser=true
The auto-detected useragent string for your browser (leave blank if you're manually filling this form out)
No response

I'm having exactly the same issue.  on the same hardware & OS, Win7,
Firefox 115.18.0esr and Vivaldi 5.6.2867.62 (Stable channel) (64-bit) both flick straight to black screen.
Chrome 109.0.5414.120 works as normal.
Hopefully this helps.

Sorry to rush, but this failure is showing quite often, precluding even to make more than 6 or 7 map changes perr changeset, and also distorting the screen once jumping to black


[Help] : in lazy loading i am not able to expand till child node dynamically as parent nodes are not present

Hi Vakata,
In my application, i am expanding the tree to the selected node. it works fine if i don;t use to lazy loading but when i use lazy loading i am not able to expand my tree till selected node.
would you please give me sample json or code by which i can able to expand till child node dynamically. i am using same code in your example. it works fine when i expand the tree manually. but through code i am not able to expand it.
 								'core' : { 									'data' : { 									    'url' : function (node) { 									      return node.id === '#' ? '/application url to fetch root node : 'url to fetch child nodes'; 									    }, 									    'data' : function (node) { 									      return { 'id' : node.id }; 									    } 									} 								}
please help with sample code

i am using simple json   { "id" : "ajson1", "parent" : "#", "text" : "Simple root node" } as input to the jstree

Please read the docs and check the demos.

i didn't find any document which tell load child node if loading is lazy load.
select_node automatically expands the parents but in lazy load, parent node is not present. would you please tell me which document i should refer. it will really help. i have recently started doing development. so don't have much experience.


Create a new endpoint to add default status for users where it does not exists

This PR closes #754
Assumption:
Default status is "idle" for the user where it does not exists.
Endpoint: (Only superuser is authorized)
[PATCH] /users/defaultStatus  : It will return usernames of user for which default status ("idle") is added
response :
{
 "message": "Default status added to users successfully!",
 "count": 2,
 "updatedUsers": [
     "vrushalisjoshi",
     "ashleytitus18"
 ]
}

The order of PR
https://github.com/Real-Dev-Squad/website-backend/pull/738#issuecomment-1297216403

@samarpan1738 made the requested changes. Can you please check?

Closing this PR. As the new status collection is getting created which make this issue irrelevant https://github.com/Real-Dev-Squad/website-backend/issues/818


add pod annotations

+1 for this.  I was just looking to add this myself so that I could leverage KIAM to switch role the pod to one that can read secrets


feat(components): add Linaria for styling

Summary:
Split off from #221
Adds Linaria and changes the existing button component to a playground example.
Test Plan:

Jest tests pass
confirm that npm start loads Storybook

@anniehu4 ok to close for now?


Read model name as env variable from all samples

...so that all samples use the same model and it's easy to update the model

I'll close this, as we probably want different models for different samples


dynamically getting the artifact url

https://github.com/debezium/debezium-ui/pull/371

@uidoyen, @mdrillin could you handle this one?


Dark mode and refactor

Added dark mode

Live link here
https://angry-albattani-4a12ba.netlify.app/
Thanks for reviewing my PR

Everything looks good, one suggestion is name your localStorage item as mode and values as dark and light.

Thanks for revieing. I am actually just toggling the dark-mode class that's why I have given names such as enableDarkMode and disableDarkMode.


feat: improve color consistency

SUMMARY

This PR improves the existing color consistency. The main optimization is that in a dashboard if a color scheme is set for the dashboard (either in the color scheme select or via the metadata editor), all charts in the dashboard with the same label have the same color, while other labels are generated in the normal color scheme order.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

before

after

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

For testing purposes, used the virtual table as Dataset.
Create a Pie Chart 1) count(*) as metric 2) color as groupby 3) a,b,c in filter, save as new chart and create a dashboard.
Create a Pie Chart 1) count(*) as metric 2) color as groupby 3) d,e,f in filter, save as new chart and insert into previous dashboard.

In this PR

Current master(19daf65b5)

You are right @zhaoyongjie that's because this PR brings back sharing the same color instance across all the charts of a dashboard, as I mentioned in a comment above. However, @stephenLYZ said that is for testing only, so I assume it will be reverted soon.

I have actually reverted. 😂 I believe this PR should behave as expected. Since the column(label) of the second chart is different from the first one, they may not share the same color.

@geido I have actually reverted. 😂 I believe this PR should behave as expected. Since the column(label) of the second chart is different from the first one, they may not share the same color.

@stephenLYZ but they have the same number of series, they should use the available colors of the color scheme from first to last, individually. What @zhaoyongjie is showing in master is the correct and expected behaviour afaik.

Hey @stephenLYZ there is a lot of improvement that went into this. I am curious to see this in action. Do you think is it a good time to spin up a test env?

I think it's okay and even though I didn't add tinyColor to resolve the color conflict, I don't think it gets in the way of the overall flow logic

Hey @stephenLYZ there is a lot of improvement that went into this. I am curious to see this in action. Do you think is it a good time to spin up a test env?

I think it's okay and even though I didn't add tinyColor to resolve the color conflict, I don't think it gets in the way of the overall flow logic

Great. Let's make CI pass so that we can spin up the test env and start testing 🥳

/testenv up

/testenv up

/testenv up

Upon first glance at the test environment, I see that the dashboard I looked at was... consistent! Hooray! 👏
I also see that this solution works by effectively populating the JSON metadata of the dashboard with ALL the colors of ALL the series. I guess that works. That seems to cause a few potential issues:

I'm not sure if storing the metadata here might cause any potential bloat issues for the data, if there's some crazy number of series on the dashboard
If you go and delete the series colors from the metadata, it doesn't seem like there's any way to get it back (adding more series, changing palettes, etc).

Part of #2 is that I'm not sure what events trigger an update/refresh of the metadata. If there is such an event, I'd want to test and make sure that

Changing palettes (or any other action) doesn't overwrite any user-added custom colors
If you add a new series to two charts on a dashboard, those are also updated to match

In other words, this seems to work at first glance, but there are a few implications that make me nervous about the approach. I'm curious what @zhaoyongjie and/or @villebro might think of it.

Thank you for the information and i will try my best in testing, also @geido and i have a test plan for color consistency and i will use that for testing

/testenv up FEATURE_DASHBOARD_NATIVE_FILTERS=true FEATURE_DASHBOARD_CROSS_FILTERS=true FEATURE_ENABLE_EXPLORE_DRAG_AND_DROP=true FEATURE_ENABLE_DND_WITH_CLICK_UX=true FEATURE_UX_BETA=true

I have found some issue when testing:
1, in the dashboard with label_color, if i edit color theme in meta data, the charts with label_color wont be able to change the color theme, but the charts without  label_color can, with this edit, the color is not consistent for the dashboard
https://user-images.githubusercontent.com/81597121/150499902-c5340b7c-405d-474a-b19a-32d33adb3373.mov
2, When user open a label_color chart in dashboard, and then try to save the chart, the label_color will be gone after the chart is saved.
https://user-images.githubusercontent.com/81597121/150500623-c423a4e8-ca23-4a47-8076-e518396d098c.mov
3, Under specific condition of color#, the pie chart will show  same color for connected section, this issue is filed before and still exist
https://github.com/apache/superset/issues/17443

Upon first glance at the test environment, I see that the dashboard I looked at was... consistent! Hooray! 👏
I also see that this solution works by effectively populating the JSON metadata of the dashboard with ALL the colors of ALL the series. I guess that works. That seems to cause a few potential issues:

I'm not sure if storing the metadata here might cause any potential bloat issues for the data, if there's some crazy number of series on the dashboard
If you go and delete the series colors from shared_label_colors, it doesn't seem like there's any way to get it back (adding more series, changing palettes, etc).
Normally people would add custom colors to label_colors, but it's now possible (and easy and discoverable) to edit the shared_label_colors object instead. This may lead to problems with these colors being overwritten when changing color palette.

I totally agree with this. I think we should hide the object from the user as discussed previously. @stephenLYZ this shouldn't be a big problem. We should be able to delete the object from the editor before its presented to the user.
As for what concerns storing ALL labels, I think we agreed to only store the labels that are shared across different charts and not all of them. This might require a bit more logic in the code but it's important to avoid flooding the metadata.

There are more cases I didn't test. For example, If you add a new series to two charts on a dashboard, hopefully those are also updated to match other existing charts. We should capture all the test cases for either manual or (better yet) automated tests to make sure it all works. Pinging @jinghua-qa to make her aware of the complexity here.
In other words, this seems to work at first glance, but there are a few implications that make me nervous about the approach. I'm curious what @zhaoyongjie and/or @villebro might think of it.
One solution to much of the above might be to forgo putting the shared_label_colors in the (user editable) json metadata, and instead using the dashboard's new key/value endpoint. Then it's in the database, and (hopefully?) could be edited repeatedly without the user even knowing it exists. I think it could still be accessed as needed by Explore, too, but I'm not sure.

I think using the new key/value endpoint is a good idea. We could make sure the approach works correctly by using the metadata and then moving away from it in phase 2.

@stephenLYZ I also made a little bit of testing, below are some thoughts/findings:

When I remove the charts that share labels, I would expect that the shared_label_colors will get cleaned up from all of the labels that are not shared or that do not exist anymore in the context of the Dashboard. Imagine someone working on a large Dashboard and adding/removing charts multiple times, the shared_label_colors will eventually get very large with data that is not really needed.
When changing the color scheme from the Dashboard properties -> Color scheme, I saw that the shared_label_colors was wiped out and not re-generated, even though I have charts sharing labels. I was expecting that to be re-generated with the updated colors from the chosen scheme. It looks like this only happens when you remove or add a new chart in the Dashboard and then you change the color scheme. Watch the video below

https://user-images.githubusercontent.com/60598000/150560405-1d169b24-686f-4204-831e-252f3b4dc42f.mp4

When you add charts to the Dashboard, it looks like the shared_label_colors is not updated. It only gets updated when you change the color scheme. Also, it appears that the previous labels that were there already got deleted while they shouldn't.

https://user-images.githubusercontent.com/60598000/150562945-c3d0f2ab-6c48-41b5-a376-4f13dee72a9c.mp4

I deleted some charts and cleaned up the shared_label_colors for testing. Then I changed the color scheme. The shared_label_colors  is still getting the labels also of the charts that I deleted. I think it's not getting the most updated state of the Dashboard. It gets the correct state only when hard-refreshing the Dashboard page.

I'll be back to testing once these things are tackled as I don't want to put too much on the plate right now. Let's jump on a call if anything isn't clear or if you need any help sorting these out!

I have found some issue when testing:
1, in the dashboard with label_color, if i edit color theme in meta data, the charts with label_color wont be able to change the color theme, but the charts without label_color can, with this edit, the color is not consistent for the dashboard
Screen.Recording.2022-01-21.at.12.52.32.AM.mov
2, When user open a label_color chart in dashboard, and then try to save the chart, the label_color will be gone after the chart is saved.
Screen.Recording.2022-01-21.at.12.36.14.AM.mov
3, Under specific condition of color#, the pie chart will show same color for connected section, this issue is filed before and still exist #17443

fixed.
2 & 3. Currently the master has the same problem, I will open another PR to solve.

@geido @rusackas For now I will hide the shared_label_color field(here) and filter out the shared_label_color entered by the user(here).  And there are only two ways for users to update shared_label_color:

update color scheme, whether by color scheme select or json metadata editor:

https://user-images.githubusercontent.com/11830681/153238011-91ec5b2a-a6e3-4f20-9165-1330be04923f.mov

add chart or delete chart:

https://user-images.githubusercontent.com/11830681/153238370-0e9d9b71-295a-4eba-a522-c545d088f84d.mov

/testenv up FEATURE_DASHBOARD_NATIVE_FILTERS=true FEATURE_DASHBOARD_CROSS_FILTERS=true FEATURE_ENABLE_EXPLORE_DRAG_AND_DROP=true FEATURE_ENABLE_DND_WITH_CLICK_UX=true FEATURE_UX_BETA=true

Hey @stephenLYZ some comments after additional testing.
I had three charts sharing labels and the colors look right with using the Lyft Colors scheme. I removed two of the charts, leaving just one. Then I added back another chart that shared labels with the remaining one and the colors are not the same any longer. Check the screenshot.
Even after changing the color scheme the label colors still look incorrect for the first chart. It's like at some point it got ignored.
You can see this behavior in Dashboard http://<IP_ADDRESS>:8080/superset/dashboard/12 that I'll leave untouched on the test env for you to reproduce the problem.

In a Dashboard where no color scheme is used, I see that the same labels do not share the same colors. I am not sure if this is the wanted behavior but probably worth checking

For some reason, in that same Dashboard without color scheme, after applying the Lyft Colors color scheme, the label colors of the first chart are different from the others. I was under the assumption that with changing the color scheme the label colors would be regenerated, but it looks like this does not happen for the first chart for some reasons. You can check that behavior here http://<IP_ADDRESS>:8080/superset/dashboard/11. I think it's the same problem as the first point above.

/testenv up FEATURE_DASHBOARD_NATIVE_FILTERS=true FEATURE_DASHBOARD_CROSS_FILTERS=true FEATURE_ENABLE_EXPLORE_DRAG_AND_DROP=true FEATURE_ENABLE_DND_WITH_CLICK_UX=true FEATURE_UX_BETA=true

Hello @stephenLYZ some more comments after another round of testing:

In the sample Dashboard "World Bank's Data" I noticed different label colors. I am showing one of the cases in the screenshot. Some other charts seem to work correctly instead within the same Dashboard. FYI this Dashboard has a color scheme and no custom label colors.

This looks so nearly done! I am going to do another code related review as I think manual testing was pretty awesome so far. Great job!

I found an issue that when user customize the color for a label in a chart, then user open the chart in explore and save, the color order got refreshed and  the customize color label did not stay,  is this expected?
https://user-images.githubusercontent.com/81597121/154443187-4d434edc-3fd0-4676-8437-8a2498585027.mov

Hello @stephenLYZ some more comments after another round of testing:

In the sample Dashboard "World Bank's Data" I noticed different label colors. I am showing one of the cases in the screenshot. Some other charts seem to work correctly instead within the same Dashboard. FYI this Dashboard has a color scheme and no custom label colors.

This looks so nearly done! I am going to do another code-related review as I think manual testing was pretty awesome so far. Great job! 🚀 🧑🚀

Yes. Here need to add color consistency to the map charts.

/testenv up FEATURE_DASHBOARD_NATIVE_FILTERS=true FEATURE_DASHBOARD_CROSS_FILTERS=true FEATURE_ENABLE_EXPLORE_DRAG_AND_DROP=true FEATURE_ENABLE_DND_WITH_CLICK_UX=true FEATURE_UX_BETA=true

Hey @stephenLYZ I think I found a rather impactful issue.
Basically, when you change Dashboard, the color of the labels is persisted and applied also to other Dashboards. You can see that those get correctly removed only when hard-refreshing the page.
https://user-images.githubusercontent.com/60598000/155148263-29f8336e-1c20-4184-b6e6-e78f331a4677.mp4

/testenv up FEATURE_DASHBOARD_NATIVE_FILTERS=true FEATURE_DASHBOARD_CROSS_FILTERS=true FEATURE_ENABLE_EXPLORE_DRAG_AND_DROP=true FEATURE_ENABLE_DND_WITH_CLICK_UX=true FEATURE_UX_BETA=true

Hey @stephenLYZ I found that two labels within the same chart share exactly the same color. This reminded me of the color conflict resolution requirements that I think we skipped to prioritize testing looking at the threads above. What do you think?

Discussion result with @geido :
using tinycolor.analogous to generate similar colors for each color scheme, for example, if our color scheme is [A, B, C] then the generated colors are [A1, B1, C1, A2, B2, C2 ....]. Here the number of color labels is always less than or equal to the generated list of colors, in order to prevent getting the same colors.

Just for the sake of clarity, I am adding a bit more info here.
Let's say that we have Chart A and Chart B that share two labels. The same color for both labels would be set, but now Chart B has an adjacent label that also uses the same color. That's the main color conflict we need to resolve here.
The solution that Stephen is proposing above, generates new colors for the shared_label_colors using the analogous utility. That would assure (in most cases) that two adjacent labels would not use exactly the same color.
I think the solution works on paper, but I'd like to hear feedback from @rusackas @zhaoyongjie @jinghua-qa

/testenv up FEATURE_DASHBOARD_NATIVE_FILTERS=true FEATURE_DASHBOARD_CROSS_FILTERS=true FEATURE_ENABLE_EXPLORE_DRAG_AND_DROP=true FEATURE_ENABLE_DND_WITH_CLICK_UX=true FEATURE_UX_BETA=true


Editability Metrics

I have a question about the editability metrics presented in Table 1. Specifically, I am curious about the fitting power used in these metrics. Was the same power applied to all methods in the comparisons?
I ask because the choice of edit power can significantly influence the output image. For example, when adding glasses, a higher power could result in darker glasses, which could significantly affect the final FID estimate. Could you clarify what value was used for the editability power when calculating FID as an editability metric?
Thanks in advance.

Hi, thanks for your question!
The choice of editing power indeed has a lot of influence on metrics, and moreover, different methods often have different sensitivities to this parameter. For example, using the "Smile" edit at power 6 will be fine for e4e, but HyperInverter will produce artefacts -- too wide, unnatural smile. In this case it is necessary to reduce the power for the HyperInverter to make the edit work well.
To solve this problem and to compare fairly, we fix the editing direction and infer the method with a grid search of the editing power. For each power, we then calculate FID (according to the editing metric from paper) and choose the lowest one. If the difference of FID between the chosen power and its neighbourhood is large, we repeat this procedure within a smaller search range and use a smaller step in the greedy search.

Have we answered all your questions regarding the issue? Can we close it?

Hi,
Thank you for your first answer; however, I have the following question
Do you visually confirm the results of the edit, or do you have a minimum and maximum power threshold within the search grid? I ask because some methods shows that when the power is set to 1, the edit is applied to only a few images or is very minimal. This may result in a low FID score that does not accurately reflect the level of editing.

For this FID metric it is important to check that the edits applied in the method have the same editing effect as assumed in the chosen CelebaHQ markup.
We set a boundary and search for the minimum FID. If the found minimum is close to the boundary, we abroad the boundary and continue our repetitive procedure. In most cases, the found minimum correlates with the visual results: the editing was correctly applied to most of the images we saw. In our study, when we see that the editing power for the minimum FID is very low, we found that in most cases the reason for this is that the chosen editing direction and the CelebaHQ markup assume different edits. In this case, indeed, the optimum of the editing power will be close to inversion (i.e. editing power = 0), because increasing the editing power will increase the differences between the distributions of the image sets.
Does this answer your question?

Yes, thank you very much.


design: add gpuconfig

What type of PR is this?
/kind design
What this PR does / why we need it:
Adding GpuConfig to the contract
Which issue(s) this PR fixes:

Fixes #
Requirements:

[ ] uses conventional commit messages

[ ] includes documentation
[ ] adds unit tests
[ ] tested upgrade from previous version

Special notes for your reviewer:
Release note:
none

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 79.095%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2255

Relevant Lines:
2851

💛  - Coveralls


Adding Login to Policy-reporter UI screen

Hi,
I needed to know if there's any way to implement a logon screen/any authentication method for the policy reporter UI?

Thanks for the hint, will check it out!

I close it as duplicate to #221


Update Helper index.ts

Updating

Hey @FINEshaa , if you want to change the name of your group-generator, you need to change it for all the code using it. You can have a look to your badge-metadata and in group-generator/index.ts

thankyou:)


Support additional Singularity healthcheck settings

This is required to support the superset deploys.
At deployment time, the first container might start a RDS instance and wait for it. This can take quite a while....

@mihaitodor Do you know how to release nmesos / use the latest version in Jenkins?


fix: The  aggs result of NestedAggregator with sub NestedAggregator may be not accurately

Signed-off-by: kkewwei<EMAIL_ADDRESS>Description
The  aggs result of NestedAggregator with sub NestedAggregator may be not accurately, it depends on the face that the parentDocId > childDocId in search, but it may be wrong.
Related Issues
Resolves #13303
Check List

[x] New functionality includes testing.

[ ] All tests pass

[x] New functionality has been documented.

[ ] New functionality has javadoc added

[x] Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
[x] Commits are signed per the DCO using --signoff
[ ] Commit changes are listed out in CHANGELOG.md file (See: Changelog)
[x] Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@kkewwei thanks for opening the PR, it looks like the existing nested agg integration tests are failing after your change, could you please take a look? https://build.ci.opensearch.org/job/gradle-check/37514/#showFailuresLink
Also would you mind adding a changelog item as well? Thanks!

@jed326 I will process soon.

@jed326  please help review.

UNSTABLE

testThreeNodesNoClusterManagerBlock  #10006

❕ Gradle check result for a95ca76: UNSTABLE

TEST FAILURES:

      3 org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

testThreeNodesNoClusterManagerBlock #10006

Thanks @kkewwei, could you add an entry to CHANGELOG.md for this as well?

Other than that I think this PR looks good to me, @msfroh @reta mind giving a maintainer review here as well?

❌ Gradle check result for f3e4c6f: FAILURE
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

testReadRangeBlobWithRetries  #6090
testIndexingWithReplicaOnBwcNodes #9407
testSeqNoCheckpoints #9407
testIndexingWithPrimaryOnBwcNodes  #9407
testIndexVersionPropagation #9407
MixedClusterClientYamlTestSuiteIT #2440

testReadRangeBlobWithRetries #6090
testIndexingWithReplicaOnBwcNodes #9407
testSeqNoCheckpoints #9407
testIndexingWithPrimaryOnBwcNodes #9407
testIndexVersionPropagation #9407
MixedClusterClientYamlTestSuiteIT #2440

testReadRangeBlobWithRetries https://github.com/opensearch-project/OpenSearch/issues/6090
testIndexingWithReplicaOnBwcNodes https://github.com/opensearch-project/OpenSearch/issues/9407
testSeqNoCheckpoints https://github.com/opensearch-project/OpenSearch/issues/9407
testIndexingWithPrimaryOnBwcNodes https://github.com/opensearch-project/OpenSearch/issues/9407
testIndexVersionPropagation https://github.com/opensearch-project/OpenSearch/issues/9407
MixedClusterClientYamlTestSuiteIT https://github.com/opensearch-project/OpenSearch/issues/2440

❌ Gradle check result for 6a222a8: FAILURE
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

testReadRangeBlobWithRetries https://github.com/opensearch-project/OpenSearch/issues/6090
testIndexingWithReplicaOnBwcNodes https://github.com/opensearch-project/OpenSearch/issues/9407
testSeqNoCheckpoints https://github.com/opensearch-project/OpenSearch/issues/9407
testIndexingWithPrimaryOnBwcNodes https://github.com/opensearch-project/OpenSearch/issues/9407
testIndexVersionPropagation https://github.com/opensearch-project/OpenSearch/issues/9407
MixedClusterClientYamlTestSuiteIT https://github.com/opensearch-project/OpenSearch/issues/2440

Thanks @kkewwei , could you try to rebase from main to see if that fixes the flaky tests? We need to get the checks to green to be able to merge.

Can @msfroh or @andrross take a look please? Or volunteer another maintainer?

@reta @msfroh @andrross would one of you be able to give a maintainer review here?

@msfroh I think we need your help, I have difficulties understanding the impact of this change (sorry @jed326 @kkewwei )

FAILURE

AwarenessAttributeDecommissionIT.testConcurrentDecommissionAction #12197
IndicesRequestCacheIT.testDeleteAndCreateSameIndexShardOnSameNode @13600

@msfroh please see again.

@msfroh could you take another look at this one? Thanks

@msfroh, please have a look when you are free, or can you recommend someone else who has free time to take a look?

Sorry for the delay -- I think I finally understand what's going on here. The word "parent" seems to be overloaded, as it considers both the block-join parent (what I was assuming previously) and the parent aggregation (which may be collecting block-join parents, but might also be collecting block-join children).
I replied to some of my earlier comments -- I think we need some very clear code comments to explain the situation and clarify when the word "parent" means the parent aggregation versus when it means the block-join parent.
Thanks for your patience, and sorry for the confusion, @kkewwei!

@msfroh I will deal with as soon as possible.

I just rebased and fixed the changelog conflict. Should be good to merge once tests pass again.

@andrross please help merge.

There was another CHANGELOG conflict, pushed a resolution

Thanks @kkewwei! I promise we'll get this merged :)


Support using customized PrivateKey object from HSM provider in Crypto.java

Issue
The implement for signing is hard-coded to use the BouncyCastle provider. As the HSM provider that we are planing to use has its own implementation on the PrivateKey object, which is not compatible with BouncyCastle's implementation for creating signature. Signature cannot be created with HSM provider specific PrivateKey object.
Fix

remove the hard-coded part in Crypto.java
leverage the provider selection to java.security file
rename system properties athenz.crypto.bc_provider to athenz.crypto.key_factory_provider

The system properties athenz.crypto.bc_provider is not documented or found in the example configuration files. We think that it is better to clarify the usage of this system property.

@WindzCUHK  this PR will generate major performance issues for all of our clients. The crypto class is used when our clients verify their access/role tokens and we've seen major performance issues when using the JDK default provider instead of BC when handling EC keys.
So we can't just relegate the provider to the java.security file as we can't require all of our clients to make such a change as their upgrade. So the PR should introduce a logic where based on some configuration setting we don't default to using BC provider.

To be eligible for swag, please sign-up for Yahoo Hack Together, if you have not done so, yet.


Please provide a license

As i told you, i develop for the iobroker smarthome system. A lot of people will possibly use your code. So please add a license (its also mandatory at git)

Thank a lot, the license was added

Thank a lot, the license was added


fix kubespray package

What type of PR is this?

/kind bug
What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes https://github.com/kubean-io/kubean/issues/857
Special notes for your reviewer:

the test ci job https://github.com/hangscer8/kubean/actions/runs/5419977057

/lgtm


prepare automated release process via lerna/conventional-commits

This is ready for review now I think. I'll fix the commits on main (or our commitlint config) today to make the commit messages and our commitlint rules agree with each other.

The PR still has a couple of todo's and I think there are some small changes needed before we can merge it e.g. comment out "- run: bin/needs-release.js" for now?
I'll wait till the PR is in a final shape and review it again then :)

I have removed bin/needs-release.js again, good call. The remaining TODOs in the Github action are actually meant to stay there for now. But we can also throw out the whole Github action. It was just meant to give us a convenient preview on how lerna publish / lerna version will behave once we move this to CI.


Optimizing simple task continuations to run synchronously

This is a simple optimization change to have some of the simple task continuations run synchronously when appropriate and avoid the additional overhead incurred by scheduling them for asynchronous execution.

:ship:


[proposal] Add the ability to use --staging as a command line argument to sails lift

This PR makes it so you can do a sails lift --staging instead of having to use NODE_ENV=staging node app.js.
You can then create a file in config/env/staging.js that is a copy of production.js that suits your staging environment specific settings.
This works great but I feel we could further improve this by adding a tasks/register/buildStaging and tasks/register/staging.js as well as generating a sample config/env/staging.js for all the n00bs out there like myself.
As discussed with @mikermcneil over email. Hopefully this is how you envisioned it Mike. If not, please help me make it better so I can help. :)

Spelled proposal wrong :oncoming_police_car:

lgtm :+1:

thanks! @RWOverdijk :smile:

@RWOverdijk thanks :+1:
@jodonnell-broadsoft looks good.  One relatively trivial thing that would be a good idea while you're in there is to change the comment in lib/app/configuration/load.js to something like:
// --prod, --dev, and --staging CLI args

NODE_ENV vs. sails.config.environment
I still like what you and I discussed re: allowing sails.config.environment === 'staging' while process.env.NODE_ENV === 'production'  (e.g. lifting with NODE_ENV=production sails_environment='whatever' sails lift).  I think it'd be great to get that in as part of this PR (or separately, whatever makes sense).
The plan I originally sent you was a little incorrect, I just realized.  Here's a corrected version as a recap:

The code that automatically sets sails.config.environment to whatever NODE_ENV is only does so if sails.config.environment was not explicitly set.  This is good, and what we want (see https://github.com/balderdashy/sails/blob/master/lib/app/configuration/load.js#L132)

However, we should change the code a few lines after that that automatically sets NODE_ENV based on sails.config.environment.   Instead of always setting NODE_ENV to be the same as sails.config.environment, we should only do so if ( !process.env.NODE_ENV ) (i.e. the NODE_ENV env var was not explicitly set). I believe the relevant change would need to occur in the if statement here: https://github.com/balderdashy/sails/blob/master/lib/app/configuration/load.js#L150   (i.e. change it to the prettified/properly-formatted version of this: if (mergedConfig.environment && !process.env.NODE_ENV))

Then, to lift in staging, using the realistic NODE_ENV that will actually be used in production (i.e. which affects dependencies), you could do:
NODE_ENV=production sails lift --staging

# or:
# NODE_ENV=production sails_environment=staging sails lift

# or:
# put `"environment": "staging"` in your .sailsrc file

Re: the grunt stuff:
How about we change the Grunt stuff in sails www to match the way it works in the Grunt hook?  (See also the relevant code here)
We obviously can't do exactly that (i.e.  default.js / prod.js / <<SAILS.CONFIG.ENVIRONMENT>>.js), but we can do something comparable, e.g:
      var pathForEnvSpecificTaskList = path.resolve( sails.config.appPath, path.join('tasks/register/','build'+_.capitalize(sails.config.environment) + '.js') );
      if ( fs.existsSync(pathForEnvSpecificTaskList) ) {
        gruntTaskListName = 'build'+_.capitalize(sails.config.environment);
      }
      // If the environment is "production", use the "prod" tasklist (unless a "production" tasklist exists)
      else if (sails.config.environment === 'production') {
        gruntTaskListName = 'buildProd';
      }
      else {
        gruntTaskListName = 'build';
      }

So then:
# run the tasklist at `tasks/register/build.js`:
sails www

# try to run the tasklist at `tasks/register/buildStaging.js`, fall back to `tasks/register/build.js`:
sails www --staging
NODE_ENV=production sails www --staging  # same as above

# try to run the tasklist at `tasks/register/buildProduction.js`, fall back to `tasks/register/buildProd.js`:
NODE_ENV=production sails www
sails www --prod    # same as above
NODE_ENV=production sails www --prod  # same as above

Generators
re: generators, I think given the grunt stuff I mentioned above, we don't actually need to change the generators at all (if we change them, I fear that this could eventually be one of those slippery slope kinds of things).
Docs
That said, we definitely need to update the relevant docs to reflect:

sails www now uses sails.config.environment to determine the appropriate build tasklist to run (but is backwards compatible with buildProd.js and build.js)
sails.config.environment can now be set separately from NODE_ENV (but NODE_ENV will still be automatically set if you didn't specify it)
the new --staging option for Sails lift

Here's all the places I can think of offhand that will need to be updated:

I spent some time thoroughly documenting this behavior when working on the internal contributor docs for the Grunt hook, and just added a reference to it here in the default.js tasklist generated in new sails projects.  It would be great to do the same thing for the build.js tasklist as well as the buildProd.js tasklist.

The examples should show sails.config.environment rather than NODE_ENV

apparently we don't have a reference page for sails www in http://sailsjs.org/documentation/reference/command-line-interface, so it'd be great to add one and include the relevant information.
we should mention this in concepts/assets/default-tasks
I think we need to check the anatomy docs, the concepts/configuration docs, and also anywhere else I'm forgetting :p

Tests
We should add a test or two to test/integration/lift.lower.test.js.  When I added the behavior that restores NODE_ENV if it was automatically set, I added this one.  But now we should also ensure that NODE_ENV is not automatically set at all if it was explicitly specified;  i.e. that you can load and/or lift an app with environment: 'staging' and NODE_ENV=production, and NODE_ENV will remain untouched.
@jodonnell-broadsoft thanks for all your help!  Looking forward to this, it's awesome.

@mikermcneil Thanks for the feedback on this. I will look into this presently and give you a V2 of this Pr. Glad this is getting into the code. Should be cool :+1:

@mikermcneil
First of all, thanks so much for making my open source experience awesome. I really appreciate the time you are putting in with me :smile: . I know you could do this yourself much faster.
I made the changes you suggested and I think I am on the right track. Now need to add the docs and the tests. Will hopefully do that tomorrow. Also pulled in latest master if you want to review.
Thanks again guys :)

Seems like merging has changed some stuff that I did not intend. Will revert the merge and try again tomorrow.

I see now that master is also failing. Added this small PR to add the travis build flag to the readme page. #3652

And here is a docs PR. https://github.com/balderdashy/sails-docs/pull/665

So I have a failing test.
should revert NODE_ENV env variable if it was set automatically on load/lift
in lift.lower.test.js
This leads me to believe I have missed the case somewhere that the process.env.NODE_ENV should be reverted. Not 100% sure on the fix here. Would one of you guys mind taking a look? @RWOverdijk or @mikermcneil ?
Other than that, I think that all of your concerns should be resolved. Also added a docs PR that is linked here. :)

Will pull in latest on this and let you know.

+1 for this issue. What's the state of this?


Validator Status Watcher

Opening PR to see CI output as tests don't run locally on my machine anymore (due to missing solidity version) and to give others a chance to adopt this PR as I'm not gonna work today.
The way I quit when we exit the validator set looks a bit weird to me, but I didn't come up with something better. Would have been much nicer with Trio!
I also don't like how the is_validating status is passed through the confirmation task planner to the transfer recorder. But I don't see a way to avoid this.
Closes #276 and #306

You have to start two additional greenlets just to set a flag on the recorder and exit the program. This is too complicated.

What's complicated about it? The greenlets themselves are pretty trivial and, conceptually, having some sort of explicit task "wait until we're a validator and then start" makes it easier to understand what's going on.

Please simplify this by giving the a reference to the recorder and a shutdown function to the validator watcher. Get rid of the events and the two additonal greenlets.

I rejected this approach because it couples the validator status watcher with the confirmation planner even though they don't have anything to do with each other. Also, I tried to avoid using callbacks because they make the code hard to follow.


Can't upload django app to webapp

This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az webapp up
Errors:
[Errno 2] No such file or directory: ''
Traceback (most recent call last):
python3.7/site-packages/knack/cli.py, ln 206, in invoke
    cmd_result = self.invocation.execute(args)
cli/core/commands/__init__.py, ln 603, in execute
    raise ex
cli/core/commands/__init__.py, ln 661, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
cli/core/commands/__init__.py, ln 652, in _run_job
    cmd_copy.exception_handler(ex)
...
cli/command_modules/appservice/_create_util.py, ln 192, in parse_netcore_version
    parsed_file = ET.parse(file_path)
python3.7/xml/etree/ElementTree.py, ln 1197, in parse
    tree.parse(source, parser)
python3.7/xml/etree/ElementTree.py, ln 587, in parse
    source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory: ''

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az webapp up --sku {} -n {} -l {} --resource-group {}

Expected Behavior
Environment Summary
Darwin-18.2.0-x86_64-i386-64bit
Python 3.7.5
Shell: bash

azure-cli 2.0.76

Additional Context

Please use the latest version 2.0.77  this was a regression in 2.0.76. Please feel free to re-open if latest version doesn't resolve the issue. Thank you.

When you say latest version of 2.0.77, latest version of what?
From: Sisira Panchagnula<EMAIL_ADDRESS>Reply-To: Azure/azure-cli<EMAIL_ADDRESS>Date: Wednesday, November 27, 2019 at 6:24 PM
To: Azure/azure-cli<EMAIL_ADDRESS>Cc: Adam Clark<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [Azure/azure-cli] Can't upload django app to webapp (#11417)
Please use the latest version 2.0.77 this was a regression in 2.0.76. Please feel free to re-open if latest version doesn't resolve the issue. Thank you.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/Azure/azure-cli/issues/11417?email_source=notifications&email_token=AN4RFRXGANRFOQJRXYSWO6TQV36THA5CNFSM4JSI7UFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFK75ZQ#issuecomment-559283942, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AN4RFRQWJGT26JDIONMESJDQV36THANCNFSM4JSI7UFA.

@aclark-ax latest version of CLI https://pypi.org/project/azure-cli/


Fix all yetus problems after switch yetus to 0.14 version

Fix all yetus problems after switch yetus to 0.14 version.

about disable some linters in golangci-lint: we have warrnings in logs like this:
src/level=warning msg="[linters context] gosimple is disabled because of go1.18. You can track the evolution of the go1.18 support by following the https://github.com/golangci/golangci-lint/issues/2649."
src/level=warning msg="[linters context] staticcheck is disabled because of go1.18. You can track the evolution of the go1.18 support by following the https://github.com/golangci/golangci-lint/issues/2649."
src/level=warning msg="[linters context] structcheck is disabled because of go1.18. You can track the evolution of the go1.18 support by following the https://github.com/golangci/golangci-lint/issues/2649."
src/level=warning msg="[linters context] unused is disabled because of go1.18. You can track the evolution of the go1.18 support by following the https://github.com/golangci/golangci-lint/issues/2649."

And these warnings don't pass the yetus test.


Follow a Wikipedia category

This probably requires calling out to the Wikimedia API (at least it doesn't seem to appear in IRC), but it would be cool to be able to follow all edits of articles in a given category, e.g. http://en.wikipedia.org/wiki/Category:Current_events.

Hi @jarib , did you manage to find a way to perform this ?
Thanks!

Never did explore it further, no.


Pro Icons not available in fa_icons.py

I have recently discovered, that the fa_icons.py file is only containing the mappings for the free icons. Therefore people owning the pro version cannot make use of the pro icons, even if they have manually installed them.
This Line will only import free icons, when generating a new fa_icons.py file.
Having mappings for pro icons in the file, while only having free icons installed, will not do any harm.
therefore I propose to remove this line, and update the icon file

ping @myh2910

Seems fine to remove the line. The metadata of the free version only has free icons, so the line is useless anyway.


Add Aurora Global Database Serverless V2 CDK Sample

Description of changes:
The PR contains a CDK sample application that creates Amazon Aurora Global database cluster across a primary and secondary region for SaaS applications that need global footprint and for disaster recovery strategies. The stack also includes a Fargate container application to test the primary and secondary regions with CRUD API operations.

Added CDK code under /samples
Includes cdk-nag unit tests
Updated README.md at the root to reference the new sample

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Great, looks good to me!


Update images.md to keep titles consistent with toc.

Page lists 'Configuring Nodes to Authenticate to a Private Registry' as one of the methods for providing private registry creds. However, subtitle of that section refers to 'Repository' which this PR aims to change.

CLA signed

Nice catch! Thank you!
/lgtm
/approve