dynamo generate escapes values each time it runs

Every time I use dynamo generate, it escapes the values in dynamo.xml whether or not they were already escaped.
Example is the database property. After generating a new project and building it, I configured it to use a sqlite database. The database property from the .properties file looks like this:
$ grep database test.properties
database=sqlite:///test.db?user=test&synchronous=OFF&encoding=UTF-8
dynamo_database=sqlite:///test.db?user=test&synchronous=OFF&encoding=UTF-8

The dynamo.xml file has following content, with doubly escaped ampersand:
$ grep database dynamo.xml
    <property name="database">sqlite:///test.db?user=test&amp;synchronous=OFF&amp;encoding=UTF-8</property>

After running dynamo generate (or make generate), the dynamo_database value in the properties file, as well as the database property in the dynamo.xml file get escaped another time:
$ make generate
mkdir -p db
dynamo generate db
aws.gpr:15:04: warning: value "static-pic" is not used as label
aws.gpr:32:07: warning: value "static-pic" is not used as label
aws.gpr:67:07: warning: value "static-pic" is not used as label
Reading model file stored in 'db'
Generating mysql creation schema in 'db/mysql/create-test-mysql.sql'
Generating postgresql creation schema in 'db/postgresql/create-test-postgresql.sql'
Generating sqlite creation schema in 'db/sqlite/create-test-sqlite.sql'
Generating mysql creation schema in 'db/mysql/drop-test-mysql.sql'
Generating postgresql creation schema in 'db/postgresql/drop-test-postgresql.sql'
Generating sqlite creation schema in 'db/sqlite/drop-test-sqlite.sql'
Generated the SQL model from 11 Dynamo projects:
  /home/oenone/Programme/GNAT/2019/share/dynamo/awa_comments/dynamo.xml
  /home/oenone/Programme/GNAT/2019/share/dynamo/awa_workspaces/dynamo.xml
  /home/oenone/Programme/GNAT/2019/share/dynamo/awa_tags/dynamo.xml
  /home/oenone/Programme/GNAT/2019/share/dynamo/awa_counters/dynamo.xml
  /home/oenone/Programme/GNAT/2019/share/dynamo/ado/dynamo.xml
  /home/oenone/Programme/GNAT/2019/share/dynamo/awa/dynamo.xml
  /home/oenone/Programme/GNAT/2019/share/dynamo/awa_storages/dynamo.xml
  /home/oenone/Programme/GNAT/2019/share/dynamo/awa_images/dynamo.xml
  /home/oenone/Programme/GNAT/2019/share/dynamo/awa_blogs/dynamo.xml
  /home/oenone/Programme/GNAT/2019/share/dynamo/awa_wikis/dynamo.xml
  /home/oenone/Programme/GNAT/2019/share/dynamo/awa_jobs/dynamo.xml
$ grep database test.properties
database=sqlite:///test.db?user=test&synchronous=OFF&encoding=UTF-8
dynamo_database=sqlite:///test.db?user=test&synchronous=OFF&encoding=UTF-8
$ grep database dynamo.xml
    <property name="database">sqlite:///test.db?user=test&amp;amp;synchronous=OFF&amp;amp;encoding=UTF-8</property>

Every time I run generate afterwards, the two values get escaped again and again:
$ make generate
[content omitted, same as above]
$ grep database test.properties
database=sqlite:///test.db?user=test&synchronous=OFF&encoding=UTF-8
dynamo_database=sqlite:///test.db?user=test&amp;synchronous=OFF&amp;encoding=UTF-8
$ grep database dynamo.xml
    <property name="database">sqlite:///test.db?user=test&amp;amp;amp;synchronous=OFF&amp;amp;amp;encoding=UTF-8</property>
$ make generate
[content omitted, same as above]
$ grep database test.properties
database=sqlite:///test.db?user=test&synchronous=OFF&encoding=UTF-8
dynamo_database=sqlite:///test.db?user=test&amp;amp;synchronous=OFF&amp;amp;encoding=UTF-8
$ grep database dynamo.xml
    <property name="database">sqlite:///test.db?user=test&amp;amp;amp;amp;synchronous=OFF&amp;amp;amp;amp;encoding=UTF-8</property>

Any idea why this happens? Or is it some weird system configuration?
I found that the Save procedure in gen-model-projects.adb uses the function Util.Strings.Transforms.Escape_Xml but I couldn't find any Unescape_Xml function being used when loading the XML file.

Thanks for the bug report and the analysis!
The unescape is made by the XML/Ada sax parser.
I guess the issue is more that the Escape_Xml is done twice somewhere.
I think the issue is in the Save procedure which uses the Util.Serialize.IO.XML.Output_Stream
stream to write the file.
The XML output stream will handle the escape XML and the Save procedure should not escape it.


How can I calibrate my predicted ROI in the MMM with my geolift result ?

@becksimpson, thanks for the idea.
the things which is not clear to me is, how the below part of the code is used for model calibration here. Would be great if you can share some details Thank you
media_contribution = jnp.einsum(media_einsum, media_transformed, coef_media)
   observations = transform_kwargs.get('observations', None)
   if observations is not None:
     for i, obs in enumerate(observations):
       _ = numpyro.sample(
         name=f'observation_{i}',
         fn=dist.Normal(
           loc=media_contribution[
             obs['time_periods'],
             obs['channel']
           ].sum(axis=1),
           scale=0.2
         ),
         obs=jnp.array(obs['attributed_values'])
       )

channel_media_einsum = 'tc, c -> tc'

@becksimpson In this implementation, is the following media_contribution, now used in the prediction calculation on on line #411?
media_contribution = jnp.einsum(media_einsum, media_transformed, coef_media)    observations = transform_kwargs.get('observations', None)    if observations is not None:      for i, obs in enumerate(observations):        _ = numpyro.sample(          name=f'observation_{i}',          fn=dist.Normal(            loc=media_contribution[              obs['time_periods'],              obs['channel']            ].sum(axis=1),            scale=0.2          ),          obs=jnp.array(obs['attributed_values'])        )
Is it used here now on line 411 to replace jnp.einsum(media_einsum, media_transformed, coef_media)? Here's how I'm thinking media_contribution is now used:
  prediction = (       intercept + coef_trend * trend ** expo_trend +       seasonality * coef_seasonality +       media_contribution)


Add no-superfluous-tags and no-homogenous-tags rules

As promised, two new rules for tags.
I'm not sure about the no-homogenous-tags name - but I can't think of anything better - unless you've got a good idea?

Looks good!

Hi @vsiakka  Is there any chance you could release this PR?
Thanks,
Simon

Done, released with v2.5.0


Build error

Hi, i tried to build the latest release of your septentrio driver, but unfortunalty this is leading to a few problems:
it seems to mess up my colcon as well as it didnt build.
System specs:

OS: Ubuntu 22.04
Ros version: ROS2 Humble

Steps:

Copied packages into ros2 src folder of workspace
run colcon build --packages-select-septentrio_gnss_driver_ros2

Result:

colcon build --packages-select septentrio_gnss_driver_ros2
/home/johannes/.local/lib/python3.10/site-packages/setuptools/dist.py:723: UserWarning: Usage of dash-separated 'script-dir' will not be supported in future versions. Please use the underscore name 'script_dir' instead
warnings.warn(
/home/johannes/.local/lib/python3.10/site-packages/setuptools/dist.py:723: UserWarning: Usage of dash-separated 'install-scripts' will not be supported in future versions. Please use the underscore name 'install_scripts' instead
warnings.warn(
/home/johannes/.local/lib/python3.10/site-packages/setuptools_scm/_integration/setuptools.py:31: RuntimeWarning:
ERROR: setuptools==59.6.0 is used in combination with setuptools_scm>=8.x
Your build configuration is incomplete and previously worked by accident!
setuptools_scm requires setuptools>=61
Suggested workaround if applicable:

migrating from the deprecated setup_requires mechanism to pep517/518
and using a pyproject.toml to declare build dependencies
which are reliably pre-installed before running the build tools

warnings.warn(
[0.677s] WARNING:colcon.colcon_core.package_selection:ignoring unknown package 'septentrio_gnss_driver_ros2' in --packages-select
[0.705s] WARNING:colcon.colcon_ros.prefix_path.ament:The path '/home/johannes/ros2_ws/install/septentrio_gnss_driver' in the environment variable AMENT_PREFIX_PATH doesn't exist
[0.705s] WARNING:colcon.colcon_ros.prefix_path.catkin:The path '/home/johannes/ros2_ws/install/septentrio_gnss_driver' in the environment variable CMAKE_PREFIX_PATH doesn't exist
Summary: 0 packages finished [0.66s]

Hi @js-fwr,
the package name is septentrio_gnss_driver, there is no _ros2 at the end.


Can't find 'MatrixVectorActivation_0_wstrm' IP

Hi
I'm trying to make own RTL System containing finn stitched IP.

Build was done and I add all the stitched IP's which are located in highlighted(in image) directories by 'Settings->IP->Repository' in Vivado.
Almost of IP's are included in my system, but only MatrixVectorActivation_0_wstrm (also 1,2) module is absent.

Where can I find that IP?
Am I doing something wrong?
Thank you!

Oops it can be solved easily by #400.
Importing *.v file is all I need. Not IP.


Feature/guidedalignment

This request adds the implementation of the guided alignment as described in Guided Alignment Training for Topic-Aware Neural Machine Translation (Chen et al. 2016)
In summary:

preprocess.py: optionally stores source-to-target alignments in parse format (-alignfile, -alignvalfile)
s2sa/data.lua: if present, loads the alignments and converts them into dense format per batch
train.py: optionally creates a parallel criterion consisting of the decoder criterion (ClassNLLCriterion) with the criterion for guided alignment (MSECriterion) (--guided_alignment, --guided_alignment_weight, --guided_alignment_decay)
s2sa/models.lua: optionally exposes attention output in the decoder model

Cool! Also relevant:
http://arxiv.org/pdf/1609.04186.pdf
(the above work claims that cross entropy does slightly better than MSE for training the attention part of the model)


[DOC] Document the configuration of RPC node

Is your feature request related to a problem? Please describe.
There is no clear guide for node operator how to configure various settings like transaction limits or view call limits.
Describe the solution you'd like
A page for node configuration details in section that contains other articles like https://docs.near.org/docs/develop/node/rpc/run-rpc-node-with-nearup

@janewang @bowenwang1996  Is this something NodeX team can own?

@frol Yes


Issue 56 create admin interface

Change Summary

created admin interface for all models
Fixed unmatched field name in Tasks model: estimated_price -> budget
Fixed validation bug in Tasks model:

budget was string, and compared to number
deadline was compared to None, because auto-filled created_at is not pollulated until instances is created.

Change Form
Fill this up (NA if not available). If a certain criteria is not met, can you please give a reason.

[x] The pull request title has an issue number
[x] The change works by "Smoke testing" or quick testing
[x] The change has tests
[x] The change has documentation

Other Information
How to use admin interface?

Create a admin user
python manage.py createsuperuser

Run developmnent server
python manage.py runserver

Access admin page by accessing
http://<IP_ADDRESS>:8000/admin/

Related issue

Resolve #56

LGTM


Fix indentation for skip_row docstring

Without the newline before the code block intersphinx isn't able to parse the docstring. I also adjusted the indentation of the block and propose to use the python environment for the code block.
Problem
Intersphinx isn't able to parse the docstring for skip_row which is results in failing documentation builds.
Related issue: #1742
Solution
I added a newline before the code block and optionally adjusted the indentation and code environment of the block.

Thanks for submitting.  Feel free to add your name to AUTHORS and an update to CHANGELOG.rst would be appreciated.

coverage: 100.0%. remained the same
when pulling 8039e0d9cce38d99953c0cc341404a8f8a880a60 on ulliholtgrave:patch-1
into eabba74bbfe0128411612c064b8e65900ab377ee on django-import-export:main.

@matthewhegarty Thanks for the quick response. I think I applied the changes for the check. I also found some more indentation issues that intersphinx wasn't happy about. If you run your setup on --nitpick mode this also breaks.

I already saw the failed test. Will look into it :)

Thanks.  I couldn't run with nitpicky - it didn't work for me, even when I installed the dev version of Sphinx.  Please could you raise a new issue which shows how to repeat the break with nitpicky?

Hi - any chance you could look into this so I can get the fix into the next release.
Many thanks

np - thanks for taking this on, much appreciated.

Thank you - will merge


Added page change  notifications [#181511767]

The page change notification displays a please wait message after 500ms on a page change along with an error with a 3000ms delay.  Previously no please wait message was shown and any error was shown in a Javascript alert.

The error case in app.tsx is not covered by tests, but I'd assume that would a bit annoying to cover. We'd need to use some time modification library so it doesn't need to wait for the timeout.

Jest has support for fake timers which should eliminate the wait, although error cases can be hard to test for other reasons as well.


template = data['template'].cuda() IndexError: too many indices for tensor of dimension 4

There was an error when I transferred the siamprn model to onnx or solidified the model：
template = data['template'].cuda()
IndexError: too many indices for tensor of dimension 4
function：
def siammodel2onnx():
# load config
cfg.merge_from_file(args.config)
cfg.CUDA = torch.cuda.is_available() and cfg.CUDA
device = torch.device('cuda' if cfg.CUDA else 'cpu')
# create model
model = ModelBuilder()
# load model
model.load_state_dict(torch.load(args.snapshot,
                                 map_location=lambda storage, loc: storage.cpu()))
model.eval().to(device)
#print(model)
#trace_model = torch.jit.trace(model, torch.Tensor(1, 3,1000, 100))
#trace_model.save('./siamrpn.pt')
#Export the trained model to ONNX
dummy_input = Variable(
    torch.randn(1, 3, 960, 540))
torch.onnx.export(model, dummy_input, "output/siamrpn.onnx")

I believe this is more a PyTorch issue.

@masommao Do you convert pth model to onnx  successfully？

Since no further question is asked, I'm closing this issue for now.
Please create a new issue should you have other questions.


feat(curriculum): dice game project

Checklist:

[x] I have read and followed the contribution guidelines.
[x] I have read and followed the how to open a pull request guide.
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

@JoyShaheb I have been testing the first few steps in the browser. When I submit and go to the next challenge, I am seeing a challenge from the leaderboard project.

Hey Joy 👋🏽
I pulled in the project locally so I can give a detailed review.
Could you please implement the review suggestions from Zaira and @gikf first? Just to make sure I don't ask you to do the same thing.

NOTE:
Currently working on cleaning up the general flow and language for this dice game.
Since this is still a WIP, some of the automated checks won't pass yet.


 wrapping button content

I found a problem with centre aligning text in a button on Safari. It's to do with the width being set and Safari (and iPhone) then ignoring the text-align attribute. Putting  around the button content resolved the problem. (One shortcoming of the solution is that onClick handlers on the button may need to check if the target is the span or the button if they need to deal with the target itself.)

LGTM 👍


Filter _changes by key to speed up replication

Expected Behavior
From what I understand, using a view for filtered replication is a lot faster than a filter function as after a view is built it can be reused. I believe that filter functions on the other hand need to be executed each time there is a replication and are not "cached." So, I've found that as a general rule, you should try to use a view whenever you need to filter replication.
The issue is that in order for me to swap out some of my filter functions with views for filtered replication then I'd need to be able to query the _changes endpoint using a key parameter. Is there a reason that a key parameter cannot be added to the _changes API. (Of course, if this is added then the key parameter could also be added to the PouchDB Replication API. This enhancement would lead to significant speed increases in CouchDB/PouchDB initial replication for many data sets)
Current Behavior
The _view API has a key parameter and it functions great when you want to get a list of all the latest docs. No such key parameter exists for the _changes endpoint.
Steps to Reproduce
The following is a simple example that illustrates the need. Assume a database named people with the design doc
{
  _id: '_design/views',

  views: {

    students_by_class: {
      map: [
        'function(doc) {',
        'if (doc.type === "student") {',
        'emit(doc.class, null);',
        '}',
        '}'
      ].join(' ')
    }

  }
}

And assume the following docs:
{
  type: 'teacher',
  name: 'Teacher 1',
  class: 'Calculus'
},
{
  type: 'student',
  name: 'Student 1',
  class: 'Calculus'
},
{
  type: 'student',
  name: 'Student 2',
  class: 'Algebra'
},
{
  type: 'student',
  name: 'Student 3',
  class: 'Calculus'
}

Currently, I can only query for all students and not for those in a specific class:people/_changes?filter=_view&view=views/students_by_class. Instead it would be great to be able to issue people/_changes?filter=_view&view=views/students_by_class&key=Calculus because then I could use a view instead of a filter function in my replication.
Context
Not having the key parameter makes replication a lot slower
Other
I'd be willing to take a stab at modifying the CouchDB source to add this feature, but I just want to confirm that it is a good idea and that I'm not oversimplifying the enhancement.

Maybe another option is to use a Mango based selector as described here:
https://blog.couchdb.org/2016/08/15/feature-replication/comment-page-1/ (see A New Way to Filter section).
Then each replication document would specify a selector to pick from a specific class only.
By default the basic view based filtering doesn't end up using the view data and just re-uses the map function in the most naive way.
There is a feature which allows using the view. It was implemented here:
https://github.com/apache/couchdb-couch-mrview/pull/2
I think it involves added these options to the view "options": {"seq_indexed": true, "keyseq_indexed": true}}. But have never used it and not sure how it works.

@nickva great point on the selector and that is exactly what we have already done as an interim solution and it appears to be a little faster. Ultimately though, using a view with a cached index would be faster, no?
So, in the case of the view endpoint: people/_design/views/_view/students_by_class?key=Calculus, isn't CouchDB using the view data? I haven't dug into the code, but from a high level testing perspective, this type of query appears to run quite fast.

For what it worth, seq and kseq indexing in mrview is broken right now #592, so in Couch 2.1 so called fast_view for replication shorted cut to regular view filtering, that evaluates each doc interactively and not actually using built index.

@redgeoff the trick is that replication needs to be incrementally resumable, so if you want to build an index to drive filtered replication the server also needs to maintain a second internal index to get a list of all the changed and deleted keys in that index since your last request. Hopefully that makes sense. That's the PR @nickva pointed you to, although as @eiri mentioned it's not currently functional.
You're certainly right that the normal _view API uses a dedicated index and is quite fast, but in the default configuration a view has no way to efficiently deliver you a subset of recently changed rows.

Ah interesting, so this is more complicated than I had hoped :(. I guess we'll have to wait until #592 is fixed before a key parameter can be added and actually used.

@nickva is theoretically possible to speed-up selectors passed in replicator to use indexes? I have the same issue as @redgeoff posted and rewriting my map functions as mango selectors. Since the map functions often time written somethine like:
function (doc) {
   if (doc.type == 'foo' && doc.save && ...) {
      emit(...)
   }
}

using partial indexes would make a lot of sense overall.

@nickva I guess i know the answer: to build _changes feed the changes themselves have to be indexed somewhere, meaning a similar to seq_indexed option must be implemented for selectors, which pretty much brings us to point 1

@evsukov89 Yes, you are exactly right!

I believe we're seeing this too. Our JS filter takes a parameter from the request.
We'd love to try mango filter, but I can't find how to access a request parameter from mango.
e.g. "$gt": request.startDate

@oliverjanik when using a mango filter for changes the entire filter is specified in the request body , so you can just adjust the filter as you need instead of passing custom parameters. For example:
POST /mydb/_changes?filter=_selector
{
    "selector": { "startDate": { "$gt": "2018-07-01" } }
}

Closing as there isn’t anything actionable in here.

@janl I wouldn’t recommend closing this as I think this is a vital feature. I’d even argue that it is a design flaw in CouchDB. Using mango selectors for this is slow and would probably be much too slow for a large dataset. If this remains closed is there a list that this will be added to?
I understand that this would take time to implement and that someone has to do the work :). I just don’t want to see something like this get forgotten.

@whoali lets take an example: you have an app that uses couchdb as the main data storage. It is intended to be used with offline-first clients (PouchDB/CouchBase Lite 1.x). Let’s assume that overall there is about 1k docs to replicate.
Generally speaking you have two options:

on the server create a new “proxy” database, replicate this 1k docs there and finally use it as a source from offline clients to replicate from
use filtered replication

Analyzing the consequences of each solution:

added complexity, data duplication, additional server usage resources - in my understanding per-user db was at least partiality made to addresss this problem, but I think the shortcomings are known by now
since the _changes feed is not indexed, what is the CPU+IO cost of filtering 1k docs over the db of 1M docs? what if you have 1k users, 10k, 100k?

@redgeoff we have a patch of indexed changes implementation based on views which works well for us, including filtering by key, etc. We can share the details if you wanna wish.
I do however agree this approach has a few shortcomings: view and _changes outputs are not exactly idempotent. For ex: if the view no longer emits the doc, how this should be reflected in the _changes feed?
@janl @whoali I’m sorry for my ignorance, but would you mind pointing me to the page explaining the process of submitting proposals for couchdb?


Update dependency on parity-wasm

Blocked on:

https://github.com/paritytech/wasm-utils/issues/119
https://github.com/paritytech/wasmi/issues/168

This is unblocked.


Integrate TravisBuddy

TravisBuddy will comment on pull requests in your public repository everytime a test failed in one of them. The comment will include only the part of the build log that applies to your testing framework, so your contirbutors won't have to enter Travis's website and search the long and annoying build log for the reason the tests failed.
Here's an example: bluzi/static-server#1

Travis tests have failed
Hey Eliran Pe'er,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.
travis_retry tox
GLOB sdist-make: /home/travis/build/msabramo/python-panci/setup.py
py26 create: /home/travis/build/msabramo/python-panci/.tox/py26
ERROR: InterpreterNotFound: python2.6
___________________________________ summary ____________________________________
ERROR:   py26: InterpreterNotFound: python2.6

The command "tox" failed. Retrying, 2 of 3.

GLOB sdist-make: /home/travis/build/msabramo/python-panci/setup.py
py26 create: /home/travis/build/msabramo/python-panci/.tox/py26
ERROR: InterpreterNotFound: python2.6
___________________________________ summary ____________________________________
ERROR:   py26: InterpreterNotFound: python2.6

The command "tox" failed. Retrying, 3 of 3.

GLOB sdist-make: /home/travis/build/msabramo/python-panci/setup.py
py26 create: /home/travis/build/msabramo/python-panci/.tox/py26
ERROR: InterpreterNotFound: python2.6
___________________________________ summary ____________________________________
ERROR:   py26: InterpreterNotFound: python2.6

The command "tox" failed 3 times.

travis_retry tox
GLOB sdist-make: /home/travis/build/msabramo/python-panci/setup.py
py27 create: /home/travis/build/msabramo/python-panci/.tox/py27
py27 installdeps: -r/home/travis/build/msabramo/python-panci/test_requirements.txt
py27 inst: /home/travis/build/msabramo/python-panci/.tox/dist/panci-0.0.1.zip
py27 installed: asn1crypto==0.23.0,attrs==17.3.0,certifi==2017.11.5,cffi==1.11.2,chardet==3.0.4,coverage==3.7.1,coveralls==1.2.0,cryptography==2.1.4,docopt==0.6.2,enum34==1.1.6,funcsigs==1.0.2,idna==2.6,ipaddress==1.0.18,panci==0.0.1,pluggy==0.6.0,py==1.5.2,pycparser==2.18,pyOpenSSL==17.5.0,pytest==3.3.0,pytest-cov==2.5.1,PyYAML==3.12,requests==2.18.4,six==1.11.0,urllib3==1.22
py27 runtests: PYTHONHASHSEED='2350375376'
py27 runtests: commands[0] | py.test --cov panci
============================= test session starts ==============================
platform linux2 -- Python 2.7.13, pytest-3.3.0, py-1.5.2, pluggy-0.6.0
rootdir: /home/travis/build/msabramo/python-panci, inifile:
plugins: cov-2.5.1
collected 38 items

panci/tests/test_ciconfig.py ................................            [ 84%]
panci/tests/test_travis2tox.py ......                                    [100%]

---------- coverage: platform linux2, python 2.7.13-final-0 ----------
Name                          Stmts   Miss  Cover
-------------------------------------------------
panci/__init__                    0      0   100%
panci/ciconfig                   58      5    91%
panci/cli                        17     17     0%
panci/tests/__init__              0      0   100%
panci/tests/test_ciconfig       142      0   100%
panci/tests/test_travis2tox      28      0   100%
panci/tox2travis                  6      6     0%
panci/toxconfig                  36     24    33%
panci/toxquickstart              97     97     0%
panci/travis2tox                 12      1    92%
panci/travisconfig               18      3    83%
-------------------------------------------------
TOTAL                           414    153    63%

========================== 38 passed in 0.18 seconds ===========================
py27 runtests: commands[1] | coveralls
Submitting coverage to coveralls.io...
Coverage submitted!
Job #67.2
https://coveralls.io/jobs/31679702
___________________________________ summary ____________________________________
  py27: commands succeeded
  congratulations :)

travis_retry tox
GLOB sdist-make: /home/travis/build/msabramo/python-panci/setup.py
py33 create: /home/travis/build/msabramo/python-panci/.tox/py33
ERROR: InterpreterNotFound: python3.3
___________________________________ summary ____________________________________
ERROR:   py33: InterpreterNotFound: python3.3

The command "tox" failed. Retrying, 2 of 3.

GLOB sdist-make: /home/travis/build/msabramo/python-panci/setup.py
py33 create: /home/travis/build/msabramo/python-panci/.tox/py33
ERROR: InterpreterNotFound: python3.3
___________________________________ summary ____________________________________
ERROR:   py33: InterpreterNotFound: python3.3

The command "tox" failed. Retrying, 3 of 3.

GLOB sdist-make: /home/travis/build/msabramo/python-panci/setup.py
py33 create: /home/travis/build/msabramo/python-panci/.tox/py33
ERROR: InterpreterNotFound: python3.3
___________________________________ summary ____________________________________
ERROR:   py33: InterpreterNotFound: python3.3

The command "tox" failed 3 times.

travis_retry tox
GLOB sdist-make: /home/travis/build/msabramo/python-panci/setup.py
py34 create: /home/travis/build/msabramo/python-panci/.tox/py34
py34 installdeps: -r/home/travis/build/msabramo/python-panci/test_requirements.txt
py34 inst: /home/travis/build/msabramo/python-panci/.tox/dist/panci-0.0.1.zip
py34 installed: attrs==17.3.0,certifi==2017.11.5,chardet==3.0.4,coverage==3.7.1,coveralls==1.2.0,docopt==0.6.2,idna==2.6,panci==0.0.1,pluggy==0.6.0,py==1.5.2,pytest==3.3.0,pytest-cov==2.5.1,PyYAML==3.12,requests==2.18.4,six==1.11.0,urllib3==1.22
py34 runtests: PYTHONHASHSEED='365014332'
py34 runtests: commands[0] | py.test --cov panci
============================= test session starts ==============================
platform linux -- Python 3.4.3, pytest-3.3.0, py-1.5.2, pluggy-0.6.0
rootdir: /home/travis/build/msabramo/python-panci, inifile:
plugins: cov-2.5.1
collected 38 items

panci/tests/test_ciconfig.py ................................            [ 84%]
panci/tests/test_travis2tox.py ......                                    [100%]

----------- coverage: platform linux, python 3.4.3-final-0 -----------
Name                          Stmts   Miss  Cover
-------------------------------------------------
panci/__init__                    0      0   100%
panci/ciconfig                   58      5    91%
panci/cli                        17     17     0%
panci/tests/__init__              0      0   100%
panci/tests/test_ciconfig       142      0   100%
panci/tests/test_travis2tox      28      0   100%
panci/tox2travis                  6      6     0%
panci/toxconfig                  36     22    39%
panci/travis2tox                 12      4    67%
panci/travisconfig               18      3    83%
-------------------------------------------------
TOTAL                           317     57    82%

========================== 38 passed in 0.23 seconds ===========================
py34 runtests: commands[1] | coveralls
Submitting coverage to coveralls.io...
Source file is not python /home/travis/build/msabramo/python-panci/panci/toxquickstart.py
Coverage submitted!
Job #67.4
https://coveralls.io/jobs/31679700
___________________________________ summary ____________________________________
  py34: commands succeeded
  congratulations :)

travis_retry tox
GLOB sdist-make: /home/travis/build/msabramo/python-panci/setup.py
py35 create: /home/travis/build/msabramo/python-panci/.tox/py35
ERROR: InvocationError: Failed to get version_info for python3.5: pyenv: python3.5: command not found

The `python3.5' command exists in these Python versions:
  3.5
  3.5.3

___________________________________ summary ____________________________________
ERROR:   py35: InvocationError: Failed to get version_info for python3.5: pyenv: python3.5: command not found

The `python3.5' command exists in these Python versions:
  3.5
  3.5.3

The command "tox" failed. Retrying, 2 of 3.

GLOB sdist-make: /home/travis/build/msabramo/python-panci/setup.py
py35 create: /home/travis/build/msabramo/python-panci/.tox/py35
ERROR: InvocationError: Failed to get version_info for python3.5: pyenv: python3.5: command not found

The `python3.5' command exists in these Python versions:
  3.5
  3.5.3

___________________________________ summary ____________________________________
ERROR:   py35: InvocationError: Failed to get version_info for python3.5: pyenv: python3.5: command not found

The `python3.5' command exists in these Python versions:
  3.5
  3.5.3

The command "tox" failed. Retrying, 3 of 3.

GLOB sdist-make: /home/travis/build/msabramo/python-panci/setup.py
py35 create: /home/travis/build/msabramo/python-panci/.tox/py35
ERROR: InvocationError: Failed to get version_info for python3.5: pyenv: python3.5: command not found

The `python3.5' command exists in these Python versions:
  3.5
  3.5.3

___________________________________ summary ____________________________________
ERROR:   py35: InvocationError: Failed to get version_info for python3.5: pyenv: python3.5: command not found

The `python3.5' command exists in these Python versions:
  3.5
  3.5.3

The command "tox" failed 3 times.

travis_retry tox
GLOB sdist-make: /home/travis/build/msabramo/python-panci/setup.py
pypy create: /home/travis/build/msabramo/python-panci/.tox/pypy
ERROR: InterpreterNotFound: pypy
___________________________________ summary ____________________________________
ERROR:   pypy: InterpreterNotFound: pypy

The command "tox" failed. Retrying, 2 of 3.

The command "tox" failed 3 times.

Coverage remained the same at 63.527% when pulling 59b3bb0591f670943bfd2b265fbd4076d831b570 on bluzi:master into fb296d9880f6c6d065579db3b13cbe7b58a67611 on msabramo:master.

Thanks, @bluzi for pointing out TravisBuddy to me! I integrated it in #27.


Save password in Safari and Firefox

My Lychee installation is one of the few pages where Safari (OS X) doesn't offer to save password. Have you labelled the input fields in some non-standard way so Safari doesn't treat them like username/password? I'm not sure what triggers Safari "save password?" dialog, but it would be very nice to save Lychee username/password and have it filled in automatically afterwards.

The same happens (or doesn't actually!) in Chrome. It's because the login dialog does not contain a <form> that gets submitted (I'm assuming the login is handled via XHR instead).
Using <form> + preventDefault() may fix it.

@Klikini Thanks for the suggestion.
The problem exists since the first version of Lychee. I always thought it's because we're adding the login-dialog via JS. Fixing it would be a great thing. PRs or additional information are welcome!

The above works fine in Chrome, but I could not test Firefox because it will not offer to save passwords regardless of anything in the page code if the site is running from localhost. I also don't have access to Safari, so it should probably be tested there before this is considered fixed.

Safari failed to show the login suggestions because Lychee filled in the username automatically. This has been removed and I successfully tested it with Chrome and Safari. I'm still not sure what triggers Safari "save password?" dialog, but we might try to use a  for the login window (as you already did in the PR. Just need to do some additional test as it hasn't worked for me).

Firefox 49, Lychee 3.1.5 - can confirm this is still happening

This issue will be closed because I'm not actively working on Lychee anymore. The repository of Lychee will stay available, but the main development continues under the Lychee Organisation. A fork maintained by the community. Important issues have been migrated. Feel free to re-add your issue(s) when necessary.


tests: Update test_restore_inc

The backup & restore should involve at least 2 blocks.

https://ci.longhorn.io/job/private/job/longhorn-tests-regression/167


Add Tutel boost for Fairseq MoE acceleration

Add optimized library implementation to accelerator MoE module.
Before submitting

[ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
[x] Did you read the contributor guideline?
[ ] Did you make sure to update the docs?
[ ] Did you write any new necessary tests?

What does this PR do?
To accelerate MoE execution in fairseq if tutel package is installed.
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃

This seems to be breaking when running on multiple processes (gpus).
  File "/private/home/namangoyal/src/fairseq_gshard/fairseq-py/fairseq/modules/moe/moe_layer.py", line 165, in forward
    l_aux, self.metadata, C, E, indices_, locations_, gates_ = self.gate(reshaped_input, reshaped_input_padding_mask)
  File "/private/home/namangoyal/.conda/envs/fairseq-20210102-py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/private/home/namangoyal/src/fairseq_gshard/fairseq-py/fairseq/modules/moe/top2gate.py", line 247, in forward
    return top2gating(
  File "/private/home/namangoyal/src/fairseq_gshard/fairseq-py/fairseq/modules/moe/top2gate.py", line 131, in top2gating
    locations1 = fused_cumsum_sub_one(mask1)
  File "/private/home/namangoyal/src/tutel/tutel/jit_kernels/gating.py", line 79, in fast_cumsum_sub_one
    return get_cumsum_kernel(data.size(0), data.size(1))(data)
  File "/private/home/namangoyal/src/tutel/tutel/jit_kernels/gating.py", line 70, in optimized_cumsum
    base_kernel(mask1.to(torch.int32), locations1)
  File "/private/home/namangoyal/src/tutel/tutel/impls/jit_compiler.py", line 44, in func
    tutel_custom_kernel.invoke(inputs, __ctx__ * 256 + key)
RuntimeError: (0) == (return_code) INTERNAL ASSERT FAILED at "/private/home/namangoyal/src/tutel/tutel/custom/custom_kernel.cpp":95, please report a bug to PyTorch. return_codeCHECK_EQ fails.

I think, this could be due to some weird interaction of how Tutel compiles and loads custom cuda kernels and maybe spawned processes.
I tried Tutel's example also on multi gpus with pytorch distribuetd launch script, that seems to work. One difference I think is, pytorch launch script forks the new processes vs fairseq spawns them. Maybe thats creating a problem?
I also created a spawn version of Tutel's helloworld example, and even that doesnt work. I can share that if that can be helpful in debugging?

Thanks, @ngoyal2707 for reporting this issue. From tutel's team, we also use pytorch distribuetd launch to test this pull requests, and they are verified running well using both pytorch's ddp backend and fully-sharded backend.
So mostly likely it is an environmental issue. I see the assertion failure you encountered happens at: https://github.com/microsoft/tutel/blob/v0.1.x/tutel/custom/custom_kernel.cpp#L95
It indicates that the JIT compiler doesn't get a source code content that matches its expected format.
May I ask whether you change some source codes and install Tutel from source? If so, the changes may be not suitable and break the following procedure.
If you don't change anything or just install tutel via pip install https://.., can you help to show the standard output of the following shell commands in your environment?
(Command-1)
$ ls /tmp/*.cu

(Command-2)
$ cat /tmp/0-0.cu

pytorch distributed launch script works for me also, I am saying that fairseq's way of launching doesn't work for me.
Can you please share the cmd you used to test fairseq integration?

The aassertion fails here for me:

https://github.com/microsoft/tutel/blob/v0.1.x/tutel/custom/custom_kernel.cpp#L92

the trace says 95 cause I have bunch of debugging print statements in that code locally.

I tried installing without any changes and it still failed.

I do see all the *.cu files and the fatbins also in the tmp dir, they just dont load with spawned processes:

/tmp/0-0.cu  /tmp/1-0.cu  /tmp/10-0.cu  /tmp/12-0.cu  /tmp/14-0.cu  /tmp/16-0.cu  /tmp/18-0.cu  /tmp/2-0.cu  /tmp/20-0.cu  /tmp/22-0.cu  /tmp/24-0.cu  /tmp/26-0.cu  /tmp/28-0.cu  /tmp/3-0.cu  /tmp/30-0.cu  /tmp/32-0.cu  /tmp/34-0.cu  /tmp/36-0.cu  /tmp/5-0.cu  /tmp/7-0.cu  /tmp/9-0.cu
/tmp/0-1.cu  /tmp/1-1.cu  /tmp/11-0.cu  /tmp/13-0.cu  /tmp/15-0.cu  /tmp/17-0.cu  /tmp/19-0.cu  /tmp/2-1.cu  /tmp/21-0.cu  /tmp/23-0.cu  /tmp/25-0.cu  /tmp/27-0.cu  /tmp/29-0.cu  /tmp/3-1.cu  /tmp/31-0.cu  /tmp/33-0.cu  /tmp/35-0.cu  /tmp/4-0.cu   /tmp/6-0.cu
```

@ngoyal2707 The outputs make sense.
The format of these JIT codes are: <code_id>-<gpu-id>.cu, and they are created with different names to avoid being overwritten by different child processes.
Seems like the fairseq's launch doesn't deal with LOCAL_RANK at https://github.com/microsoft/tutel/blob/v0.1.x/tutel/impls/jit_compiler.py#L20.
So multiple child executors from fairseq's launch are conflicts writing into the same file target.
Can you have a quick try on change that line into random values for different executors to avoid the conflict? Those values would also be smaller than 256.
If that works, the root cause is found, and I'll think about how to fix it in a gentle way.

Ahhh perfect, now I got it! thanks

@ngoyal2707 BTW, may I ask what command you use to launch the model using fairseq's way? We want to see the runtime differences of that way and it would help us to come up with a gentle fix. Thanks!

@ghostplant thanks for the awesome stuffs in Tutel. Curious if you considered using NVRTC instead of invoking nvcc. I'm asking because in some environment with VMs and so on don't have access to run nvcc. I quickly tried and functions in gating .py can be compiled with NVRTC with minor tweak but the ones in sparse.py have issues associated with __half2 doesn't seem to be supported by NVRTC. Wonder if you have any experience.

@jspark1105 A good suggestion. In fact, our first version was based on NVRTC initially, but we soon find that same kernels compiled by NVRTC is less optimized than using NVCC. NVRTC's optimization options that NVIDIA exposes are very limited.
We create a temporary branch named jit_with_nvrtc: https://github.com/microsoft/tutel/tree/jit_with_nvrtc
If you are interested, you can install it from source and see whether the hello-world example is a little slower. (about 3% slower?)

Thanks so much @ghostplant. I tried the branch but not passing a simple correctness check I wrote. If I comment out "#define USE_NVRTC" it goes through. I'll dig more and share minimum reproducer but wonder if you have any guess.

@jspisak Not quite sure, Is your check script compatible with NVRTC?

@jspark1105 One possible guess: can you remove some fast-but-low-precision compiler option at: https://github.com/microsoft/tutel/blob/v0.1.x/tutel/custom/custom_kernel.cpp#L61
Like --use_fast_math, which is to use faster instruction to get similar but not precise result.

@ghostplant following up nvrtc, found removing --restrict option from option resolves the issue.

@jspark1105 Thanks, just a little weird since this keyword is expected to work the same way of __restrict__ and it works as expected in my environment. Do you know how this NVRTC's --restrict option impacts on your check correctness?


Where are config files?

Hi,
Thank you for this great extension.
Currently, there is no information about here are jupyter_tabnine config files. Could you add this information in the readme/doc ?
As a side note for contextual information: I try to check all conf files relative to autocompletion on jupyter to fix 2 issues

I've both tabnine suggestion and the jupyter ones ==> I would like to have only tabnine's suggestions,
the autocompletion trigger immediately/always   ==> I would like to have suggestion only after tiping at least 3 keys

Thank you
gz#6874

I did not solve my 2 issues, but I've found tabnine conf file (I wonder how I did miss them: /home/user_name/.config/TabNine


Alphahex

This has to be delayed until after #15

@tobiaskaminski I performed a control merge from master and restructured the way choices are stored a bit (this will come in handy on master as well). I'd like to ask you (again) to verify the setup in connection with Hex. I hope I did not break anything along the way ...
If your tests turn out well, I think we should merge this ASAP.
There's more interesting stuff to come in bridges.

OK. Nice to hear that comms with Hex are looking good!

The exception should not occur, but I cannot determine the source of the likely bug from just looking at the Hex program. The exception indicates that during learning, it resolved until the first UIP (which usually is assigning) but the NoGood was not assigning, hence the backjumping level could not be computed and it returned -1 instead. Is it possible to simulate the run just inside Alpha (i.e., which NoGoods were added by Hex)?

I'll try to come up with a reproduction for that.

I did a control merge and am now getting following trace for many tests.
java.lang.NullPointerException
	at at.ac.tuwien.kr.alpha.solver.ChoiceManager.backtrack(ChoiceManager.java:96)
	at at.ac.tuwien.kr.alpha.solver.DefaultSolver.doBacktrack(DefaultSolver.java:256)
	at at.ac.tuwien.kr.alpha.solver.DefaultSolver.tryAdvance(DefaultSolver.java:129)
	at at.ac.tuwien.kr.alpha.solver.AbstractSolver$1.tryAdvance(AbstractSolver.java:31)
	at java.util.Spliterator.forEachRemaining(Spliterator.java:326)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at at.ac.tuwien.kr.alpha.solver.Solver.collectSet(Solver.java:21)

@tobiaskaminski In order to get this branch stable, I resorted to reverting your commit 9ca29c0 (see the difference of test runs between 068f918 and 3944f8a). Now everything should work as expected, but the branch is missing this specific changeset. Let's merge this now and get your changes in separately, or do you prefer sorting that out before merging?

@AntoniusW , for the example program, Hex does not return any nogoods before assignUnassignedToFalse() is called, and after it has been called the first time always returns the nogoods {{"aux_n_10000000_2a(a,c)", "-a(c)"}, {"aux_n_10000000_2a(b,c)", "-b(c)"}}. The identical behavior can be simulated without Hex by replacing String[][] externalNogoods = getExternalNoGoods(assignment, atomStore); in line 18 of HexBridge.java with the following, in order to add the same nogoods as Hex would return:
String[][] externalNogoods1 = {{"aux_n_10000000_2a(a,c)", "-a(c)"}, {"aux_n_10000000_2a(b,c)", "-b(c)"}};
String[][] externalNogoods2 = {};
String[][] externalNogoods;
if(assignment.toString().equals("[T_d(c)@0]")) {
	externalNogoods = externalNogoods2;
} else {
	externalNogoods = externalNogoods1;
}

Then, this is the input program for Alpha:
d(c).
b(X) :- aux_r_10000000_2a(a,X), d(X).
a(X) :- aux_r_10000000_2a(b,X), d(X).
a(c) :- g.
b(c) :- g.

After 9ca29c0 has been undone now, the error does not occur anymore, but no answer sets are returned for this program (as expected). Besides that, I discovered a theoretical problem concerning atoms that are missing in the negative extension after assignUnassignedToFalse() is called, but which the external atom may depend on. Currently, I'm trying to find a theoretical solution. @flowlo , in the meantime we can merge since there is probably no quick fix for this.


Port and rework tests

Goal
Add tests.
Motivation
The proprietary predecessor of Trackteroid had good test coverage. We have to identify if those tests can be ported and potentially rework them, but obviously, we need tests.
Considerations
We want to consider going fully with pytest instead of unittest.

in test_session.py, the following tests are failing:
test_reconnect_and_commit
test_get_cached_collections
both are reading a database file that has uncommitted operations in it and try to apply it. If the operations contain the creation of an entity (as is the case), there is no way to get rid of them with the same process (delete is not implemented), so they leave garbage in ftrack which will also fail the test the next time it is run (duplicate entry).
Since the database write/read feature seems incomplete I would assume it is barely used if at all. I started a discussion, and based on its outcome, either remove the functionality together with its tests or implement the missing feature.

Tests were failing because of modifications done in branch fix/i10-pytests. Reverting them resolved the confusion. All tests are working now.


proxy: Provide identities { host | world | cluster } in SourceEndpoint

Fixes: #1616
Provide identities { host | world | cluster } in SourceEndpoint.
This change also fixes a bug in fillIngressSourceInfo which was not filling the IP address correctly.
Signed-off-by: Manali Bhutiyani<EMAIL_ADDRESS>proxy: Provide identities { host | world | cluster } in SourceEndpoint

@tgraf @raybejjani : Addresses review comments. Pleas take a look.


Worker timeout when using https

Hi,
I got the server running. Everything works fine (creating users/uploading and finding blocks and stuff), but only if I use BII_SSL_ENABLED=False. When I use BII_SSL_ENABLED=1 I always get an error:
ERROR: Unable to connect to server to find deps

When I enter the URL (https://localhost:9020) in my webbrowser, it doesn't work either, but I get a worker timeout after some time.
Do I need to create a certificate or something like that?
Here is the log (debugging mode):
01:48:19 web.1  | started with pid 43670
01:48:19 web.1  | 2015-06-30 01:48:19 [43670] [INFO] Starting gunicorn 0.17.2
01:48:19 web.1  | 2015-06-30 01:48:19 [43670] [INFO] Listening at: http://<IP_ADDRESS>:9020 (43670)
01:48:19 web.1  | 2015-06-30 01:48:19 [43670] [INFO] Using worker: sync
01:48:19 web.1  | 2015-06-30 01:48:19 [43677] [INFO] Booting worker with pid: 43677
01:48:19 web.1  | INFO  :api_v1.py      [65]: Installing BiiReturnHandlerPlugin plugin... [2015-06-30 01:48:19,432]
01:48:19 web.1  | INFO  :api_v1.py      [71]: Installing NonSSLBlockerBottlePlugin plugin... [2015-06-30 01:48:19,432]
01:48:19 web.1  | INFO  :api_v1.py      [91]: Installing http basic authentication plugin... [2015-06-30 01:48:19,432]
01:48:19 web.1  | INFO  :api_v1.py      [96]: Installing JWT authentication plugin... [2015-06-30 01:48:19,432]
01:48:19 web.1  | INFO  :api_v1.py      [116]: Installing bson plugin... [2015-06-30 01:48:19,432]
01:48:19 web.1  | INFO  :api_v1.py      [122]: Installing BiiUserTraceBottlePlugin plugin... [2015-06-30 01:48:19,432]
01:49:45 web.1  | 2015-06-30 01:49:45 [43670] [CRITICAL] WORKER TIMEOUT (pid:43677)
01:49:45 web.1  | 2015-06-30 01:49:45 [43670] [CRITICAL] WORKER TIMEOUT (pid:43677)
01:49:45 web.1  | 2015-06-30 01:49:45 [43702] [INFO] Booting worker with pid: 43702
01:49:45 web.1  | INFO  :api_v1.py      [65]: Installing BiiReturnHandlerPlugin plugin... [2015-06-30 01:49:45,603]
01:49:45 web.1  | INFO  :api_v1.py      [71]: Installing NonSSLBlockerBottlePlugin plugin... [2015-06-30 01:49:45,604]
01:49:45 web.1  | INFO  :api_v1.py      [91]: Installing http basic authentication plugin... [2015-06-30 01:49:45,604]
01:49:45 web.1  | INFO  :api_v1.py      [96]: Installing JWT authentication plugin... [2015-06-30 01:49:45,605]
01:49:45 web.1  | INFO  :api_v1.py      [116]: Installing bson plugin... [2015-06-30 01:49:45,605]
01:49:45 web.1  | INFO  :api_v1.py      [122]: Installing BiiUserTraceBottlePlugin plugin... [2015-06-30 01:49:45,606]

Biicode server is just an application server and doesn't handle HTTPS by itself.
BII_SSL_ENABLED is used for install or not a plugin that forces redirect to https if a http request is received.
You can use nginx to handle https and proxy the port to the biicode port. Its a common use case in server infrastructures, a "router" server handles secure connections and TSL certificates and behind the wall, the local network is opened.
server {
    listen 443;
    server_name localhost;
    location / {
       proxy_pass http://localhost:9020/;
    }
    ssl on;
    ssl_certificate /etc/nginx/ssl/server.crt;
    ssl_certificate_key /etc/nginx/ssl/server.key;
}

About certificates, yes, you will need to generate a crt and key files.


Eliminate erroneous comment and strive for consistency

Rationale
Comments with lies are not desirable

@labkey-jeckels I added you to a chat group about the dependencies failure. Unrelated, but unfortunately it needs to be fixed before this very important PR can be merged.


Concat method should pass the pDestObj when we take the concat path using bigindex

fixes #3603.
It takes lot of time of test finish the test so I haven't added that.
@suwc @aneeshdk please have a look at this one.


json -> ujson.

There are various benchmarks that show ujson beats json in performance. I wrote a very simple one here and posted the results there, it also showed ujson outperforming json. The tradeoff here is that ujson does not support indent and sort_keys functionality (for pretty-printing) but I think aiohttp should not worry about that since users can pretty-print  if they want.

Hi,
In API-Hour plugin for aiohttp, I use ujson if available: https://github.com/Eyepea/API-Hour/blob/master/api_hour/plugins/aiohttp/init.py#L4
Nevertheless, the usage of ujson should be optional, because you can have issues to install ujson, json is included in Python.
Personally, with the override mechanism parameter, I've enough to customize the behaviour.
But, it should be a good idea to add a page in aiohttp documentation with some tips&tricks to improve performances with aiohttp.

-1
First, you'd forgot to add ujson into project dependencies.
Second, Windows users have to install it additionally.
Third, it brings completely different from stdlib json behaviour for happy debugging hours:
>>> import ujson
>>> ujson.encode(object())
{}
>>> ujson.encode(...)
{}

I'm +1 to make JSON library configurable at project-wide level, however. For instance, couchdb-python has a solution to manage custom json lib on fly. Then users will pick the implementation their like without passing additional parameters and patching the aiohttp.

@kxepal: Personally, I'm not a big fan to use global environment variable, this type of mechanism shot my foot several types on production because if you've forgotten this variable for X reasons, you have different behaviour between your VMs of prod, preprod, staging and dev: It's a nightmare to track this type of bugs. The developer has tested with another library than on production.
I'm more partisan to declare that explicitly by the developer, because you'll have a stacktrace directly at the startup of the daemon if the library used by the developer is missed.
Moreover, if in the same application you need to use another lib, because ujson can't transform data, you'll be happy to change that on runtime. I'd already have this issue with ujson, it isn't the silver bullet to replace json: you don't have always the same behaviour between both libraries.
With @asvetlov, during PyCON sprint code, we made something like that for aiopg to support psycopg2 and psycopg2cffi: https://github.com/Eyepea/aiopg/blob/psycopg2cffi_2nd_attempt/aiopg/connection.py#L42
Nevertheless, we may not need that, we should only add a parameter to Application to define default JSON loader globally, instead of by request, like we have now.

@GMLudo If you references to couchdb-python implementation, then environ variable is only one of the way to switch json module. It's also provides functional like json.use("simplejson").
However, I don't propose to take it as is, but to borrow the underlying idea. That idea is about to some proxy json module that is able to manage exact implementation behind it. How this would be implemented for aiohttp - that's a second sort question.

I'm with @kxepal
ujson should be optional if any.
Exact api is a question. Please keep in mind we use json library not only for server but for client side also.

@kxepal: Thanks for the tip, I thought it was possible to choose Python library only via global environment.

-1 for dependency, but I like @kxepal approach


How can I do disable  GA4 feature for executing error

I m httprunner1.0 -> 2.0 user. so update to 4.0 for go
catch a few questions and bolck the autotest run in my origination
requirement 2:
alse run testcase with hrp,  encounter this GA4 error
hrp startproject demop
{"level":"info","time":"2023-09-27T14:02:39.794+0800","message":"log with colorized console"}
......
2023-09-27T14:02:40.251+08:00 ERR send GA4 event failed error="request GA4 failed: Post "https://www.google-analytics.com/mp/collect?api_secret=w7lKNQIrQsKNS4ikgMPp0Q&measurement_id=G-9KHR3VC2LN\": dial tcp [::]:443: connect: connection refused"
2023-09-27T14:02:40.251+08:00 INF create scaffold success projectName=demop
env:
mac pro os M1
requirement 2:
Now run hrp without logs,  Could u pls get out hrun logs to direct folder when run hrp
Already，catch runcase data from log for mail

from resoue code get "self.__is_ci==os.getenv"
so close requirement 1

also, another error "hc-grpc-py: plugin process exited: path=~/.hrp/venv/bin/python3 pid=7719 error="signal: killed"
so.. need uprade grpc packages to fix this problem? ths


Build Failure Due to Import Misuse in framesjs-starter

The framesjs-starter boilerplate fails to build in the most recent version. This is caused by a problem in the framesjs package when running next build. There seems to be a misuse when protobufjs/minimal is imported without a [.js](https://nodejs.org/api/esm.html#mandatory-file-extensions) extension. This seems to be a similar issue https://github.com/vercel/next.js/issues/31974.
Temporary workarounds:

Add frames.js to next config's transpilePackages option

Error
Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/frames/node_modules/protobufjs/minimal' imported from /frames/node_modules/frames.js/dist/chunk-5PU7OFBJ.js

Did you mean to import protobufjs/minimal.js?
    at new NodeError (node:internal/errors:405:5)
    at finalizeResolution (node:internal/modules/esm/resolve:324:11)
    at moduleResolve (node:internal/modules/esm/resolve:943:10)
    at defaultResolve (node:internal/modules/esm/resolve:1129:11)
    at nextResolve (node:internal/modules/esm/loader:163:28)
    at ESMLoader.resolve (node:internal/modules/esm/loader:835:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:424:18)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:77:40)
    at link (node:internal/modules/esm/module_job:76:36) {
  code: 'ERR_MODULE_NOT_FOUND'```

Thanks for your thorough evaluation of the issue. Looks like it's cause by the pages-router example which was recently added. If you're not using pages router, removing the pages directory should help it build in the meantime.
We're looking into a fix

Closed by #292

@cashd Please update to the latest frames.js version and follow the guide here for instructions on how to integrate with pages router. Thanks!


Apache DolphinScheduler deployment for Amazon EKS

Deploy a distributed and highly reliable architecture of Apache DolphinScheduler on Amazon Elastic Kubernetes Service (EKS) using Terraform.

Purpose of the pull request

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:

(or)
Pull Request Notice
Pull Request Notice
If your pull request contain incompatible change, you should also add it to docs/docs/en/guide/upgrede/incompatible.md

DS is an open source component, like Flink and Doris. We only want to support open source k8s. Each public cloud has its own customized implementation of k8s, and we don't provide deployment solutions for all of them. These should be implemented by the public cloud itself. I will close this PR, if you insist on your point of view, welcome to open an issue for discussion first. @SEZ9

DS is an open source component, like Flink and Doris. We only want to support open source k8s. Each public cloud has its own customized implementation of k8s, and we don't provide deployment solutions for all of them. These should be implemented by the public cloud itself. I will close this PR, if you insist on your point of view, welcome to open an issue for discussion first. @SEZ9

Thank you for the suggestion, but I still have a different perspective and hope you can reconsider. I have raised an issue here: https://github.com/apache/dolphinscheduler/issues/16434. @SbloodyS

Looks like features this PR introduces include two parts:

Support setting up EKS related stuff with terraform.
Deploy dolphin to EKS with helm chart.

For the first part, is there any way you could keep those terraform related stuff tested against each future release? Since few maintainers in this community are familiar with terraform and have free access to AWS resources at the same time, it is not possible for the community to maintain the code.
For the second part, this PR introduces too many repeated code. As said in this comment https://github.com/apache/dolphinscheduler/issues/16434#issuecomment-2280273926 , if EKS fully compatible with K8S, there is no need to repeat this code. In the future, with other cloud providers, it could be a disaster to maintain these repeated and probably inconsistent code.
Above all, if you could offer a mechanism to keep the terraform related code well-tested against each future release, I'm ok with the first part. For the second part, I don't think it is appropriate to generate repeated code.

@EricGao888  Thank you for your response. Your analysis is correct. This feature is composed of two parts: one is Terraform code, and the other is a Helm chart. Given the unique requirements of cloud service deployment, users need to use Terraform to provision resources suitable for DolphinScheduler and replace resource-related configurations in the Helm scripts.
Part One: Although most of the Helm chart is repetitive, variables have been replaced, and some critical configurations have been exposed for public cloud settings. This part will need to be maintained and updated according to the specific version of the DolphinScheduler Helm chart.
Part Two: As you mentioned, the Terraform script needs to be tested on the cloud. On the AWS side, we will provide dedicated one-time AWS accounts for DolphinScheduler users to conduct testing and releases. I can take responsibility for maintaining these test versions and promptly updating the DolphinScheduler documentation and code.
For this feature, I can also add a DolphinScheduler document explaining how to obtain a test account, perform the deployment, and update it with major releases. Would this be sufficient to proceed with merging the PR?


Notifications references

Section Number *3.2.4 *
Issue the text talks about push and pull style for notifcations and in the end of the section there is a reference to DSUB. Bu this reference points just to normative text, thus only for Push-style.
Proposed Change add a reference for DSUB extension where it is explained the pull-style notification
Priority:medium

Can you suggest specific text? I don't know DSUB.

i suggest to replace the two bullet points:

Using a push-style method of notification where the Subscriber subscribes to receive notifications about the availability of documents based on specific criteria (see ITI TF Vol 1 )
or

Using a pull-style method which creates a Pull Point resource able to store notifications targeted to a specific recipient. Notifications stored in the Pull Point can be retrieved using a specific transaction (see ITI TI Extensions to Document Metadata Subscription ).

note that the Pubblicaton and Discovery slide deck talks just about push style notification

I think we only have DSUB now days. I suspect the text is a left over from the NAV days. We no longer have NAV, so that should be cleaned up.

updated the text to use the actor names. The link to DSUB needs to be just to the DSUB. We can't include a direct link to the supplement or will need to revise the whitepaper when that supplement becomes final. But the DSUB profile pointed to does reference the extension supplement. When the reader does not find the pull point actors, they should be able to understand to go to the supplement.


learning_rate in boosting

Hi, is there a way for setting the learning_rate in the boosting regressors and classifiers?
EDIT:
Also, is LinearBoostingRegressor fitting a linear regression first and then boosting the residual via regression trees or boosting via a series of linear regression trees?

Hi, thanks for your feedback.
No, here learning_rate is not involved.
Linear Boosting is a two-stage learning process. Firstly, a linear model is trained on the initial dataset to obtain predictions. Secondly, the residuals of the previous step are modeled with a decision tree using all the available features. The tree identifies the path leading to the highest error (i.e. the worst leaf). The leaf contributing to the error the most is used to generate a new binary feature to be used in the first stage.
In this sense, I think the code is quite explicative.
If you support the project don't forget to leave a star ;-)


Release 22.11.1

I think we should do a new release for 3.11 support. I can help if needed

I think this can wait ..


Add fixer selection options

Implements 3 new flags as discussed on Issue#278. Influenced by davidszotten's previous PR#333.

New flag --skip <fixer_name> will skip the selected fixer from running.
New flag --only <fixer_name> will only run the selected fixer.
Both of the above are repeatable by chaining the flags
New flag --list-fixers will list all available fixers.
Documentation and Unit test updates

Fixes #278

🙏 Thank you for your work here.
I rebased and made changes until it looked good. The main code changes were to lean on argparse more and to make FIXERS a dictionary. I also tweaked the docs including making #449 to pre-document the existing options. See the individual commits for details.


CRLF

Am I not correct in saying that a return should be counted as 2 characters? I have a fix for this if you like, but I just wanted to check this hadn't been deliberately omitted for any reason.

Hi @acurrieclark, it is counted as 2 characters if the text contains both CR and LF:
SmsCounter.count("\r\n").length // 2

But only one if there's just a LF:
SmsCounter.count("\n").length // 1

It depends on what your text input element actually inserts when you hit return. It may vary between browsers and OSes.

@danxexe +1. Why not close this issue ?

What I am really suggesting is that an SMS will always (to my understanding) use a /r/n and therefore 2 characters. In that case, regardless of whether or not the browser is using /n or /r/n, it should always be counted as 2 characters.
__
Alex Currie-Clark
Tel: 07825 346673
On Tue, May 5, 2015 at 6:59 AM, Dhruba Adhikari<EMAIL_ADDRESS>wrote:

@danxexe +1. Why not close this issue ?
Reply to this email directly or view it on GitHub:
https://github.com/danxexe/sms-counter/issues/9#issuecomment-98964089

I didn't find any specification stating that SMS messages must use \r\n, and in practice it seems several systems do send messages with just \n and most clients seem to work fine with that. I don't see any reason to limit the user and display an incorrect value if the user wishes to use just a \n.
You could always use something like message.replace(/\r?\n/g, "\r\n") if you wish to ensure you are always sending both characters.


Update to bevy 0.11

Haven't tested yet outside of the examples but it seems to work as expected

Thanks! Addressed in #8.


Test case issue_142 fails for VHPI

The test fails because of the following code:
m_value.numElems = value.length();
The signal that is being written to is 64 bits.  The test writes a BinaryValue() that only has 55 bits and the vhpi interface does not like the numElems being less than 64 bits.
The first part of this fix is to do the following:
m_value.numElems = m_num_elems;
This solution brings up another issue though.  There is code that tries to "pad" and code that will "truncate" a string value that doesn't match the length of the handle.
The VHPI code currently pads after all values written, but given the value is descending it would need to put the pad in first to get the correct value out.
We really don't have enough information to be padding or truncating, when you start considering the underlying data might be signed which would require sign extension, etc.
@chiggs and @stuarthodgson
I am going to do a solution here that does the following:

Remove Pad/Truncate functionality from VPI, VHPI, and FLI
Add a check that will LOG_ERROR and return if the length of the value does not match the length of handle
Update the test to not create a BinaryValue() that has the incorrect length.

Would you agree with this solution?

Tested in the latest and works as expected.  Closing.


Peek one level to handle common symbol link starting point collisions

Bug/issue #, if applicable: rdar://108666088
Summary
Provide a description of what your PR addresses, explaining the expected user experience.
Also, provide an overview of your implementation.
Dependencies
Describe any dependencies this PR might have, such as an associated branch in another repository.
Testing
Describe how a reviewer can test the functionality of your PR. Provide test content to test with if
applicable.
Steps:

Provide setup instructions.
Explain in detail how the functionality can be tested.

Checklist
Make sure you check off the following items. If they cannot be completed, provide a reason.

[ ] Added tests
[ ] Ran the ./bin/test script and it succeeded
[ ] Updated documentation if necessary

@swift-ci please test

This simplified approach is too simple. I'll open a new PR when I have a better solution to this.


[Idea] Show Chapters

This is a nice-to-have and not a must. Since you already have timestamps in the description, it would be amazing if the audio file contained chapters.
Tools

https://overcast.fm/forecast (I thought @wesbos said he used this app)
https://chaptersapp.com/

Resources

https://medium.com/podcast-101/podcast-chapters-explained-bed72832640c

This will be in the new app.

@stolinski, the audio file you put on your RSS already has chapters. Thanks for doing that a few months back!


Browser normalization for scroll zoom

Your plugin works great. However, the step property does not normalize to the variances between browsers. Chrome and Firefox step large while others step small. See stackoverflow issue and github issue

Thanks for opening an issue. But those issues are pretty old and a lot has change since then. For instance, Opera and Edge are now Blink browsers and work the same as Chrome. This makes me wonder if any of those solutions still work. I also wonder if this is really an issue anymore. For instance, I don't really see a difference between Firefox, Safari, and Chrome in the mousewheel example on the demo page.
That said, it would be very helpful if you could provide a small test showing different deltas across browsers on the wheel event. I could then use that in Panzoom tests and work off of the old facebook solution to normalize deltas, but this issue needs a test case to go further. Thanks.

Hi Timmy,
Thanks for responding so quickly. I just got a chance to test the functionality on all my environments. It looks like the problem is with windows versions of Chrome and Firefox. Otherwise, everything else seems fine with your plugin. Here are the results of the tests. I’m using fairly large SVG maps inline. The zoom in the Chrome and Firefox windows versions seems to step closer to that 120 value instead of the 3 value for all the other browser versions. I program in WSL, but my browsers are all in windows so I didn’t test linux browsers.
Windows OS (20H2 build 19042.928):
Closer to a step of 120 with my scroll wheel:
Chrome: Version 90.0.4430.85 (64-bit)
Firefox: Version 88.0 (64-bit)
Closer to a step of 3 with my scroll wheel:
Edge: Version 90.0.818.42 (64-bit)
Opera: Version:75.0.3969.218
Dissenter: Version: 1.5.114
macOS Catalina (Version 10.15.7):
Closer to a step of 3 with my scroll wheel:
Safari: Version 13.1.3 (15609.4.1)
Chrome: Version 90.0.4430.85 (x86_64)
Firefox: Version 88.0 (64-bit)
Opera: Version 75.0.3969.218 (x86_64)
Dissenter: Version: 0.70.122 Chromium 78.0.3904.87 (64 bit)
Android (Version 11 build RQ2A.210405.005):
Pinch zoom works fine:
Chrome: Version 90.0.4430.66
Opera Touch: Version 2.9.5
Firefox: Version 88.1.1 (build #2015804891) – slower probably due to file size not your plugin
DuckDuckGo mobile browser: Version 5.80.0
IPhone 8 Version 14.1:
Pinch zoom works fine:
Safari version that goes with this phone
Firefox Daylight 31.0 (3537) – slower and crashy probably due to svg file size not your plugin
Chrome version 87.0.4280.77 – slower and crashy probably due to svg file size not your plugin
Thanks for the work and the use of your plugin.
Trent
From: Timmy Willison @.>
Sent: Wednesday, April 21, 2021 3:27 PM
To: timmywil/panzoom @.>
Cc: tlarsonlgdor @.>; Author @.>
Subject: Re: [timmywil/panzoom] Browser normalization for scroll zoom (#560)
Thanks for opening an issue. But those issues are pretty old and a lot has change since then. For instance, Opera and Edge are now Blink browsers and work the same as Chrome. This makes me wonder if any of those solutions still work. I also wonder if this is really an issue anymore. For instance, I don't really see a difference between Firefox, Safari, and Chrome in the mousewheel example on the demo page https://timmywil.com/panzoom/demo/ .
That said, it would be very helpful if you could provide a small test showing different deltas across browsers on the wheel event. I could then use that in Panzoom tests and work off of the old facebook solution to normalize deltas, but this issue needs a test case to go further. Thanks.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/timmywil/panzoom/issues/560#issuecomment-824301772 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AJAGZAG2CHPVICPDTQYH67TTJ4RJBANCNFSM43K2VK3Q .  https://github.com/notifications/beacon/AJAGZADC4SIEBY4XSFRVHFDTJ4RJBA5CNFSM43K2VK32YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOGEQ5RTA.gif

Okay thanks.  I'll have a look at the deltas on Windows when I get a chance.

I’m trying to work around using react-device-detect, but the only way to distinguish the difference between Chrome and opera, brave, and dissenter is their engine version. All of them use Blink like you said, but for some reason Chrome is a larger step while the others are smaller steps even though they are earlier Blink versions. I just installed Brave on windows and it is almost the same Blink version as Chrome, but it steps great with your default setting of 0.3 (same for opera and dissenter). While I have to use 0.01 for Chrome or lower to prevent stepping too far with a scroll wheel tick. I’m using logi roller ball. This work around is fine for me right now. For Firefox, I’m using a step of 0.01, but there seems to be some centering issues or random jumps when I scroll. Only other option I set was animate and it is false.
From: Timmy Willison @.>
Sent: Wednesday, April 21, 2021 5:20 PM
To: timmywil/panzoom @.>
Cc: tlarsonlgdor @.>; Author @.>
Subject: Re: [timmywil/panzoom] Browser normalization for scroll zoom (#560)
Okay thanks.  I'll have a look at the deltas on Windows when I get a chance.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/timmywil/panzoom/issues/560#issuecomment-824364801 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AJAGZADRWHCG2YN7XZZ32BDTJ46QLANCNFSM43K2VK3Q .  https://github.com/notifications/beacon/AJAGZAHTL7ZPL3AAPHDXNHTTJ46QLA5CNFSM43K2VK32YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOGERM6AI.gif

Interesting. I suppose it could have something to do with the underlying Blink version. Hopefully the browsers will align as time goes on.

Too many inconsistencies across browsers, even today. Related web spec issue:

https://github.com/w3c/uievents/issues/181


Add options checking based on phoenetic spellings

Similar to the implementation in the spark-cef reader, implement a phoenetic check against the user supplied options to provide useful error messages if they have been spelt incorrectly

Closed with merge of pull request #3


Query Granular Certificates

Description
As a user I want to be able to query the wallet and get a response of all the Granular Certificates it contains.
Acceptance criteria

User should be identified by the sub field on the JWT.

Scope limit

No filter on the queries at this point in time.
No pagination at this point in time.

Solution plan

[ ] Design and document functionality
[ ] Document Sequence diagram
[ ] Write Protobuf scheme
[ ] Implement code
[ ] Write test

fixed by #23


Unable to build spark docker image

While building spark, using below commnd
docker build -t molab/spark-yarn spark
I got this error:
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

Hey, thanks for raising an issue!
The repo hasn't been touched for a couple of years and we have stopped using it. Therefore we can't really provide you with any support.
Sorry!

Can you point me to any repo where I can find spark cluster running in yarn mode? Thanks in advance.

I'm afraid not. We are currently focusing our attention on dask.


Move test location of ARR back to westus2, because service is not supported in westus

Description
The default test location changed a while ago to westus instead of westus2. Unfortunately the Azure Remote Rendering service is not supported in westus and thus all tests are failing. This PR specifies westus2 for the location of the tests of the ARR test.

/azp run

/azp run python - remoterendering - tests

API change check
API changes are not detected in this pull request.

/check-enforcer evaluate

/check-enforcer evaluate


Update rotation/translation matrix

The matrices returned by https://activebrainatlas.ucsd.edu/activebrainatlas/alignatlas?animal=junk should match https://activebrainatlas.ucsd.edu/activebrainatlas/alignatlas?animal=DK39
Please test and implement.

Hi Ed,
It works pretty fine on my end.
Just need to confirm one thing before I push my commits: Should we only consider precomputed://https://activebrainatlas.ucsd.edu/data/DK39/neuroglancer_data/atlas_v7 as atlas, or all the URLs with the format precomputed://https://activebrainatlas.ucsd.edu/data/[SOME_BRAIN]/neuroglancer_data/atlas[*]

Junjie,
We should only be using the generic first one you mention in the comment above. I started creating those other ones under each brain but they are not necessary at the moment.

This has been dealt with and we are moving onto another issue.


Progress bar is not working

Hi, Progress bar does not seem to work where xhr request is sent, progress bar should also move for that file..

I've fixed the progress bar issue something like this
div.somecontainer[rel=pos] -> find uppbr ( was not working )
i added pos (position variable in our case i) to .uppbr
and replace the lines where you was searching for div.somecontainer[rel=post] -> find bar
to
.uppbr[rel=pos]

Hi thanks for reporting the bug and for your interest in this plugin, please check the latest version 2c9f3bafb89247ef20f9b911a3b9f49554d7d57b and see if this problem persist. Thanks!


Consistent table headers on submissions tab and dashboard tab

Is your feature request related to a problem? Please describe.
The "My dashboard" (/apply/dashboard/) page has a table that lists past reviews.
The "Submissions" (/apply/submissions/) page has a very similar table (the "Role" field is removed)
The content is almost the same, but the order of the columns is different.  One of our users requested that the order be consistent between the two interfaces to create a more predictable user experience.  I wanted to raise the issue in case this would be of interest to the Hypha team as well.

Describe the solution you'd like
The same order of columns between both tables.
Describe alternatives you've considered
The alternative would be to leave it as is, or make the column for each table order completely customizable by the local instance.
Additional context
I'd be happy to open a PR with this for the team's consideration.
Priority

Low priority (annoying, would be nice to not see)

Affected roles

Staff
Reviewers

Ideal deadline
None.

Good to make them look the same, as much as possible.

@slifty Thank you for flagging this. At the moment we will select one column order and ensure this design is consistent throughout the platform.

Would love this UX bug fixed :) we'll happily receive that PR

This is super small (XXS) and I do have the local fix (we actually diverged which columns to render on both these tables in our local instance -- adding ID and removing one of the columns).
Ultimately it comes down to:

One of the table explicitly specifies the sequence of ALL columns.
The other uses the Django table's "..." magic sequence key to specify sequence.  Unfortunately the field order Django calculates doesn't match the specified sequence.

The solution is to explicitly specify the order on both.
I'll open that PR shortly.


Adds new character_utilization_ratio function to check if a single

Adds new character_utilization_ratio function to check if a single character is over used in a post.
Initially only active on judaism.stackexchange.com

Codecov Report

Merging #821 into master will increase coverage by 0.29%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master    #821      +/-   ##
=========================================
+ Coverage   63.51%   63.8%   +0.29%
=========================================
  Files           6       6
  Lines        1732    1746      +14
=========================================
+ Hits         1100    1114      +14
  Misses        632     632

Impacted Files
Coverage Δ

findspam.py
82.38% <100%> (+0.66%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d045ea1...3b62ae0. Read the comment docs.

If we aren't using the codeclimate checks, can we remove it from CI?

!!/aprove

!!/approve

Ignoring the CodeClimate thing for now. Need to figure out what we're doing with it.


Fixed #24927 -- Remove duplicate function.

python_2_unicode_compatible is now included in six, so include that function from inside django.utils.encoding instead. https://code.djangoproject.com/ticket/24927
I wondered if the encoding should be deprecated and the code should all be updated to refer to it in the new location, but I thought that perhaps six may change its behaviour at some point in future and you'd want to be able to switch back to a Django specific version. (See e.g. discussion of the six PR at https://bitbucket.org/gutworth/six/pull-request/48 for things six might do in future.)

If we're going to use the method from six, seems we should to deprecate import from django utils, and replace imports inside the django code to use six import instead of utils.

I think it may be worth adding a comment about the reason for the import-shim line. Otherwise this LGTM.

Personally, I don't find much value in the documentation changes. IMO, this will just cause confusion and cause people to wonder which way to use when there is nothing wrong with using the existing style. Changing code may cause needless merge conflicts, etc.

Merge code change in c91bc68e9a5f9f1e2cce00d01d62d16a858155f0.

Great :) As I couldn't get the docs to pass because there is both module: django.utils.six and module:: django.utils.six in python3.txt and it looked to be harder than I would like to make the references work…

Thanks very much @dracos for the report and fix! And for the extra work on docs, even though it wasn't merged in the end :-)


tensorflow raises exception when importing

Work Around
https://github.com/Miserlou/Zappa/issues/779#issuecomment-292672317
Context
I'm not sure if this is a Zappa bug but since it involves tensorflow, I hope it would be of interest to others at least.
Expected Behavior
Tensorflow import without error.
Actual Behavior
Traceback:
Traceback (most recent call last):
  File "/var/task/handler.py", line 508, in lambda_handler
  return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 239, in lambda_handler
  handler = cls()
  File "/var/task/handler.py", line 128, in __init__
  self.app_module = importlib.import_module(self.settings.APP_MODULE)
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
  __import__(name)
  File "/tmp/zappaserv/app.py", line 9, in <module>
  import tensorflow as tf
  File "/tmp/pip-build-bxV1IK/tensorflow/tensorflow-1.0.1.data/purelib/tensorflow/__init__.py", line 24, in <module>
  File "/tmp/pip-build-bxV1IK/tensorflow/tensorflow-1.0.1.data/purelib/tensorflow/python/__init__.py", line 124, in <module>
  File "/tmp/pip-build-bxV1IK/tensorflow/tensorflow-1.0.1.data/purelib/tensorflow/python/platform/test.py", line 50, in <module>
  File "/tmp/pip-build-c1VilV/mock/mock/__init__.py", line 2, in <module>
  File "/tmp/pip-build-c1VilV/mock/mock/mock.py", line 71, in <module>
  File "/tmp/pip-build-c1VilV/pbr/pbr/version.py", line 460, in semantic_version
  File "/tmp/pip-build-c1VilV/pbr/pbr/version.py", line 447, in _get_version_from_pkg_resources
  File "/tmp/pip-build-c1VilV/pbr/pbr/packaging.py", line 750, in get_version
Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name mock was given, but was not able to be found.

Possible Fix
Unsure
Steps to Reproduce
app.py
import os
from flask import Flask
app = Flask(__name__)
app.debug = True

import tensorflow as tf
import numpy as np

@app.route('/')
def index():
    print(np.__version__)
    y = np.ones(5)
    return "Hello, world!" + str(y), 200

# We only need this for local development.
if __name__ == '__main__':
    app.run()

settings.json
{
    "dev": {
        "app_function": "app.app",
        "s3_bucket": "zappa-qxuea5nia",
        "memory_size": 1024,
        "timeout_seconds": 300,
        "debug": true,
        "use_precompiled_packages": false,
        "slim_handler": true,
        "keep_warm": false
    }
}

Your Environment

Zappa version used:
Operating System and Python version:

Ubuntu 14.04.5 LTS
Python 2.7.13

pip freeze
appdirs==1.4.3
argcomplete==1.8.2
base58==0.2.4
boto3==1.4.4
botocore==1.5.31
click==6.7
docutils==0.13.1
durationpy==0.3
Flask==0.12.1
funcsigs==1.0.2
futures==3.0.5
hjson==2.0.2
itsdangerous==0.24
Jinja2==2.9.6
jmespath==0.9.2
kappa==0.6.0
lambda-packages==0.14.2
MarkupSafe==1.0
mock==2.0.0
numpy==1.12.1
packaging==16.8
pbr==2.0.0
placebo==0.8.1
protobuf==3.2.0
pyparsing==2.2.0
python-dateutil==2.6.0
python-slugify==1.2.1
PyYAML==3.12
requests==2.13.0
s3transfer==0.1.10
six==1.10.0
tensorflow==1.0.1
toml==0.9.2
tqdm==4.11.2
troposphere==1.9.2
Unidecode==0.4.20
Werkzeug==0.12
wsgi-request-logger==0.4.6
zappa==0.40.0

Thanks for reporting this, definitely need this to be supported.
Surely related to: https://github.com/Miserlou/lambda-packages/issues/31

Still trying to root cause this, but have narrowed it down to a problem with the mock package in combination with slim_handler: true. It seems it can't figure out the version info in this case.

Ahhhh, in that case, also related to:
https://github.com/Miserlou/Zappa/issues/755
https://github.com/Miserlou/Zappa/issues/776

There might be some relevant info with the following issues. At it's core. This line of code
_v = VersionInfo('mock').semantic_version()

fails, in particular, the call to semantic_version().
https://github.com/testing-cabal/mock/search?q=semantic_version&type=Issues&utf8=✓

Well considering that all pbr is doing for mock is returning its version info I think I'm going to stop trying to figure this one out and just replace those lines of code above with some constants. I don't know enough about python packaging to effectively debug this.
Work Around
Replace https://github.com/testing-cabal/mock/blob/master/mock/mock.py#L71-L73 with
__version__ = '2.0.0'
version_info = (2, 0, 0, 'final', 0)

Now tensorflow runs!

Good bug hunting! Looks like something funky is going on upstream - but where? Do you want to file related an upstream ticket for this?

Yea I was thinking about what I could tell them, without them pointing a finger back saying what does "slim_handler" do. I'm looking at this issue which appears similar.
Obviously I'd rather not patch mock everytime I deploy a new tensorflow service.

Upstream responded saying that the dist-info metadata is missing. They also pointed to this issue as reference.

That should be easy to patch by changing the ZIP_EXCLUDES, want to send a lil' PR? :)

Sure I can give it a shot :)

It seems that the dist-info is already being kept in the zipfile. I'm thinking the problem is that it can't find it. I'll probably have to understand https://github.com/testing-cabal/mock/issues/385 to make progress here.

Okay. I'd also make sure that it isn't related to https://github.com/Miserlou/Zappa/issues/755#issuecomment-293590537 and the related discussion on Slack w/ @mcrowson

It's probably worth pointing out that this issue doesn't exist in python 3. I suppose since it uses the mock functionality provided by unittest.
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/pip_package/setup.py#L54-L57


[WIP] Jasmine thesis branch for easy comparison

No, I did this to deal with a memory issue but it doesn’t make a big
difference, and this pull request won’t be merged I just made it to compare
and see why things werent working.
Eventually it would probably be good to merge in the Monte Carlo sample
batching, but that would be in a separate cleaner PR
On Thu, Jun 17, 2021 at 2:27 PM tachukao @.***> wrote:

@.**** commented on this pull request.
In mgplvm/fast_utils/toeplitz.py
https://github.com/tachukao/mgplvm-pytorch/pull/65#discussion_r653563119
:

@@ -174,8 +175,19 @@ def toeplitz_matmul(toeplitz_column, toeplitz_row, tensor):
fft_M = fft(temp_tensor.transpose(-1, -2).contiguous())
fft_c = fft(c_r_rev).unsqueeze(-2).expand_as(fft_M)
fft_product = fft_M.mul_(fft_c)

output = ifft(fft_product).real.transpose(-1, -2)

del fft_M

is there a reason we need to manually delete these objects?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tachukao/mgplvm-pytorch/pull/65#pullrequestreview-686306677,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABWICK4NPK5NJJ56N7M26UDTTHZ3NANCNFSM46W6W77Q
.


Navigating to second logged in account wallet makes app restart

Expected behavior
I expect to see the second logged in account wallet details.
Actual behavior
App restarts when I navigate to Wallet  on the second logged in account.
How to reproduce

Open app & Login
Login second account
Make the second account active
Goto Wallet Tab

App version: v2.0.8
Operating system: Android 8.0.1

Recording Of The Bug

This issue has been solved in our latest update, can you check from below links and if it is okay for you or you have any feedback please let me know, thank you :) @horpeyloaded
iOS : install.appcenter.ms/orgs/esteem.app/apps/esteem-1/distribution_groups/internal
Android : install.appcenter.ms/orgs/esteem.app/apps/esteem/distribution_groups/internal


Unify cost postprocessing

This unifies some functions in postprocessing.

[x] Fix #69 by better function reusing
[x] Fix SettingsWithCopyWarnings

The new use of get_calculated_parameters() made some SettingWithCopyWarnings pop up. Precise slicing silenced that.
I did this also to fix this long dragged-along warning:
14:28:41-INFO-Restoring attributes will overwrite existing attributes.
/home/unndreay/.virtualenvs/oemo-flex/lib/python3.7/site-packages/pandas/core/frame.py:2963: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self[k1] = value[k2]
sys:1: PerformanceWarning: indexing past lexsort depth may impact performance.

Normally, the source is given not so indirect. Only found via debugging and watching the console (aggregate_other_capacities()).
Tested with usecase 2a. Similar warnings could pop up again with other functions not used in the usecase.

If you have further ideas for unifying this, please let me know.

After merging dev, I checked all postprocessed results and found them equal to the defaults.


disable for short movies dont work

i want to disable the dimming when i am wathcing a episode.
if i select the option  it is not working any more.

Can you let us know you ~/.kodi/temp/kodi.log ?
Maybe there is some info inside...

here you go:
2020-04-19 18:05:56.593 T:1915311664  NOTICE: VideoPlayer::OpenFile: nfs://<IP_ADDRESS>/volume1/Serien/It's Always Sunny in Philadelphia/Staffel 7/Its.Always.Sunny.in.Philadelphia.S07E06.HDTV.XviD-ASAP.avi
2020-04-19 18:05:56.595 T:1713365728  NOTICE: Creating InputStream
2020-04-19 18:05:56.620 T:1713365728  NOTICE: Creating Demuxer
2020-04-19 18:05:57.488 T:1713365728  NOTICE: Opening stream: 0 source: 256
2020-04-19 18:05:57.488 T:1713365728  NOTICE: Whitelist search for: width: 624, height: 352, fps: 23.976, 3D: false
2020-04-19 18:05:57.491 T:1713365728  NOTICE: Display resolution ADJUST : 1920x1080 (1920x1080) @ 23.98 - Full Screen (52) (weight: 0.000)
2020-04-19 18:05:57.692 T:1915311664  NOTICE: VideoPlayer: OnLostDisplay received
2020-04-19 18:05:57.692 T:1915311664 WARNING: CDVDMessageQueue(audio)::Put MSGQ_NOT_INITIALIZED
2020-04-19 18:05:57.692 T:1915311664 WARNING: CDVDMessageQueue(video)::Put MSGQ_NOT_INITIALIZED
2020-04-19 18:05:57.940 T:1915311664  NOTICE: VideoPlayer: OnResetDisplay received
2020-04-19 18:05:57.941 T:1915311664 WARNING: CDVDMessageQueue(audio)::Put MSGQ_NOT_INITIALIZED
2020-04-19 18:05:57.941 T:1915311664 WARNING: CDVDMessageQueue(video)::Put MSGQ_NOT_INITIALIZED
2020-04-19 18:05:57.956 T:1713365728  NOTICE: Creating video codec with codec id: 12
2020-04-19 18:05:57.973 T:1713365728  NOTICE: Creating video thread
2020-04-19 18:05:57.973 T:1543500512  NOTICE: running thread: video_thread
2020-04-19 18:05:57.975 T:1713365728  NOTICE: Opening stream: 1 source: 256
2020-04-19 18:05:57.975 T:1713365728  NOTICE: Finding audio codec for: 86017
2020-04-19 18:05:57.975 T:1713365728  NOTICE: CDVDAudioCodecFFmpeg::Open() Successful opened audio decoder mp3float
2020-04-19 18:05:57.975 T:1713365728  NOTICE: Creating audio thread
2020-04-19 18:05:57.976 T:1458930400  NOTICE: running thread: CVideoPlayerAudio::Process()
2020-04-19 18:05:57.987 T:1458930400  NOTICE: Creating audio stream (codec id: 86017, channels: 2, sample rate: 48000, no pass-through)
2020-04-19 18:05:58.211 T:1915311664  NOTICE: Whitelist search for: width: 624, height: 352, fps: 23.976, 3D: false
2020-04-19 18:05:58.216 T:1915311664  NOTICE: Display resolution ADJUST : 1920x1080 (1920x1080) @ 23.98 - Full Screen (52) (weight: 0.000)
2020-04-19 18:06:07.497 T:1915311664  NOTICE: VideoPlayer: OnLostDisplay received
2020-04-19 18:06:07.750 T:1915311664  NOTICE: VideoPlayer: OnResetDisplay received
2020-04-19 18:06:07.788 T:1915311664  NOTICE: CVideoPlayer::CloseFile()
2020-04-19 18:06:07.794 T:1713365728  NOTICE: CVideoPlayer::OnExit()
2020-04-19 18:06:07.795 T:1713365728  NOTICE: Closing stream player 1
2020-04-19 18:06:07.795 T:1713365728  NOTICE: Waiting for audio thread to exit
2020-04-19 18:06:07.812 T:1458930400  NOTICE: thread end: CVideoPlayerAudio::OnExit()
2020-04-19 18:06:07.813 T:1713365728  NOTICE: Closing audio device
2020-04-19 18:06:07.820 T:1915311664  NOTICE: VideoPlayer: waiting for threads to exit
2020-04-19 18:06:07.863 T:1713365728  NOTICE: Deleting audio codec
2020-04-19 18:06:07.863 T:1713365728  NOTICE: Closing stream player 2
2020-04-19 18:06:07.863 T:1713365728  NOTICE: waiting for video thread to exit
2020-04-19 18:06:07.903 T:1543500512  NOTICE: thread end: video_thread
2020-04-19 18:06:07.905 T:1713365728  NOTICE: deleting video codec
2020-04-19 18:06:07.912 T:1915311664  NOTICE: VideoPlayer: finished waiting
2020-04-19 18:06:07.913 T:1915311664  NOTICE: CVideoPlayer::CloseFile()
2020-04-19 18:06:07.913 T:1915311664  NOTICE: VideoPlayer: waiting for threads to exit
2020-04-19 18:06:07.913 T:1915311664  NOTICE: VideoPlayer: finished waiting

hello @urstin ,
Same issue for me and after tests, duration seems to be in minutes and not in seconds.
If I put 35 as value, episode under 35 minutes don't affect lights and episode longer than 35 minutes does.


build: the go setup has been configured to 1.18 for retest build

Signed-off-by: Humble Chirammal<EMAIL_ADDRESS>

@Mergifyio rebase

@Mergifyio refresh