Can't finish syncdb

Did nothing, just created mysql db, user, started using "getting started" instructions and it fails:
(oduty)[bajal@bajal:~/git/openduty]# python manage.py syncdb
/home/bajal/oduty/lib/python2.7/site-packages/schedule/models/calendars.py:8: RemovedInDjango19Warning: django.contrib.contenttypes.generic is deprecated and will be removed in Django 1.9. Its contents have been moved to the fields, forms, and admin submodules of django.contrib.contenttypes.
  from django.contrib.contenttypes import generic

/home/bajal/oduty/lib/python2.7/site-packages/django/core/management/commands/syncdb.py:24: RemovedInDjango19Warning: The syncdb command will be removed in Django 1.9
  warnings.warn("The syncdb command will be removed in Django 1.9", RemovedInDjango19Warning)

Operations to perform:
  Synchronize unmigrated apps: openduty, staticfiles, djcelery, messages, templatetags, django_tables2, notification, rest_framework, bootstrap3, django_tables2_simplefilter
  Apply all migrations: schedule, admin, django_twilio, sessions, kombu_transport_django, contenttypes, auth
Synchronizing apps without migrations:
  Creating tables...
    Creating table openduty_token
    Creating table openduty_schedulepolicy
    Creating table openduty_service
    Creating table openduty_eventlog
    Creating table openduty_incident
    Creating table openduty_servicetokens
    Creating table openduty_schedulepolicyrule
    Creating table openduty_userprofile
    Creating table openduty_servicesilenced
    Creating table openduty_incidentsilenced
    Creating table celery_taskmeta
    Creating table celery_tasksetmeta
    Creating table djcelery_intervalschedule
    Creating table djcelery_crontabschedule
    Creating table djcelery_periodictasks
    Creating table djcelery_periodictask
    Creating table djcelery_workerstate
    Creating table djcelery_taskstate
    Creating table openduty_usernotificationmethod
    Creating table openduty_schedulednotification
    Running deferred SQL...
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/bajal/oduty/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
    utility.execute()
  File "/home/bajal/oduty/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/bajal/oduty/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/bajal/oduty/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/home/bajal/oduty/lib/python2.7/site-packages/django/core/management/commands/syncdb.py", line 25, in handle
    call_command("migrate", **options)
  File "/home/bajal/oduty/lib/python2.7/site-packages/django/core/management/__init__.py", line 120, in call_command
    return command.execute(*args, **defaults)
  File "/home/bajal/oduty/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/home/bajal/oduty/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 179, in handle
    created_models = self.sync_apps(connection, executor.loader.unmigrated_apps)
  File "/home/bajal/oduty/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 318, in sync_apps
    cursor.execute(statement)
  File "/home/bajal/oduty/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/home/bajal/oduty/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/home/bajal/oduty/lib/python2.7/site-packages/django/db/utils.py", line 98, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/bajal/oduty/lib/python2.7/site-packages/django/db/backends/utils.py", line 62, in execute
    return self.cursor.execute(sql)
  File "/home/bajal/oduty/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
    return self.cursor.execute(query, args)
  File "/home/bajal/oduty/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/home/bajal/oduty/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
django.db.utils.OperationalError: (1005, 'Can\'t create table `openduty`.`#sql-331e_14` (errno: 150 "Foreign key constraint is incorrectly formed")')

let me check

Are you using MySQL or MariaDB?

It's mariadb

That's where the problem lies.  I had similar problems when using MariaDB and ended up chucking it and going back to MySQL via the community repos.  I simply forgot to file a ticket about it while having the problem, but I can confirm it's related to the interactiong with MariaDB.

I found it was a permissions issue. I had created the openduty user before the database. all I had to do was grant the user permissions on the DB. I chose to just give them full permissions as this is my initial test.
 TO 'openduty'@'%';

You can also do the following:
python manage.py migrate auth
python manage.py migrate
python manage.py syncdb

since @sofixa helped with this one i'm closing this one.

Hi @deathowl,
I tried to create database from scratch in docker (openduty and database are in different containers), but I got the same error as @dbazhal had. I've tried mysql 5.5/5.6 and mariadb 10.1, and got errors about foreign key on all the three. in addition, I've tried steps provided by @sofixa as well. My script is shown below.
NOTE: openduty docker is from https://github.com/openduty/openduty
if ! mysql -h $DATABASES_DEFAULT_HOST -u $DATABASES_DEFAULT_USER -p$DATABASES_DEFAULT_PASSWORD -e 'SHOW DATABASES;' | grep -q $DATABASES_DEFAULT_NAME; then
	mysql -h $DATABASES_DEFAULT_HOST -u $DATABASES_DEFAULT_USER -p$DATABASES_DEFAULT_PASSWORD -e "CREATE DATABASE $DATABASES_DEFAULT_NAME;"

	export DJANGO_SETTINGS_MODULE=openduty.settings_docker
	python manage.py syncdb
	python manage.py migrate
	python manage.py collectstatic --noinput
fi

Same here. Manually tried with all solutions in the thread, both with MySQL and MariaDB and always getting foreign key errors. In both cases I've manually granted ALL privileges to OD MySQL user, just slightly different error texts:
With MySQL 5.6.x:
/usr/local/lib/python2.7/site-packages/schedule/models/calendars.py:8: RemovedInDjango19Warning: django.contrib.contenttypes.generic is deprecated and will be removed in Django 1.9. Its contents have been moved to the fields, forms, and admin submodules of django.contrib.contenttypes.
  from django.contrib.contenttypes import generic

Operations to perform:
  Synchronize unmigrated apps: openduty, staticfiles, djcelery, messages, templatetags, django_tables2, notification, rest_framework, bootstrap3, django_tables2_simplefilter
  Apply all migrations: schedule, admin, django_twilio, sessions, kombu_transport_django, contenttypes, auth
Synchronizing apps without migrations:
  Creating tables...
    Creating table openduty_token
    Creating table openduty_schedulepolicy
    Creating table openduty_service
    Creating table openduty_eventlog
    Creating table openduty_incident
    Creating table openduty_servicetokens
    Creating table openduty_schedulepolicyrule
    Creating table openduty_userprofile
    Creating table openduty_servicesilenced
    Creating table openduty_incidentsilenced
    Creating table celery_taskmeta
    Creating table celery_tasksetmeta
    Creating table djcelery_intervalschedule
    Creating table djcelery_crontabschedule
    Creating table djcelery_periodictasks
    Creating table djcelery_periodictask
    Creating table djcelery_workerstate
    Creating table djcelery_taskstate
    Creating table openduty_usernotificationmethod
    Creating table openduty_schedulednotification
    Running deferred SQL...
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 179, in handle
    created_models = self.sync_apps(connection, executor.loader.unmigrated_apps)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 318, in sync_apps
    cursor.execute(statement)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 98, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 62, in execute
    return self.cursor.execute(sql)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
    return self.cursor.execute(query, args)
  File "/usr/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/usr/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
django.db.utils.IntegrityError: (1215, 'Cannot add foreign key constraint')

With MariaDB 10.x:
/usr/local/lib/python2.7/site-packages/schedule/models/calendars.py:8: RemovedInDjango19Warning: django.contrib.contenttypes.generic is deprecated and will be removed in Django 1.9. Its contents have been moved to the fields, forms, and admin submodules of django.contrib.contenttypes.
  from django.contrib.contenttypes import generic

Operations to perform:
  Synchronize unmigrated apps: openduty, staticfiles, djcelery, messages, templatetags, django_tables2, notification, rest_framework, bootstrap3, django_tables2_simplefilter
  Apply all migrations: schedule, admin, django_twilio, sessions, kombu_transport_django, contenttypes, auth
Synchronizing apps without migrations:
  Creating tables...
    Creating table openduty_token
    Creating table openduty_schedulepolicy
    Creating table openduty_service
    Creating table openduty_eventlog
    Creating table openduty_incident
    Creating table openduty_servicetokens
    Creating table openduty_schedulepolicyrule
    Creating table openduty_userprofile
    Creating table openduty_servicesilenced
    Creating table openduty_incidentsilenced
    Creating table celery_taskmeta
    Creating table celery_tasksetmeta
    Creating table djcelery_intervalschedule
    Creating table djcelery_crontabschedule
    Creating table djcelery_periodictasks
    Creating table djcelery_periodictask
    Creating table djcelery_workerstate
    Creating table djcelery_taskstate
    Creating table openduty_usernotificationmethod
    Creating table openduty_schedulednotification
    Running deferred SQL...
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 179, in handle
    created_models = self.sync_apps(connection, executor.loader.unmigrated_apps)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 318, in sync_apps
    cursor.execute(statement)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 98, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 62, in execute
    return self.cursor.execute(sql)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
    return self.cursor.execute(query, args)
  File "/usr/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/usr/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
django.db.utils.OperationalError: (1005, 'Can\'t create table `openduty`.`#sql-1_6` (errno: 150 "Foreign key constraint is incorrectly formed")')

It's the same with me.
I can not install it

@deathowl : Any tips? Anything else needed from us to help diagnose where is the problem?
TIA.

I got the same error, but changed the order to this instead:
python manage.py migrate auth
python manage.py migrate
python manage.py syncdb
python manage.py collectstatic
python manage.py runserve

guys, please help me out. Stuck on this:
self.build_graph()

File "/home/openduty/openduty/env/lib/python2.7/site-packages/django/db/migrations/loader.py", line 191, in build_graph
self.applied_migrations = recorder.applied_migrations()
File "/home/openduty/openduty/env/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 59, in applied_migrations
self.ensure_schema()
File "/home/openduty/openduty/env/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 49, in ensure_schema
if self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()):
File "/home/openduty/openduty/env/lib/python2.7/site-packages/django/db/backends/base/base.py", line 164, in cursor
cursor = self.make_cursor(self._cursor())
File "/home/openduty/openduty/env/lib/python2.7/site-packages/django/db/backends/dummy/base.py", line 21, in complain
raise ImproperlyConfigured("settings.DATABASES is improperly configured. "
django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.
**Unfortunately, I did not get through **. Here is the database portion of settings.py.  I am using Mysql 5.7(which is sort of running perfect.).
if 'test' in sys.argv:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'test',
'USER': 'openduty',
'PASSWORD': 'Openduty@123',
'HOST': 'localhost',
'PORT': '3306',
}
}
Any idea where I am going wrong @deathowl  @pataquets  @dbazhal

The installation docs should be updated


Just formatting (black & isort)

Ran the following commands:
$ black .  # https://pypi.org/project/black/
$ isort .  # https://pypi.org/project/isort/

Please review/merge https://github.com/imohitmayank/jaal/pull/15 first. After it is submitted, I will rebase this PR since it is branched off of it.


fix: import libraries instead of require

The require() calls trigger an exception ReferenceError: require is not defined
This fixes this by using import instead of require.
To be entirely honest, I'm not sure if there are potential side effects in doing this, but it seems to be a common issue, as seen at: https://github.com/vitejs/vite/issues/3409
None of the suggested solutions there worked, though.

That's weird that shouldn't happen. The reason why I have used require is to let the users choose whether they want to use sharp or svgo or both. That's why the requires are kept within the function blocks instead of top level.
If we keep the imports at the top level then we lose that flexibility and users would have to install both sharp and svgo.
I'm using the package in some of my repo's, I didn't face any issue with requires there.
https://github.com/FatehAK/navigo-app
https://github.com/FatehAK/memrise
Could you check and compare with your repo or provide a minimal repro for me to check?

Okay I've finally figured out the root cause. Had a ton of twist and turns to understand deeply what was going on but in the end it was as simple as the vite config file being named vite.config.mjs.  This indicated that the vite config file was a ES6 module file and therefore it loaded the index.mjs file of this plugin, which does not allow CommonJS require().
Supposedly you can enable transformMixedEsModules in commonjsOptions but even this, for some reason, didn't work.
So there are a few options before closing/resolving this PR.

I believe you can simply remove the module filename in package.json to only build the common JS file file and not the module JS file.
It could be explicitly mentioned in the config that .mjs vite config is not supported, though this seems not optimal.
Use dynamic import to keep the conditional loading, but still remove the require calls. I've adapted the PR for this already if you want to test it out.

During my research into what was going on I also found the maintainer of Rollup mentioning mixing es6 and cjs as a bad practice: https://github.com/rollup/plugins/issues/518#issuecomment-666133709
Which is why I ended up with the dynamic import solution rather than simply renaming my vite config file. (As I also don't know what the potential side-effects of that might be)

Ah no wonder! Thanks for taking the time to figure it out made me understand as well 😄. The dynamic import option looks good I should have went with it from the start my bad.
I just tested it out works well with .mjs and .js vite configs, there was just an issue with vite/rollup bundling svgo and sharp in the final bundle as well. We have to mark them as external seems like and put them in the globals definition.

Merged it, thanks for the suggesting the fix 👍 . Will make a release shortly

Please update to v1.0.6 it has the fixes


Feature request:  Support for Commodore .d71 (1571) disks and images

Hello!  Just got my GreaseWeazle!  Awesome!
I was wondering if support for Commodore 1571 disk images (.d71) can be added please?  The 1571 GCR format is very similar to the 1541 GCR format, but it's double sided.
There is support in the GreaseWeazle for the .d64 to read and write both sides of a flippy disk, surely it wouldn't be too much more complicated to add double-sided 1571 GCR (.d71) format to the awesome list of supported disk formats?  Pleaaasee???  Surely it would also bear some resemblance to the already available 1581 (.d81) format??  (Just guessing--what do I know?)
Thank you SO MUCH.  That would absolutely make GreaseWeazle the ultimate value to me.

WOW!  I can’t believe the rapid response and how eager you are to accommodate!  THANK YOU!
YES!  I would be happy to raw dump a 1571 GCR disk.  I am EXTREMELY new to greaseweazle, so for expedience, What command do I give the GreaseWeazle to do that for you (obviously gw read -–format commodore.1571 won’t work)?  What file extension should the output be?  Just .img?
Yes:  you are correct!  35 cylinders per side.  The disk layout is the same on both sides.
Of course, in the event I am wrong, here is the Commodore 1571 User Guide from Zimmers where it discusses the disk layout in detail:  http://http://www.zimmers.net/anonftp/pub/cbm/manuals/drives/1571_users_Guide_252095-04_(1985_Aug).pdf The technical stuff needed can be found on Pages 104-106.
Looking forward to hearing back from you! :)
THANK YOU!!

Just do gw read --tracks=c=40 a.scp which will read 40 cylinders to an SCP image file. If you are reading on a 96tpi 80 cyl drive then you will need --tracks=c=40:step=2

OK I tried sending the attached SCP file via email.  Were you able to receive it?
Thanks :)

I haven't received an email. Can you upload to a cloud drive such as Google Drive or Dropbox etc and share a link?

GitHub strips attachments from emails but another alternative is to zip the file and then attach it here using the GitHub web interface.

C1571SCP.zip
Ahhh very good!  Thanks! :)

I'm on it!

OMG!!  My old 360K PC Floppy disk drive can write Commodore 1571 double-sided diskettes and read them to .d71 format!!
WOW!!!  You are AMAZING!!
THANK YOU!!!!

--Even WITHIN Commodore software (running on a C128) I can think of only TWO titles that can convert a .d71 to a full 1571 double sided GCR image, but you need a 1581 drive to hold the .d71 file AND a 1571 drive to accomplish this.  I have YET to see the REU used as a RAMdisk to achieve this.
WAY TO GO!  Congratulations!
If you wish to add more functionality, I can provide you many more CP/M MFM 5 1/4" disk images in various formats (I think 100+ e.g. Coleco ADAM, Microbee, SuperBrain, Epson, MAXI, NEC, Olivetti, TRS, Zenith ... MANY more)
Even if not, THANK YOU!!!!!  YOU ROCK!

Great to hear! Adding more formats ia always good. Also I will add a generic IBM format handler soon which will scrape up all sectors it can find on a track, rather than necessarily needing to know the exact track layout ahead of time. Although the latter is better for error checking.


Enable codecov (and up deps)

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@1bd6f23). Click here to learn what that means.
Patch has no changes to coverable lines.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@           Coverage Diff           @@
##             main      #53   +/-   ##
=======================================
  Coverage        ?   84.98%
=======================================
  Files           ?      110
  Lines           ?     8844
  Branches        ?      720
=======================================
  Hits            ?     7516
  Misses          ?     1328
  Partials        ?        0

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Installation issues: I adapted the TSN Docker container – feel free to use it

Hi everyone!
I was trying to make the code run on my machine and I faced significant trouble installing the correct versions of packages because they became obsolete. Then I tried docker. The experience with it wasn't smooth either because dense_flow installation wasn't compatible with my GPU. So, I tried the docker container provided for the TSN network. I tested it with 1080Ti (worked) and 2080Ti (didn't work w/ CUDNN_STATUS_EXECUTION_FAILED) with 64GB RAM.
docker pull iashin/anet2016-cuhk:latest

or build on top of it by specifying the header of your Dokerfile:
FROM iashin/anet2016-cuhk:latest

The output from the examples:

python examples/classify_video.py --use_flow data/plastering.avi

frame sample 53: 0.322223 second
frame sample 54: 0.322393 second
total time: 20.200887 second
----------------Classification Results----------------------
Plastering 0.9559713
Rock climbing 0.020194435
Painting fence 0.011123132
Hanging wallpaper 0.010004199
Painting furniture 0.0007557427
Hand washing clothes 0.00065550825
Laying tile 0.00040866304
Throwing darts 0.00033438313
Paintball 0.00019233384
Hitting a pinata 0.000125195

python examples/classify_video.py --use_flow https://www.youtube.com/watch?v=QkuC0lvMAX0

total time: 91.161525 second
----------------Classification Results----------------------
Windsurfing 1.0
Sailing 6.5195724e-16
Surfing 4.3629453e-17
Kite flying 2.1080195e-18
Wakeboarding 5.306861e-19
Snowboarding 6.466279e-21
Plataform diving 3.3573223e-22
Doing motocross 1.9524087e-22
Scuba diving 1.3819956e-22
Fixing bicycle 1.2276588e-22

Here is the Dockerfile I used to build it (also might be useful if you are not familiar with docker and would like to install it on your machine or as a reference for the package version)
FROM bitxiong/tsn:cuda9_cudnn7

RUN git clone --recursive https://github.com/yjxiong/anet2016-cuhk
WORKDIR /app/anet2016-cuhk
RUN bash models/get_reference_models.sh

RUN cp -r ../lib/ .
RUN cp ../cv2.so .

RUN pip install easydict==1.6
# YouTube API changes often so you may want to install a more recent version of youtube_dl:
RUN pip install youtube_dl==2020.9.20

RUN echo "export ANET_HOME="/app/anet2016-cuhk"" >> ~/.bashrc

Thanks, that is really helpful (:

Hi again, i've the same trouble cas of dense_flow package. There's no opencv version that it works compatible with dense_flow. Btw cuhk project does not work with python 3 version. My question is after i pull your docker image to my locally and enter inside to container, when i try to run following command prompt in order to see the classification results for plastering.avi, i'm getting the error below, so i cann't run the classification.
python examples/classify_video.py --use_flow data/plastering.avi
root@488aab6d3886:/app/anet2016-cuhk# python examples/classify_video.py --use_flow data/plastering.avi
WARNING: Logging before InitGoogleLogging() is written to STDERR
E0805 18:02:37.780357    16 common.cpp:138] Cannot create Cublas handle. Cublas won't be available.
E0805 18:02:37.780797    16 common.cpp:145] Cannot create Curand generator. Curand won't be available.
Setting device 0
F0805 18:02:37.781041    16 common.cpp:190] Check failed: error == cudaSuccess (35 vs. 0)  CUDA driver version is insufficient for CUDA runtime version
*** Check failure stack trace: ***
Aborted (core dumped)

The error i still keep getting, when i run following script.
python examples/classify_video.py  data/plastering.avi

F0807 20:06:21.927471  1948 cudnn_conv_layer.cu:34] Check failed: status == CUDNN_STATUS_SUCCESS (8 vs. 0)  CUDNN_STATUS_EXECUTION_FAILED
*** Check failure stack trace: ***
Aborted (core dumped)

The error i still keep getting, when i run following script. python examples/classify_video.py data/plastering.avi

F0807 20:06:21.927471  1948 cudnn_conv_layer.cu:34] Check failed: status == CUDNN_STATUS_SUCCESS (8 vs. 0)  CUDNN_STATUS_EXECUTION_FAILED
*** Check failure stack trace: ***
Aborted (core dumped)

I'm having the same problem. @yagmurshn

Regrettably, I've exhausted all local solutions, including using nvidia-docker with the --gpus all parameter, but I continue to encounter this error.

It's quite disappointing that the CUHK paper received such widespread recognition in various articles and even secured the winner rank, yet its code failed to work at all, the code did not function as expected.

It's quite disappointing that the CUHK paper received such widespread recognition in various articles and even secured the winner rank, yet its code failed to work at all, the code did not function as expected.

Regrettably, I've exhausted all local solutions, including using nvidia-docker with the --gpus all parameter, but I continue to encounter this error.

Thank you very much for your reply!


Add missing thread include

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (768cfe6) 69.82% compared to head (dffc020) 69.82%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #87   +/-   ##
=======================================
  Coverage   69.82%   69.82%
=======================================
  Files          60       60
  Lines        1455     1455
  Branches      388      388
=======================================
  Hits         1016     1016
  Misses        213      213
  Partials      226      226

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


docker push fails

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
tomcat              latest              f384263102d1        4 days ago          352.2 MB
hello-world         latest              94df4f0ce8a4        4 days ago          967 B
node                latest              708e372a5f46        4 days ago          646.6 MB
ubuntu              latest              44776f55294a        6 days ago          120.1 MB

$ docker tag 94df4f0ce8a4 sarbbottam/hello-world

$ docker images
REPOSITORY               TAG                 IMAGE ID            CREATED             SIZE
tomcat                   latest              f384263102d1        4 days ago          352.2 MB
hello-world              latest              94df4f0ce8a4        4 days ago          967 B
sarbbottam/hello-world   latest              94df4f0ce8a4        4 days ago          967 B
node                     latest              708e372a5f46        4 days ago          646.6 MB
ubuntu                   latest              44776f55294a        6 days ago          120.1 MB

$ docker push sarbbottam/hello-world
The push refers to a repository [docker.io/sarbbottam/hello-world]
5f70bf18a086: Preparing
33e7801ac047: Preparing
unauthorized: authentication required

Output of docker version:
docker version
Client:
 Version:      1.11.0
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   4dc5990
 Built:        Wed Apr 13 19:36:04 2016
 OS/Arch:      darwin/amd64

Server:
 Version:      1.11.0
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   a5315b8
 Built:        Tue Apr 26 15:23:39 2016
 OS/Arch:      linux/amd64

Output of docker info:
docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 4
Server Version: 1.11.0
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 37
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: host bridge null
Kernel Version: 4.4.6-moby
Operating System: Alpine Linux v3.3
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.956 GiB
Name: docker
ID: K2NL:TO5F:IR5M:7GFP:CJTV:GDMX:6NTF:DPFC:7W7P:OVSY:FNKA:4QBU
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): true
 File Descriptors: 18
 Goroutines: 37
 System Time: 2016-05-01T19:40:35.587046655Z
 EventsListeners: 1
Registry: https://index.docker.io/v1/

Additional environment details (AWS, VirtualBox, physical, etc.):
Using docker beta for mac
Steps to reproduce the issue:
$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
tomcat              latest              f384263102d1        4 days ago          352.2 MB
hello-world         latest              94df4f0ce8a4        4 days ago          967 B
node                latest              708e372a5f46        4 days ago          646.6 MB
ubuntu              latest              44776f55294a        6 days ago          120.1 MB

$ docker tag 94df4f0ce8a4 sarbbottam/hello-world

$ docker images
REPOSITORY               TAG                 IMAGE ID            CREATED             SIZE
tomcat                   latest              f384263102d1        4 days ago          352.2 MB
hello-world              latest              94df4f0ce8a4        4 days ago          967 B
sarbbottam/hello-world   latest              94df4f0ce8a4        4 days ago          967 B
node                     latest              708e372a5f46        4 days ago          646.6 MB
ubuntu                   latest              44776f55294a        6 days ago          120.1 MB

$ docker push sarbbottam/hello-world
The push refers to a repository [docker.io/sarbbottam/hello-world]
5f70bf18a086: Preparing
33e7801ac047: Preparing
unauthorized: authentication required

Describe the results you received:
unauthorized: authentication required
Describe the results you expected:
repo should have been pushed to docker hub
Additional information you deem important (e.g. issue happens only occasionally):
I am new to docker, I believe I have to update the registry. But how do I do that?

I am new to docker, I believe I have to update the registry. But how do I do that?
I need to login to docker prior pushing, as suggested in this comment

$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: sarbbottam
Password:
Login Succeeded

$docker push sarbbottam/hello-world
The push refers to a repository [docker.io/sarbbottam/hello-world]
5f70bf18a086: Pushed
33e7801ac047: Pushed
latest: digest: sha256:d35591d3754d16cf250206fc1af4d667eedba0a197d83006844f46b3a3b7bb85 size: 1940

But http://docker.io/sarbbottam/hello-world returns 404 and there is no update at https://hub.docker.com/r/sarbbottam/hello-world/
Any pointer would be helpful.

But http://docker.io/sarbbottam/hello-world returns 404 and there is no update at https://hub.docker.com/r/sarbbottam/hello-world/

Any pointer would be helpful.

Ok, I did notice the following, IMHO, this is really not clear.

If I may suggest,

The push refers to a repository [docker.io/sarbbottam/hello-world] should be replaced with The push refers to a repository [hub.docker.com/r/sarbbottam/hello-world]
and the UI at https://hub.docker.com/r/sarbbottam/hello-world/ should default to the UI of https://hub.docker.com/r/sarbbottam/hello-world/tags/

Redirecting http://docker.io/sarbbottam/hello-world to https://hub.docker.com/r/sarbbottam/hello-world/ would be something for the docker hub team; perhaps you can open an issue at http://github.com/docker/hub-feedback/issues
I'm closing this here, because this is not an issue in the docker engine, but thanks for suggesting


Improve quattor logging during tests

this is not ready to be merged in. i'm investigating the logger during unittesting

@stdweird any update?

@jrha you can merge it for completeness, but logging is now by default handled by code block starting from
https://github.com/quattor/maven-tools/blob/master/build-scripts/src/main/perl/Test/Quattor.pm#L195 (and provides enough control atm)


For Windows?

I need something like this for Windows. I tried to compile in WSL but it fails.

This project is no longer maintained


Tailing comma

Currently Tact doesn't allow tailing comma in struct instantiation:
send(SendParameters{
    to: safeAddress,
    value: self.deployValue, // <--- error here
});

@ex3ndr should this apply to argument list as well? eg:
    receive(
      from: String,
      msg: String,
    ) {
        self.reply(
          msg.toCell(),
        );
    }


SslHandler aggregation prefer copy over CompositeByteBuf

Motivation:
SslHandler will do aggregation of writes by default in an attempt to improve goodput and reduce the number of discrete buffers which must be accumulated. However if aggregation is not possible then a CompositeByteBuf is used to accumulate multiple buffers. Using a CompositeByteBuf doesn't provide any of the benefits of better goodput and in the case of small + large writes (e.g. http/2 frame header + data) this can reduce the amount of data that can be passed to writev by about half. This has the impact of increasing latency as well as reducing goodput.
Modifications:

SslHandler should prefer copying instead of using a CompositeByteBuf

Result:
Better goodput (and potentially improved latency) at the cost of copy operations.

@Scottmitch any benchmarks you can share?

@johnou - Unfortunately not.

4.1 (d7c977dd71)


Blank (transparent) map

I made the terrible mistake of updating cordova, and now, after updating every single plugin, I get no map.
I'm able to get MAP_READY event, everything is transparent, and the map itself is transparent.
I feel bad right now... :disappointed:
What version of cordova/plugins should I use for everything to work again ?

Newest cordova works well with this Plugin.

Please tell me if something is wrong here (any incompatibility ?) :
<EMAIL_ADDRESS>cordova-plugin-chrome-apps-common 1.0.7 "Chrome Apps Common Utils"
cordova-plugin-chrome-apps-storage 1.0.4 "Chrome Apps Storage API"
cordova-plugin-crosswalk-webview 1.4.0 "Crosswalk WebView Engine"
cordova-plugin-device-motion 1.1.1 "Device Motion"
cordova-plugin-geolocation 2.0.0 "Geolocation"
cordova-plugin-shake 0.5.3 "Shake Gesture Detection"
cordova-plugin-splashscreen 3.0.0 "Splashscreen"
cordova-plugin-statusbar 2.0.0 "StatusBar"
cordova-plugin-vibration 2.0.0 "Vibration"
cordova-plugin-whitelist 1.2.0 "Whitelist"
plugin.google.maps 1.3.6 "phonegap-googlemaps-plugin"

I guess you made something wrong on the Crosswalk Tutorial.
Try
        <preference name="xwalkZOrderOnTop" value="true" />

inside your config.xml

I have reinstalled and can confirm that everything (step 5) still need to be done for the plugin to work. The message in the beginning of the tutorial is misleading a bit.
Sorry for the false alarm, I was tired yesterday.
Thank you for your help.

Thanks. Weird, cause genymotion it works just by installing

I will change the text

Hello there,
i'm also having trouble with blank map.
using :

genymotion with android 4.3 - API 18
crosswalk 15+
the maps got blank
(black background when using android:theme="@android:style/Theme.Black.NoTitleBar")
(transparent background when using android:theme="@android:style/Theme.Translucent.NoTitleBar")

but its works fine on android 4.4.4 and android 5.1.0
i dont know what was wrong.. since i did not have experience in java
i attach the screenshot

sorry for bad english..

should be fixed in master


Remote player crashes in small private server

A player, referred to in the logs as "Tiki", is frequently crashing lately. An error within makes mention of Magic Storage Extra each time his client crashes. Within the log file (attached), the relevant errors are near the end.
server.log

Could you send Tiki's client.log?


[Question] Database Choice

I noticed you are using LiteDB by default and suggesting to use Postgres. But why not MySQL? I have mariaDB docker container on my server, and I use it with few other containers. Would be great if I could use it instead of Postgres.
Or is it similar and I can just use MariaDB?

See this comment here:
https://github.com/hargata/lubelog/issues/4#issuecomment-1937787071


Revel cannot create new application with command prompt

Hello,
When creating a new application on my Windows machine using command prompt, I get the below error:

CRIT  11:11:37 command_config.go:286: Abort: could not create a Revel application outside of GOPATH.

However, when using Bash for windows, the application builds successfully.

What is your GOPATH environment variable set to in the windows environment ?

GOPATH is set to "c:/users/d.onaolap/go". This is the same as in my bash shell, but revel only works from bash not command prompt.

me too getting same error what to do now

Try running with powershell
On Tue, Sep 17, 2019, 10:03 AM Secret-Arya<EMAIL_ADDRESS>wrote:

me too getting same error what to do now
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/revel/revel/issues/1441?email_source=notifications&email_token=AGTNKZPYTJTB6HTCJEOL2CTQKCMOFA5CNFSM4HFPLGAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD632ZUQ#issuecomment-532131026,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGTNKZNMWBN46JIKFGAUMO3QKCMOFANCNFSM4HFPLGAA
.

@eikcalb I feel like I've seen this issue on Mac OS as well. Did you find a solution? I don't think this is a Revel issue as much as it is a Go env. config issue.

@brendensoares I could only get this to work using Powershell or got bash for Windows.
What's the experience when not using the default MacOS terminal?

If you still have this problem, I fix it by adding a new line in command_config.go before the error, this is to be sure that c.SrcRoot has the same content of c.GoPath that is the path of our apps.

	 if len(c.SrcRoot) == 0 { 	c.SrcRoot = c.GoPath }
after that you must recompile revel, i did it with the original go get command:
go get github.com/revel/cmd/revel


Tabular array display kicks in with non-scalar values

I mean I can view the contents of the array, but it feels like a bug

The problem here is that I frequently have tabular data where 1 or 2 columns is an array or an object.
If we wanted to check each element (Against a percentage limit for instance) we'd probably kill performance when generating tables too. Any other ideas?

the goal is to not make it feel like it's a bug :) I think the main problem is that the arrays are not expandable in the table maybe

Yeah I briefly considered having it swap tab to contents and unfold the correct element when you click on one of those, but the implementation would be a pain so I pushed it to the back of my mind.

maybe if Kint just displays top level variable with rich renderer and it expands into simple display mode, but I don't even know if that's in the realm of possibility 🗡


Is UPGRADE.md up to date with 0.7.x?

I noticed that UPGRADE.md hasn't been updated in 2 years. Does that mean there haven't been any breaking changes since 0.6.x or has this document fallen out of date?

Hi @mpeteuil. Likely, it has fallen out of date. As a new maintainer, i don't have insight into the progress prior to 0.7. My plan is to change the scope of the document and to track all notable changes for releases moving forward. I will do this today.
I am following semver guidelines for this release and going forward, so there are not meant to be any breaking changes in this release.

Thank you. :+1:

Sounds good, thanks @hubert


question about .rswiftignore management.

Hello,
Someone can please explain me how to exclude a directory and all his childs sub directories EXCEPT one named folder.
I've tried :
/excludedDirectory/**
!/excludedDirectory/child3/**
But don't seems to work.
Thank you.

Hello, no one has an idea ?


Removing duplicate fix target

Signed-off-by: jesus m. rodriguez<EMAIL_ADDRESS>

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 89.223%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1631

Relevant Lines:
1828

💛  - Coveralls


Add a progress bar for loading images

We need to add a progress bar to our GUI so users can clearly see which images are being loaded into Fiji and how close they are to being open. I'm thinking we can either have a separate popup attached to the bottom of our main GUI or make it a part of our current GUI.

Okay, I will be adding that feature in additional to the changes in regarding the file path text area box.


Datepicker opens to current time instead of set date value

Example: https://jsfiddle.net/fkaqzqne/
For some reason, viewDate == now the first time I open the datepicker.  If I close it (without changing anything), the second time I open the datepicker, the viewDate changes to the specified date and displays correctly.  I can get around this by manually setting the viewDate, but this is just a work-around.

Dupe of #1831


Move token addresses to the tokens config file

Description
Moves all the token addresses from the contracts file to the tokens file. This is a more correct place for them and they aren't statically defined so the list can be custom to the network instead of having so many empty strings on networks that don't support LSD tokens.
I also changed a few places referencing "Weth" when really they meant "Wrapped Native Asset".
Type of change

[x] Code refactor / cleanup

How should this be tested?

Ensure places involving stETH / stMATIC work as expected.
Ensure SOR and other things that might be affected by this work correctly.

Checklist:

[x] I have performed a self-review of my own code
[x] I have requested at least 1 review (If the PR is significant enough, use best judgement here)
[x] I have commented my code where relevant, particularly in hard-to-understand areas
[x] If package-lock.json has changes, it was intentional.
[x] The base of this PR is master if hotfix, develop if not

Tests failing.

@timjrobinson
It was the lido mock. I pushed a commit to fix it. We should replace this untyped vi.mock by a better solution if it fails again.


Where is the processing server launched?

Hello.
I'm sorry, I didn't understand where you run the whisper.cpp server.
And where do you download and run the language processing files, which are several gigabytes in size?
Is the processing workload running on a computer? Or is there a specific web server (like Microsoft or Google speech processing servers) that the program interacts with exclusively?
I need an independent framework that is not dependent on any other server service.
In other words, I need to connect to my personal server where I have executed whisper.cpp.
Thank you very much.

There is no need to run whisper.cpp server, this library will use interop and will call directly whisper.cpp native methods.
In other words: it runs fully locally where your C# code is running (in the same process)


initial commit for pmc and committer update

updated pmc & committer team

@xushiyan  @bhasudha ready for review

Change Udit to PMC, Committer.
Add entry for Voon Hou as a committer - https://home.apache.org/phonebook.html?uid=vhs

@bhasudha updated


Add minimal ChainReader interface & types

This adds theChainReader() method to the PluginProvider type definition, and ChainReader type definitions in chain_reader.go
This is intended to be merged just before merging these related PR's which depend on it:

https://github.com/smartcontractkit/chainlink/pull/10990
https://github.com/smartcontractkit/chainlink-cosmos/pull/379
https://github.com/smartcontractkit/chainlink-solana/pull/552
https://github.com/smartcontractkit/chainlink-starknet/pull/331

This commit should get reverted immediately after--it comments out the changes here which depend on the above PR's:
https://github.com/smartcontractkit/chainlink-relay/pull/196/commits/cd7d4e437e026ff154caea82f2eab18f2f15e60a

Looks pretty good - just missing the relay/common change


Box Carousel - regression issue list

Bug 1

The story title's ( if are long ) are not completely visible on the frontend. In backend the story title's are displayed properly but at the frontend title's appear incomplete.

Expected Behaviour

Title should be shown completely

Steps to Reproduce

Go to webstories list
Select latest storeis > Box carousel
Check the stories title

Screenshots
https://user-images.githubusercontent.com/41000648/103637085-f1682d80-4f70-11eb-88e3-811f4330914f.mp4

Backend View

Frontend view

Bug 2

View All Stories is not shown in the carousel instead it is shown above the container. ( In backend the link is not shown ) This is how it should be as per issue #204

Bug 3

Backend the box carosuel looks broken, page is scrolling horizontally

8th Feb 2021 QA status on the changes done as per the client requirements
Cross-checked the above listed issues with the latest changes on https://google-webstories.rt.gw/ and these issues are not valid now.
Hence, closing this issue.
cc: @maitreyie-chavan


how to render children in dsl?

Its not clear (or perhaps there is no way) to render children in the DSL.
For example
class ReverseDiv
  render
    div do
      params[:children].reverse.each do |child|
        # what do I do here?
      end
   end
end

Yeah, you are right on this.. any suggestion?
I am thinking of maybe passing a builder handler as the argument of the block, thus you can manually append element on it.
class ReverseDiv
  render
    div do |handle|
      params[:children].reverse.each do |child|
        handle << child
      end
   end
end

why not just let present check its parameter?  If its of type React::Element then it just pushes it on.
By the way any reason we could not use display as an alias for present?
Another alternative is to add a "display" or "present" method onto React::Element
So then you could say child.display... I like this the best for the following reason:
If you have children then you are probably going to want to clone them to set props on them (http://jaketrent.com/post/send-props-to-children-react/)
So... if we go with child.display, the display method can check, and if there are arguments then it first does a clone of the child and treats the arguments as the new props...
We could also add the display method to string (not 100% sure I like that) so that strings can be emitted as well.
Finally we would need a way to check to child's component class.  Not sure if there is already a way to do this in react.rb, but if not we could redefine is_a? for React::Elements to check the underlying component class type.
Cool?
Jakes example would then read like this:
class RadioOption
  include React::Component
  def render
    label do
      input(type: :radio, value: params[:value], name: params[:name])
      params[:label].display
    end
  end
end

class RadioGroup
  include React::Component
  def render
    div(className: "radio-group") do
      children.each do |child|
        child.display child.is_a? RadioOption ? {name: params[:name]} : *[]
      end
    end
  end
end

Actually, I just found out according to the current implementation, the yielded result of the block provided to the React.create_element or dsl methods would be used as the children. (if none of other dsl methods are invoked in the block), so you could write as,
class ReverseDiv
  render
    div do
      params[:children].reverse
   end
end

Also, I will suggest using React.cloneElement (https://facebook.github.io/react/docs/top-level-api.html#react.cloneelement) (which is not yet integrated to react.rb yet) which is more verbose to people already experienced with reactjs.

Finally we would need a way to check to child's component class. Not sure if there is already a way to do this in react.rb, but if not we could redefine is_a? for React::Elements to check the underlying component class type.

React::Element#element_type(https://github.com/zetachang/react.rb/blob/master/lib/react/element.rb#L9) may be a start point to do this. I won't suggest redefining is_a? given that only the actual rendered component (the one returned by React.render(element)) is the instance of the component class.

Thanks I will work on this with this approach

Fixed.

This issue was moved to reactrb/reactrb#35


TE-3514 fields changes when tabbing out

JIRA Ticket: https://kaligo.atlassian.net/browse/TE-3514
Issue:
During selection, when ENTER and TAB are pressed in quick succession, the selected value changes back to the value at index 0.
Cause:
The current condition for selection with TAB allows either ctrl.open = false or ctrl.open = true.  This should be ok for most cases however, if the user pressed ENTER and TAB in quick succession, the selection will be triggered twice - one for the ENTER and another for TAB.  Since the index will be reset (to 0) from the ENTER selection, the active index when the TAB selection is run will now be 0, hence changing back the value to the first option.
Fix:
Update the condition such that, for single selection, a selection will not be triggered when ctrl.open = false.
Current condition if (!ctrl.multiple || ctrl.open), this is truthy when:
ctrl.multiple = false && ctrl.open = true
ctrl.multiple = false && ctrl.open = false
ctrl.multiple = true && ctrl.open = true
Updated condition if (ctrl.open), this is truthy when:
ctrl.multiple = false && ctrl.open = true
ctrl.multiple = true && ctrl.open = true
How to test in TE:

Apply the change in TE: kaligo-web/node_modules/ui-select/dist/select.js
Recompile webpack
Go to ANZ hotel/car landing page
Tab to the destination search box and search for a destination
When selecting, press enter then tab in quick succession
Validate that the highlighted destination is selected

TODO:
Create PR for TE when this is ready (update in package.json - ui-select version).
"ui-select": "git+https://github.com/Kaligo/ui-select#v<IP_ADDRESS>"

Let's create a PR in kaligo-web, and point to this branch by doing this in package.json.
"ui-select": "git+https://github.com/Kaligo/ui-select#bugfix/TE-3514-fields-changes-when-tabbing-out"

Then merge that PR to staging so the QA can test. Then after the PR in kaligo-web is signed off, let's merge this current PR and release a new tag and update the kaligo-web PR to point to the proper tag release

@playerace Can we merge this and add a release tag?


Docker registry sync

@oybed @JayKayy please review

@etsauer looks good to me - see comment about adding some info re: input file.
@JayKayy I'll let you have the final word  / merge if you're OK with the implementation

@oybed Enhanced usage output.

@etsauer thumbs up from me. :-) :+1:


added List methods push, pop, unshift, shift to List IndexedCursor

I tried to add some methods to the Cursor API because they are missing. This solves #346.
I'm not sure if this is the right way to do it, because all other methods extend both KeyedCursorPrototype and IndexedCursorPrototype and this only extends IndexedCursorPrototype. So all feedback is more then welcome!

It looks like something went wrong with the dist. Everything has src_.... in front of it.
I have no idea what caused this. I've just ran grunt.

closing in favor of new and corrected pull request


Update README.md

Update README.md

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 82.686%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
6323

Relevant Lines:
7647

💛  - Coveralls

@TheJulianJES  @javicalle @MattWestb What do you think about https://github.com/zigpy/zha-device-handlers/pull/2400


Updated race to support array of promises, not just varargs

Duplicate of #374
I apologize that your effort went unthanked this time around but please continue to contribute!


README improvement

Hey Rick, excellent work. Would you be so kind as to provide an example for observing childList (added, removed)? Thx in advance, bm2u

Got it.
$list.watch({
        properties: 'prop_innerHTML',
        watchChildren: true,
        callback: function (data, i, mutations) {

            mutations.forEach(function(record) {
                if (record.type === 'childList' && record.removedNodes.length > 0) {
                    console.info('removed item');
                } else if (record.addedNodes.length > 0) {
                    console.info('added item');
                }
            });
        }
    });

Great - thanks for posting!


Option to only allow POST request methods

The Blackboard LMS LTI implementation appears to send a HEAD request to the LTI callback path. When that doesn't return a failure HTTP status, it follows up by trying a GET (which doesn't include any credentials/params etc so fails)
The proposed change would return a HTTP 400 Bad Request when the request is not a POST forcing it to 'do the right thing' and pass the full request with body etc as a post.
To ensure backward compatibility (if there are some LTI tool consumers that require using a method other than POST), the default is to keep the status quo and allow all request methods. However, with the only_accept_posts option set true, a 400 will be returned.
On a side note, is there a reason the tool provider valid_request! method is used over valid_request? ?? It appears to defeat the purpose of returning fail!(:invalid_credentials) in the callback_phase method.  Thoughts?

Thanks @abrom!
We should definitely try to use valid_request?instead of valid_request!. Maybe we are using the last one for raising the exception rather than dealing with the return value.
Thanks again.

I have been using a patched version with valid_request? for months now and the end user experience is much better. Using the bang version it raises the exception and kills the request instead of allowing a 'nice' response. I will put a PR together and let you think it over :)


Improvement/adding dot net

USDA asked us to add a couple of domains, one of which is a usda.net domain, in OPS-2673.  Hence I needed to adapt the gatherer to handle this particular SLD and .net domains in general.

I did a test run on staging (c4b4) and confirmed that the two new USDA domains are gathered.


CashScript debugger (@mainnet-pat's flipstarter)

@mainnet-pat is working on a debugger for CashScript. For this he is making certain changes to the CashScript compiler and SDK. We'll use this issue to document the progress on that.
This issue is related (and solves) the following long-standing issues as well:
https://github.com/CashScript/cashscript/issues/72
https://github.com/CashScript/cashscript/issues/76

pat, mathieu and I had a call today to discuss some of these things.
pat has been working on integrating source mapping into the CashScript compiler. He's doing that by keeping track of the AST's Location data in the GenerateBytecodeTraversal. This sourcemap information is represented in the same format that Solidity uses for its source mapping and is included in the artifact output.
To make it work for now, we're also adding a "unoptimised bytecode" output that will be used for debugging purposes, because it is much simpler to apply the source-mapping to that compared to the optimised bytecode. As an extension it may be interesting to make it work properly with optimised bytecode as well.
To use this sourcemap, we also require a Libauth template to be exported from CashScript so that it can be evaluated through libauth (and combined with the sourcemap). This is related to #72 and #76. For now this will be implemented in the cashscript library at first, but might be good to create a separate library for in the future. This evaluation can then also be used to write automated tests for your smart contracts that can execute in a Libauth evaluation environment.
pat is also working on adding console.log and require(x, "Failure Message") syntax, which requires the evaluation library (definitely Libauth, but we could try to extend this to node software as well) to keep track of an "instruction pointer" and return it when the transaction is rejected, so we can map the instruction pointer to the corresponding message.

Mathieu and I did some refactoring and adding of tests today, both in the utils and the cashc packages.
Next time, we want to refactor utils/source-map.ts and the CashScript compiler (especially GenerateTargetTraversal). After that we can move on to the most important parts: the libauth template generation.

Today we refactored the ConsoleParameterNode stuff so that it uses pre-existing compiler features for symbol table lookups. We also merged the ANTLR changes from master.
Next time we want to continue looking at the compiler stuff (most of the other GenerateTargetTraversal code). After that we want to refactor utils/source-map.ts and from there we can continue on to the libauth template generation stuff.

Today we finished refactoring the cashc and utils code. Next we will start refactoring the cashscript SDK code.
Looking at the CashScript SDK code, some private properties of classes have been made public. I remember when pat walked us through the code, that this was because that data is required when building the libauth template. We should see if we can change the interface of that function so that it requires fewer public properties. But this is not incredibly important if we can't change it.
We also noticed that there is a special case for the MockProvider in transaction.send(). Ideally this transaction.send() function would be fully provider-agnostic and the special case code should be moved into the MockProvider if possible. Perhaps we also need to look at changing the Libauth template builder interface to enable this.
We also looked at the error messages generated when a transaction failed (see https://twitter.com/CashScriptBCH/status/1745102982255939640), and we should try to make the structure of this message easier to parse quickly (such as changing Test.cash:5 Error to Error in Test.cash on line 5 and splitting up the message on multiple lines, etc).
Maybe we should also add a function to export the built libauth template (similar to bitauthUri() but without converting it to a bitauth URI).
At some point we should probably try to make the JestExtensions export a bit cleaner (e.g. cashscript/test instead of cashscript/dist/test), but that is not a high priority.

Today we added an additional test that checks the compiled source map with our fixture source map. We noticed that the source map generation fixtures were not the same as what was being generated by the latest compiler. We spent a bunch of time debugging, but we have a good indication that the current version is correct. Next time we just want to go back a few commits in the past again to double check that the source map fixture for TransferWithTimeout was definitely incorrect.
We also ran into a few other things:

It could be cool to incorporate all potential functions into a single Libauth Template (right now it is only for the function that is being called).
When using debug functionality (like debugTemplate, getBitauthUri etc), and the artifact does not contain debug information, we should disallow or at least warn the user of this.
If a source file has multi-line comments, then our bitauth script "sourcemap" breaks. We should probably solve this by escaping the end-comments (ie replace */ with *\/).

We double checked that the initial source map fixture was incorrect / outdated, so the one that we replaced it with is correct.
Today we also updated the console.log tests to include more edge cases. Here we noticed some weird behaviour on the JestExtensions file that we added a TODO for in the code. We can look into fixing that next time.
We also solved the issue of multiline-style comments breaking the BitAuth IDE integration due to prematurely ending the "sourcemap" comments.
Next time we want to also update the tests for require statement handling, and look at the JestExtensions stuff mentioned above. After that we want to add tests / fixtures for the full Libauth Template generation process. And the session after that we can finally get to refactoring the libauth template generation process.

Today we double checked the require statement tests, which were mostly fine. Then we got started on defining fixtures for Libauth Template generation (mostly so we can be more confident when refactoring). During this process we found some small issues with the template generation that we fixed.
There are still a few edge cases for which we want  to add template generation fixtures, which we'll do next time + the aforementioned JestExtensions issues. After that we can get started on refactoring the libauth template generation.

Today we added the remaining edge case fixtures for template generation. We probably want to support other hashtypes and signing algorithms in the future, but we decided to skip that for now, since we couldn't think of a very good use case that really needs that.
After that, we implemented the fix that Pat sent us in telegram for an issue with if-else blocks. This turned out to affect some of the existing fixtures/tests, so we updated the fixtures, which were technically incorrect before.
While debugging, we encountered an issue where failed Libauth template evaluation in transaction.send() would report "Error [Object object]". So we looked into the debugTemplate function and noticed that it was throwing errors in different ways (some Error objects, some strings, and presumably either createProgram or vm.debug is also throwing error objects in certain cases). We need to make sure that all these errors are thrown in the same way and are forwarded to the user message correctly.
Next time we want to start the refactor, and the debugTemplate function looks like a good place to start so we can tackle the issues mentioned above.
PS: We're postponing dealing with the JestExtension stuff, since it's less important.

Would you make a new @next release?

Today we published the new next release and shared it in the compilers group - after quickly fixing some fixtures that we forgot to update last time. After this we looked into Sayoshi Nakamario's issue that he reported to Mathieu. In the end we asked him to create a reproduction repo so we can build a test case from it. Finally we made a start on adding tests for the JestExtensions functionality where we ran into some weirdness that we'll tackle next time - we wrote down the TODOs in the code.

Noticed the testing-suite in the examples is currently missing some things: it does not have a README.md or a .gitignore.
Also the initial lockfile is using a currently invalid cashscript version and the compilation fails with the latest node versions.
documenting these issues to so we remember to improve them in the future

Today we fixed some bugs in the console.log statement handling. It looks like console.log statements are now fully working and have a good test coverage.
We also ran into new bugs with the require statement message handling for the final require statement case. For this we've created a list of new test cases we need to create. We've also implemented a likely fix, but we'll need to verify it with the new test cases once those are implemented.
There is also still one outstanding issue with the JestExtensions, where the JestExtensions will pass a test that expects a failure, even when no failure occurs.
We didn't have time to commit the require statement changes, so we want to do another session in the coming days to finish that and look at the final JestExtensions issue.

Today we fixed the last remaining issues with require statement handling. We also expanded the test cases so now we're pretty confident that most edge cases are fully covered. We also moved all debugging related functionality into a new debugging.ts file and refactored that code.
We also figured out the issue with the JestExtensions bug, but didn't have the time to fully implement a fix, so I'll do that early next week and then also release a new prerelease version.

I just did the last fixes to the JestExtensions and published a new next release. Next time we'll start refactoring the LibauthTemplate generation.

Noticed 2 issues while testing
provider.addUtxo(contract.address, randomUtxo() doesn't have the same behavior as
provider.addUtxo(contract.tokenAddress, randomUtxo()
so it seems there is a bug with the mocknetwork provider
When you create a single randomUtxo, there are two inputs created in the libauthTemplate, which causes the contract to fail is certain situations (like tx.inputs.length or this.activeInputIndex)

Today we fixed the first issue mentioned by @mr-zwets above - we now index on lockingBytecode rather than address.
Then we wrote tests for an issue that @mr-zwets had run into where signature checks that failed the NULLFAIL rule did not display custom require messages. We then fixed this issue (so now it still displays the require message when a signature check fails the NULLFAIL rule).
While we were working on that issue we noticed some issues with the template generation regarding signatures:

The template generation always assumes that sig parameters get passed in as SignatureTemplates (but they can also be literal byte sequences).
The template generation always assumes that sig parameters use Schnorr signatures. This causes issues with multisig checks which do not support Schnorr (at least not the way CashScript implements multisig).

We decided to tackle these issues while working on the Libauth template generation refactor, since we didn't want to touch just those parts. We made a start on the overall refactor and plan to continue next time (we documented our progress with TODO comments in the WIP code).

Today we refactored the generation of template.entities and template.scripts. Next time we still need to refactor the following parts of the libauth generation:

All P2PKH / signature template related things (e.g. p2pkh_placeholder_unlock (not including changes in functionality yet)
template.scenarios

createScenarioSourceOutputs
createScenarioTransaction

If we want to we can then tackle the other signature template related todos that are mentioned in the comment above. We also still want to tackle some other new functionality, most importantly, we still want to log the specific require statement (which line), even when users don't specify a custom error message.
There also might be some other TODOs sprinkled in the code that we should look at.
Before releasing the full new version, we also need to make some docs changes:

Document transaction.debug() and transaction.getLibauthTemplate()
Double check the require statement and console.log docs
Go over examples throughout the docs and update if needed
More docs mentioning MockNetworkProvider for testing purposes
Mention playground in the debugging docs

today we almost finished the refactoring, but we still need to push the final changes to createScenarioTransaction. While refactoring, we ran into this Libauth issue (https://github.com/bitauth/libauth/issues/133), so we'll wait to hear back from Jason about that and push the final refactors after.
In the past weeks we also ran into some more issues that we want to fix before full release:

console.log causes bytecode changes in certain cases, due to it changing the OP_PICK / OP_ROLL behaviour of CashScript's stack management. This can be overcome, but we'll need some medium size changes to the way we handle logging
Double check zero-handling (should 0 be 0x or 0x00 and when) and add this to the automated tests
MockNetworkProvider does not work with anything timelock related because values are hardcoded
'slot' -> What to do if a contract has different behaviour depending on input index, how do we evaluate all indexes

here's a proposed checklist
for 0.10.0-next.5

[ ] upstream libauth changes to allow for P2SH32 outputs in template
[ ] investigate and fix zero-handling
[ ] fix timelocks on mocknet
[ ]  log the specific require statement (which line), even when users don't specify a custom error message

for the full v0.10.0

[ ] rework console.log internals not to change bytecode
[ ] investigate slot workings
[ ]  allow for ECDSA signing in template generation
[ ] move Jestextensions (for example to cashscript/test)
[ ] When using debug functionality (like debugTemplate, getBitauthUri etc), and the artifact does not contain debug information, we should disallow or at least warn the user of this.
[ ]  Ideally this transaction.send() function would be fully provider-agnostic and the special case code should be moved into the MockProvider if possible
[ ]  debugTemplate function and noticed that it was throwing errors in different ways (some Error objects, some strings) make sure that error is thrown to user, never [object object]

go over any and all TODOs left in the code
before releasing v0.10.0

[ ] Document transaction.debug() and transaction.getLibauthTemplate()
[ ] Double check the require statement and console.log docs
[ ] Go over examples throughout the docs and update if needed
[ ] More docs mentioning MockNetworkProvider for testing purposes
[ ] update NetworkProvider documentation
[ ] Mention playground in the debugging docs
[ ] reach out to have syntax highlighting plugin updated

We essentially finished all of the big refactoring. We encountered some issues with Libauth not fully supporting P2SH32 in the BCH_SPEC VM. We solved this by using the BCH_2023_05 VM instead. This looks to be working properly. We want to add two more tests for completeness: one that uses token introspection opcodes, and one that uses p2sh20.
We cut a new 0.10.0-next.5 release with all the refactors + some bug fixes. We updated the todo list above for the coming time.
Note about multiple input handling / advanced transaction builder:
Earlier we mentioned that we're not sure how multiple inputs can be tested at once, since there can only be one 'slot' field in a scenario. We'll have to discuss this with Jason, but our expectation is that we'd need to generate multiple scenarios, one for each input, and check that they all pass.
However, we noticed that our scenarios do support P2PKH inputs, which are evaluated separately. So it does seem like it's possible to evaluate multiple inputs using multiple scripts. We will need to look into this further.

Today we looked into a bug with require message handling that Mathieu found. This was caused by an incorrect instruction pointer due to removal of the final verify statement in the bytecode generation.
We tried to move the "final verify remove" code out of the main bytecode generation process and into the static optimisation process (so that the debugging tooling uses the code without the final VERIFY removed). However, this turned out to be very hard if not impossible, due to the fact that we need to remove the final VERIFY for every function, and we lose the function / AST context once we reach the static optimisation stage of compiling.
We've scheduled another session this week to fix the issue in a different way.

Today we finally solved the require message handling edge case that Mathieu found. We also tried to reproduce the mocknet timelock issue, but couldn't reproduce it. Next time we'll need to fix the console.log internals so that console.log statements don't impact production bytecode. We added a first failing test for this.

Today we fixed the two bugs with console.log statements.
One bug we fixed was that console.log statements could not occur as the final statement, even though they don't actually add any opcodes to the compiled bytecode. console.log statements can now be used as the final statement.
The other bug was that console.log statements would change the production bytecode due to stack handling. We changed the way these console.log statements are stored in the artifact so that this bug no longer occurs.
We also updated the tests so this functionality is properly tested.
We made a start towards logging specific require statements (line numbers) even when these statements don't use a custom message. We'll finish that work next time.

Today we finished the work towards logging specific require statements (line numbers) even when those statements don't use custom messages.  We've also updated some of the error handling and transaction sending code to work better with this new functionality. And we've made some changes to the artifact naming.
While working on this, we found an existing bug that likely has something to do with incorrect handling of CHECKDATASIG. We have a failing test case for this and will look at it later.
Finally we released a version 0.10.0-next.6 with the recent changes included.

Today we fixed the failing HodlVault test (which was just due to incorrect test logic). Then we added a new test for P2SH20 template generation, and decided not to add one for token introspection, since that has no impact on the template generation and is already properly tested in the template evaluation tests.
We also ran into a bug with the 0x000...000 placeholder key data, which we resolved by not passing in any default placeholder key. We then added tests for zero-encoding since we still weren't sure how that was supposed to work, but everything seems to be working as intended. We also added support for different signing algorithms / hashtypes in unlock parameters, but still need to do the same for fromP2PKH inputs.
Finally, we updated the JestExtensions so that they can be imported from cashscript/jest.
Next time we'll continue with the other points on the checklist a few messages up.

Today we double checked that all thrown errors were instances of Error, and not literal strings etc. While doing so we ran into a few other smaller issues that we solved along the way.
Afterwards we cleaned up the Transaction send() function and we added a console.warn when using debug functionality with an artifact that does not contain debug information.
Finally we went over some TODOs and moved some of them into the checklist above.
Next time, we'll continue ticking off small issues on the checklist.

Today we added support for multiple P2PKH inputs (with custom signing parameters). After that we added tests for non-require transaction evaluation errors (e.g. invalid OP_SPLIT range or invalid input index).
We then started to enhance the "non-require" transaction errors to also display a line number and failing statement text. This is mostly working, but there are still a few edge cases remaining that we need to tackle. And we need to update all tests to match the new error messages.

Today we updated all e2e tests for the new error messages (including failing statement). While doing that we also fixed a bug that the failing tests showed us. Next time we'll still need to update all of the debugging tests with more extensive error message tests, and then continue the rest of the checklist.

Today we extended our debugging tests to also include the expected failing statement. We upgraded libauth to add a multisig debugging test. We also fixed some export syntax that was causing failing examples.
We then cleaned up / merged the location mapping code in Errors.ts, and got started on making sure multiline errors also display a proper "failing statement". We'll have to continue this next time.

Today we added support and tests for multiline failing statements and fixed some PositionHint bugs.
Next session we can hopefully go over all TODOs + either fix them or decide to delay them until a 0.10.x release. We'll also go over the docs, and plan to release either later this week or next week if possible.

Today we released v0.10.0 🥳
We went over all the docs to improve documentation, list the breaking changes.
We also updated old fixtures, fixed the build and made some class properties public.
With this milestone we can close this issue 'CashScript debugger (mainnet-pat's flipstarter)'
Thanks again @mainnet-pat for the initiative and significant contributions. 🙏
We have created a separate todo list for other refactors we might still want to do later. 👍


combine gas refund to the same address in one vout

Currently gas refunds in a block in one vout per sendtocontract call in the coinstake.
In this tx there are multiple small gas refunds to the space chain owner:
https://explorer.qtum.org/block/3e8f1e483de2690d9ea867a2d2efde249bc2a8b6a998a34e33cdb103be0b337f

Would this create too many fragment UTXOs? Could we maybe combine all gas refunds in a block into one single vout?

Very interesting question, but I don’t think this is architecturally possible (even with protocol changes) in a decentralized asynchronous system. Here is why. Wallets are sending transactions, including token transfers, asynchronously, but because of how queuing in the mempool works, the wallet cannot know which block will contain a transaction. Likewise, for the smart contract, even if the coders wanted to add complex logic to combine gas refunds to a given address, the contract could not know which block would consolidate those transactions. Perhaps the smart contract could wait three blocks and then send the refund, but this would take much code for essentially a corner case, and I'm not sure the contract knows the block height. Keep in mind that the blockchain is run by wallets acting individually, and executing smart contract code individually - there is no server running the smart contract.
There is some good news though. The wallet will sweep up and combine small UTXOs in future transactions, 50, 80, maybe more small UTXOs. And the wallet can probably deal with really small UTXOs (called “dust” in bitcoin), throwing these tiny amounts in with transaction fees to clean them up.

I believe the gas refunds and block rewards are handled by the miners, in the staking reward transaction. And to consolidate these vouts it is only necessary that the miner obey the consolidation rules. No collaboration is necessary from wallet or smart contract.
Whether this change requires hard or soft fork depends on whether the consensus checks each refund vout separately, or if the total refund is correct.

My understanding is that small UTXOs would bloat transactions size. And if relay fee rises enough, smaller UTXOs become uneconomical. In general it's better to avoid output fragmentation.

Good points. Agree only the miners are doing these transactions, with refunds in the coinstake transaction. I haven’t studied the code, but think the way it works is the block reward winning wallet would run all the contract calls, update all the smart contact states/storage, and publish the refund transactions in that specific block. Feels like this would be a hard fork to make this change, because all the other nodes would have to validate the block.
The coinstake transaction, where refunds are issued, pays no fees, so an increase in fees due to increased coinstake size would be a secondary effect on other transactions.
There is some behavior like this already, within a transaction (but not across transactions), if you look at block 100719 for example, part of the BOT air drop. There were seven contract call transactions with 100 token transfers in each. The refunds were consolidated within each of those seven transactions back to address …A3sF, for seven refunds in the ~6.7 QTUM size.

This requires a consensus rule change and so won't be done anytime soon. I agree it would be a good feature to have though. This should be evaluated for inclusion in the x86 fork. I don't believe this is too difficult, but it does require changes to the gas refund consensus check code, as well as the staking code to properly generated the gas refunds

There is a new gas design being proposed with Neutron (previously "x86") which would definitely solve this issue. The AAL will be revamed in NAAL and this behavior completely overhauled. There's three proposals for this behavior right now:

NAAL works somewhat similar to today, except gas fees are attached as a separate UTXO. The NAAL will create a generalized balance map at the end of each block and do sends all at once to each pubkeyhash etc address (would be gas fees+contract sent amount) so that effectively every address would have at most 1 UTXO per block
NAAL works similar to above, but uses a 100% condensed contract system UTXO. In other words, every contract will share a single UTXO for the entire system, otherwise though it'd be a similar situation, 1 UTXO per (non-contract) address per block
Gas fees are completely revamped by implementing a separate GAS token which lives similar to an ERC20 token, and is completely held within the contract system. This would effectively make it so that it could be 1 or less UTXO per address per block

Will leave this open for now for further thoughts, but it'll be blocking on getting the decisions around Neutron decided


Authentication using okta

Hi, i have a requirement to integrate breakerbox with okta single signon for authentication using Active directory. will it be useful here?

Of course! We have had some of the community already add support for Rancher/Kubernetes!
You can see here: https://github.com/yammer/breakerbox/tree/master/breakerbox-turbine/src/main/java/com/yammer/breakerbox/turbine
Feel free to submit a PR.

Closing due to inactivity


Onboarding and Account creation

What is this PR doing?

Updated Onboarding flow
HD Wallet generation + Account creation
Interaction with QuillController via window object
commented out some currently unused static UI elements

How can these changes be manually tested?

run a local node with contracts deployed
build and install extension

Does this PR resolve or contribute to any issues?
closes #131
partially addresses #63
Checklist

[x] I have manually tested these changes
[x] Post a link to the PR in the group chat

Guidelines

If your PR is not ready, mark it as a draft
The resolve conversation button is for reviewers, not authors

(But add a 'done' comment or similar)

Interaction with QuillController via window object

Similar to my comments about ts-ignore, could you update this to use a react context?
Alternatively, if there's any complication there, there are a few different ways to preserve the type:

Use a module for the shared object export default new QuillController(...);
Use a helper function that adds the type information before returning it: function getQuillController(): QuillController { return (window as any).quillController; }
Use inline casts: ((window as any).quillController as QuillController).someQuillControllerFunction();

Updated Onboarding flow

Could you be a bit more specific?

commented out some currently unused static UI elements

Is the commented code still valuable? If so could you convert it into some documentation that explains it as an example or at least add some kind of minimal comment explaining why we're keeping commented code around.

Also, I'm not sure what we'd do about this PR once the points above have been addressed since it's targeting extension-controllers-local-rpc instead of main. Could you do something about that? It looks like the right thing to do is rebase onto main and then include a diff in the PR description representing extension-controllers-local-rpc to explain how to do manual testing.

Similar to my comments about ts-ignore, could you update this to use a react context?

Yes, intended to do that in a follow-up PR. We decided to just use the window object for now for quick POC.

minimal comment explaining why we're keeping commented code around.

To avoid confusion about the dynamic and static elements on page. eg - it will create confusion to the reviewer if clicking a button doesn't change anything on the static UI. Will keep bringing back commented elements as they get integrated with the controllers.

Also, I'm not sure what we'd do about this PR

To me it makes the most sense to continue development with local RPC (into main) for faster testing and once the things are in place, create another PR for changing the target from local to testnet and/or create a env config to auto select that.

Similar to my comments about ts-ignore, could you update this to use a react context?

Per just getting this to work, I recommended using the window for now. I have some changes I'm going to try and wrap up today that make this type safe and make it easier for us to refactor that pattern later. This includes extending the Window type so we don't need to do any weird casts, and we can easily refactor to another access pattern later.

To me it makes the most sense to continue development with local RPC (into main) for faster testing and once the things are in place, create another PR for changing the target from local to testnet and/or create a env config to auto select that. Would like to hear from @jacque006 about it as well.

I'm fine including this in the local RPC work I started if it will make things easier from a local dev perspective.

Use a helper function that adds the type information before returning it: function getQuillController(): QuillController { return (window as any).quillController; }

Ended up going with something similar to this with throws if the controller is accessed before being initialized.
@voltrevo I believe all comments have been addressed/commented on, ready for your re-review.

I'm fine including this in the local RPC work I started if it will make things easier from a local dev perspective.

I agree, changed the base to main so we can just merge this in


⚠ Bofrost.lu has degraded performance

In 94c4fa3, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1898 ms

Resolved: Bofrost.lu performance has improved in 273c985 after 9 minutes.


[FEATURE] Return more fine-grained errors from WADM handlers

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Currently handlers respond with a single type of error ("result": "error"), which makes it tedious to handle errors on the client-side when calling in.
Describe the solution you'd like
It would be fantastic, if each of the wadm handlers had a set of errors that represented the different scenarios that might go wrong or validations on wadm's side which it could then return to the caller. This would make it easier to handle error results from wadm in a more graceful and intelligent fashion.
Describe alternatives you've considered
None.
Additional context
None.

This isn't stable 🙂

@joonas Do you think this is resolved with the wadm-client crate and the inclusion of a ClientError? Or is there more detail you'd like to have here https://docs.rs/wadm-client/0.3.0/wadm_client/error/enum.ClientError.html

@joonas Do you think this is resolved with the wadm-client crate and the inclusion of a ClientError? Or is there more detail you'd like to have here https://docs.rs/wadm-client/0.3.0/wadm_client/error/enum.ClientError.html

Yep, I think that's a great starting point that we can expand upon as new cases crop up, going to close this out.


Significant number of discrepancies for LUAD clinical data -  vital_status / days_to_death / or days_to_last_followup

I was using TCGAmutations to obtain LUAD mutation and clinical data from 'TCGA MC3', so I can plot survival data (Kaplan–Meier) using the mafsurvival function of the maftools library.
I noticed differences between the mafsurvival generated graph (LUAD for the CDKN2A gene) with the survival graph generated on-line (https://www.tcga-survival.com/data-table?view=gene&gene=CDKN2A&filter=cancer_type&cancer_type=LUAD).
To determine the cause of this, I then downloaded the mc3 MAF and the clinical data from "https://gdc.cancer.gov/node/905/":

mc3.v0.2.8.PUBLIC.maf
clinical_PANCAN_patient_with_followup.tsv
I then processed the files with maftools after subsetting for LUAD samples.
When I performed 'mafsurvival' for the CDKN2A gene with this data, it was identical to the www.tcga-survival.com website.

To determine the discrepancies, I compared the specific clinical data downloaded by TCGAmutations and the ones downloaded from "https://gdc.cancer.gov/node/905/".
Here are a few examples of the inaccurate info I found from the pre-compiled TCGAmutations clinical data:
TCGA-44-5645-01A-51D-A27T-08 ,       days to last followup=208,  (should be 852)
TCGA-44-7662-01A-11D-2063-08 ,       days to last followup=50,   (should be 218)
TCGA-53-A4EZ-01A-12D-A24P-08 ,     days to last followup=280,  (should be 1071)
TCGA-55-8089-01A-11D-2238-08 ,      vital status = Alive ;  days to death=NA,  (should be dead and 702, respectively)
TCGA-73-A9RS-01A-11D-A410-08 ,     vital status = Alive ;  days to death=NA,  (should be dead and 340, respectively)
I also went to "https://portal.gdc.cancer.gov/" to search for these specific cases and confirmed that the pre-compiled TCGAmutations data was inaccurate.
Maybe I'm not processing the clinical data from TCGAmutations correctly:
library(TCGAmutations)
library(maftools)
tcga_luad <- tcga_load(study = "LUAD")
luad_maf = read.maf(maf = tcga_luad@data, clinicalData =<EMAIL_ADDRESS>

Hi,
Thanks for the issue and for checking the discrepancies. Note that the clinical data I compiled is from almost 7 years ago. It must have been updated significantly. Thanks for pointing to the clinical_PANCAN_patient_with_followup.tsv file. I will update the current MAFs and clinical info.

Hi,
Thank you for your reply. That's what I assumed, that TCGA updated the data.
Concerning your comment that I don't need to use the read.maf command, I thought it was useful because it would remove "duplicated variants", that is, variants that map to different transcripts of the same gene?
For example, if I do run the command, I got the following response:
> luad_maf = read.maf(maf = tcga_luad@data, clinicalData =<EMAIL_ADDRESS>-Validating
--Removed 12420 duplicated variants
-Summarizing
--Possible FLAGS among top ten genes:
  TTN
  MUC16
  USH2A
-Processing clinical data
-Finished in 4.650s elapsed (4.600s cpu)

Hi,
Thanks again for the issue. I have updated the MC3 cohorts along with the clinical data. Please check the README file regarding the survival columns (they are derived from CDR files as per recommendation).
The duplicated variants are due to some tumors being compared to multiple control samples. I have removed duplicated variants in the current update, so you can use it without rereading the MAF.
I hope this helps.

Could you please confirm if this is what the expected output for overall survival?
luad = TCGAmutations::tcga_load(study = "LUAD")
# OS
maftools::mafSurvival(maf = luad, genes = "CDKN2A", time = "CDR_OS.time", Status = "CDR_OS")

Hi,
My plot I obtained when manually uploading the 2 TCGA files into R, processing the mutation and clinical text files through 'read.maf', then determining survival with 'mafsurvival',  is very similar to yours (except for small differences in the number of WT samples).  Here is what I got:

Here is my code:
library(dplyr)

setwd(file.path(Sys.getenv("USERPROFILE"), "Desktop/TCGA/MAF and Copy number/raw data download from GDC/"))

############## Speed everything up by skipping here, and downloading the processed files instead ###################
library(readr) # for importing tsv
# Load MAF and clinical data (files downloaded from  https://gdc.cancer.gov/node/905/ )
All_cancers_mafs<-read_tsv("mc3.v0.2.8.PUBLIC.maf")
All_cancers_clinical<-read_tsv("clinical_PANCAN_patient_with_followup.tsv")

# Subsetting the LUADs (Want the "acronym" column to be "LUAD")
LUAD_subset_clinical <- subset(All_cancers_clinical, grepl("LUAD", acronym))
# rename the bcr_patient_barcode column name to Tumor_Sample_Barcode
colnames(LUAD_subset_clinical)[colnames(LUAD_subset_clinical) == "bcr_patient_barcode"] <- "Tumor_Sample_Barcode"

# Modify All_cancers_mafs$Tumor_Sample_Barcode such that only first eight characters are kept.
All_cancers_mafs$Tumor_Sample_Barcode <- substr(All_cancers_mafs$Tumor_Sample_Barcode, 1, 12)

# Subsetting the MAF for LUAD samples only
# Match Tumor_Sample_Barcode between data and clinical data
matched_LUADsamples <- intersect(LUAD_subset_clinical$Tumor_Sample_Barcode, All_cancers_mafs$Tumor_Sample_Barcode)

# Filter clinical data based on matched samples
LUAD_mafs_text <- All_cancers_mafs[All_cancers_mafs$Tumor_Sample_Barcode %in% matched_LUADsamples, ]

# Cleanup
rm(All_cancers_mafs)
rm(All_cancers_clinical)
rm(matched_LUADsamples)

write_tsv(LUAD_mafs_text, "GDC_LUAD_mafs_text.tsv")
write_tsv(LUAD_subset_clinical, "GDC_LUAD_subset_clinical.tsv")
############## Speed everything up by skipping above, and downloading the processed files instead

#### Speed things up, Download the processed maf and clinical data (filtered for LUAD)
library(readr) # for importing tsv
LUAD_mafs_text <- read_tsv("GDC_LUAD_mafs_text.tsv")
LUAD_subset_clinical <- read_tsv("GDC_LUAD_subset_clinical.tsv")

library(maftools)
# Converting the maf and clinical text files into a maftools S4 object
luad_maf = read.maf(maf = LUAD_mafs_text, clinicalData = LUAD_subset_clinical)

## Survival based on mutation
luad_maf_v2 <- luad_maf # Make a copy first

# Create<EMAIL_ADDRESS>such that 0 is alive and 1 is dead,
#     based on $vital_status being Alive or Dead, respectively.
<EMAIL_ADDRESS><-<EMAIL_ADDRESS>== "Alive", 0,
<EMAIL_ADDRESS>== "Dead", 1, NA))

# Create an "overall_survival" variable that is equal to days_to_death for dead patients,
#         and to days_to_last_follow_up for patients who are still alive
#    Checks each row of<EMAIL_ADDRESS>#         If the value is "1", it fills the corresponding row of<EMAIL_ADDRESS>#                   with the value from<EMAIL_ADDRESS>#         If the value is "0", it fills the corresponding row of<EMAIL_ADDRESS>#                   with the value from<EMAIL_ADDRESS><EMAIL_ADDRESS><- ifelse(
<EMAIL_ADDRESS>== "1",
<EMAIL_ADDRESS><EMAIL_ADDRESS>)

{genesurvival <- "CDKN2A"
  mafSurvival(
    luad_maf_v2,
    genes = genesurvival,
    samples = NULL,
    clinicalData = NULL,
    time = "overall_survival",
    Status = "surv_event",
    groupNames = c("Mutant","WT"),
    showConfInt = TRUE,
    addInfo = TRUE,
    col = c("maroon", "royalblue"),
    isTCGA = FALSE,
    textSize = 12
  )
}

When using the website to check on the LUAD - CDKN2A mutation status on survival, it was also very similar but not exactly the same:

When I downloaded the CSV file that is related that this particular survival plot, I counted mutant=20 and Wild-type=486, though the p-value of 0.7971 is still slightly different (I don't think the p-value difference was due to it being a rounded off value and if it was, rounding off 0.7971 would be 0.797 and not 0.798).  Thus, there are tiny differences between the website's survival analysis and our data. Not sure if it is worth analyzing in "microscopic" detail of what caused the small differences and I don't know if there is a wrong or a right.

Hi,
Thanks for the detailed reply. Appreciate it.
Let me know how it goes. If the differences are not too significant, we can ignore them.

Anand,
Thank you for your reply and insight on the TCGA barcodes.  I'll take a look at your code.  Also, thank you for your work on the various R packages (e.g. maftools, TCGAmutations), as they have been super helpful for my studies!
Best regards,
Mitchell