s3 Integration for distributed storage

As indicated in https://github.com/fridiculous/django-estimators/issues/7, aws s3 can be a useful file storage system, for estimators and for datasets.
Please make integration simple and easy-to-integrate with django estimators.

Hello buddy,
Here is my contribution for you:
If you want, you can grant me access to a specific AWS S3 development branch as a contributor.
I can test if the library is working in a production environment.
Warning: There is a default behavior, which should not be kept:

settings.DEBUG is True: Then storage happen on disk
settings.DEBUG is False: Then storage happen on S3

models_estimators.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import unicode_literals

# original based on sci-kit hashing function
from django.conf import settings
from django.core.exceptions import ValidationError
from django.db import models
from django.core.files.base import ContentFile
from django.core.files.uploadedfile import InMemoryUploadedFile
from django.utils import timezone

from cStringIO import StringIO

import os, string, pickle, random, uuid, urllib

def id_generator(size=20, chars=string.ascii_uppercase + string.ascii_lowercase + string.digits):
	return ''.join(random.choice(chars) for _ in range(size))

def get_upload_path():

	while True:
		filename = "%s.%s" % (id_generator(), "model")

		file_url = "%s%s" % (settings.MEDIA_URL, filename)

		if settings.DEBUG or urllib.urlopen(file_url).getcode() != 200:
			break

	return settings.USER_ESTIMATOR_PATH + filename

def get_upload_path_instance(instance, filename):
	if instance.object_file:

		filename = instance.object_file.name

		if settings.DEBUG:
			instance.object_file.delete(save=True)

		return filename

	else:
		return get_upload_path()

class Estimator(models.Model):

	"""This class creates estimator objects that persists predictive models

		An Estimator instance has multiple attributes

			:description:
			:estimator:

			>>> from estimators.models import Estimator
			>>> est = Estimator()
			>>> est.estimator = object
			>>> est.description = "k-means with 5 clusters"
			>>> est.save()

	"""

	creation_date = models.DateTimeField(auto_now_add=True, blank=False, null=False, editable=False)
	last_modified_date = models.DateTimeField(auto_now_add=True, blank=False, null=False)
	object_file = models.FileField(upload_to=get_upload_path_instance, default="", blank=True, editable=False)
	description = models.CharField(max_length=256)

	def __repr__(self):
		return '<Estimator <Id %s>: %s>' % (self.id, self.estimator)

	@property
	def estimator(self):
		"""return the estimator, and load it into memory if it hasn't been loaded yet"""
		return self.get_object()

	@estimator.setter
	def estimator(self, obj):
		self.set_object(obj)

	def get_object(self):
		if settings.DEBUG:
			return pickle.load(self.object_file)
		else:
			object_url = "%s%s" % (settings.MEDIA_URL, self.object_file)
			model = urllib.urlopen(object_url).read()
			return pickle.loads(model)

	def set_object(self, object_file):
		old_name = self.object_file.name # Not used for now

		buf = StringIO(pickle.dumps(object_file))  # `data` is your stream of bytes
		buf.seek(0, 2)  # Seek to the end of the stream, so we can get its length with `buf.tell()`
		temp_filename = str(uuid.uuid4())
		file = InMemoryUploadedFile(buf, "Scikit Model", temp_filename, "application/octet-stream", buf.tell(), None)
		self.object_file.save(file.name, file)  # `object_file` is an instance of `Estimator`
		self.save()

	def save(self, *args, **kwargs):
		self.last_modified_date = timezone.now()
		super(Estimator, self).save(*args, **kwargs)

	def delete(self, *args, **kwargs):
		self.object_file.delete(save=True)
		super(Estimator, self).delete(*args, **kwargs)

Environment Variables
AWS_USER='########################'
AWS_SECRET_KEY='###########################################'

custom_storages.py
from django.conf import settings
from storages.backends.s3boto import S3BotoStorage

class StaticStorage(S3BotoStorage):
	location = settings.STATICFILES_LOCATION

class MediaStorage(S3BotoStorage):
	location = settings.MEDIAFILES_LOCATION

settings.py
STATICFILES_LOCATION = 'static'
STATIC_ROOT = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'staticfiles')

MEDIAFILES_LOCATION = 'media'
MEDIA_ROOT = os.path.join(os.path.dirname(os.path.dirname(__file__)), MEDIAFILES_LOCATION)

if DEBUG:

	# Simplified static file serving.
	# https://warehouse.python.org/project/whitenoise/
	STATICFILES_STORAGE = 'whitenoise.django.GzipManifestStaticFilesStorage'
	STATIC_URL = "/%s/" % STATICFILES_LOCATION

	DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'

	# Storage URL
	MEDIA_URL = "/%s/" % MEDIAFILES_LOCATION

else:

	AWS_HEADERS = {  # see http://developer.yahoo.com/performance/rules.html#expires
		'Expires': 'Thu, 31 Dec 2099 20:00:00 GMT',
		'Cache-Control': 'max-age=120', #max age in seconds => i.e : 94608000 = 3 years, 3600 = 1 hour, 120 = 2min
	}

	AWS_STORAGE_BUCKET_NAME = '##################' #Bucket Name
	AWS_ACCESS_KEY_ID = "#########################"
	AWS_SECRET_ACCESS_KEY = "##############################"

	# Tell django-storages that when coming up with the URL for an item in S3 storage, keep
	# it simple - just use this domain plus the path. (If this isn't set, things get complicated).
	# This controls how the `static` template tag from `staticfiles` gets expanded, if you're using it.
	# We also use it in the next setting.
	AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME

	################################################################################ STATIC FILES ################################################

	# Tell the staticfiles app to use S3Boto storage when writing the collected static files (when you run `collectstatic`).
	STATICFILES_STORAGE = 'application.custom_storages.StaticStorage'
	STATIC_URL = "https://%s/%s/" % (AWS_S3_CUSTOM_DOMAIN, STATICFILES_LOCATION)

	################################################################################ STATIC FILES ################################################

	# Tell the staticfiles app to use S3Boto storage when serving and uploading media files.
	DEFAULT_FILE_STORAGE = 'application.custom_storages.MediaStorage'
	MEDIA_URL = "https://%s/%s/" % (AWS_S3_CUSTOM_DOMAIN, MEDIAFILES_LOCATION)


Unable to easily screenshot certain elements

Issue
Github pages tend to have lots of elements which are not uniquely referenceable via CSS (at least with the selectors available in our Poltergeist library).  This means that we're unable to grab the second div in a list, unless they have id's.
XPath selectors would solve this, but we're unable to use them as they don't exist in Poltergeist.  An issue has been raised about this.
Solution

Wait for a Poltergeist/PhantonJS fix upstream.
Execute Javascript within our PhantomJS session to compute bounding rectangles, and crop full screen images.

What's wrong with nth-child or nth-child-of-type CSS selectors?

@twalpole Great!  That's exactly what I was looking for.  I mis-remembered CSS pseudo classes and was trying el:nth(n).
This solves my use case.


postinstall just sucks without proper preinstall tooling

need to add a preinstall rm dist/* script.
part of #207

will revert back to prepare instead as the above might pose some issues with the windows platform.


Using test.resolves will throw on failure and prevent collection of journal logs

Either we are using this function incorrectly in the meta-balena suite, or something is broken in our tap.
https://node-tap.org/docs/api/asserts/#tresolvespromise--fn-message-extra
Example of our usage in meta-balena:
https://github.com/balena-os/meta-balena/blob/f2690121369a4481e735cfa79b0ae6da034009f1/tests/suites/os/tests/chrony/index.js#L84-L95

leviathan-client-1  | [2022-07-13T17:38:33.989Z][worker-hup]         not ok 1 - Should catch rejection
leviathan-client-1  | [2022-07-13T17:38:33.989Z][worker-hup]           ---
leviathan-client-1  | [2022-07-13T17:38:33.990Z][worker-hup]           at:
leviathan-client-1  | [2022-07-13T17:38:33.990Z][worker-hup]             line: 50
leviathan-client-1  | [2022-07-13T17:38:33.990Z][worker-hup]             column: 14
leviathan-client-1  | [2022-07-13T17:38:33.990Z][worker-hup]             file: /data/suite/tests/rollback-altboot.js
leviathan-client-1  | [2022-07-13T17:38:33.990Z][worker-hup]             type: Proxy
leviathan-client-1  | [2022-07-13T17:38:33.990Z][worker-hup]             function: run
leviathan-client-1  |           found:
leviathan-client-1  | [2022-07-13T17:38:33.990Z][worker-hup]             !error
leviathan-client-1  | [2022-07-13T17:38:33.991Z][worker-hup]             name: Error
leviathan-client-1  |             message: Fake error
leviathan-client-1  | [2022-07-13T17:38:33.991Z][worker-hup]             stack: |-
leviathan-client-1  | [2022-07-13T17:38:33.991Z][worker-hup]               Error: Fake error
leviathan-client-1  | [2022-07-13T17:38:33.991Z][worker-hup]                   at Proxy.run (/data/suite/tests/rollback-altboot.js:51:28)
leviathan-client-1  |           source: |
leviathan-client-1  | [2022-07-13T17:38:33.991Z][worker-hup]                            // 0 means file exists, 1 means file does not exist
leviathan-client-1  | [2022-07-13T17:38:33.991Z][worker-hup]                            await test.resolves(
leviathan-client-1  | [2022-07-13T17:38:33.991Z][worker-hup]             -------------^
leviathan-client-1  |                         Promise.reject(new Error("Fake error")),
leviathan-client-1  | [2022-07-13T17:38:33.992Z][worker-hup]                                    'Should catch rejection'
leviathan-client-1  | [2022-07-13T17:38:33.992Z][worker-hup]           ...
leviathan-client-1  | [2022-07-13T17:38:34.000Z][worker-hup]
leviathan-client-1  | [2022-07-13T17:38:34.000Z][worker-hup]         Bail out! Should catch rejection

Looking at the docs + our code, it seems test.resolves expects promises to be handled in both events of resolution/rejection. From first looks of the example, it seems we should be catching the promise rejection too?
test.resolves probably doesn't have any error catching functionality inbuilt to not hide errors.

Fair enough, thanks @vipulgupta2048 !


Create sample user stories

The task is to create scenarios of software usage and place them on Wiki.

 Create sample user stories

Sample user stories (not too detailed for now) are here: https://github.com/maldi-team/msi-divik/wiki/user-stories-index


Compress Integer attributes

JOINT attributes for example are integer values that are currently stored as floats in a lot of models. A compressIntegerAttributes stage could be written that checks if an accessor is marked as float, but only contains integers, then choosing a suitable int size (and sign) to contain the range.

@JoshuaStorm feel free to tackle this one if you have time; otherwise I'll do it eventually

Good eye, @lasalvavida!  @JoshuaStorm this is not a priority now.

Closed by #121


Issue writing the dataset as parquet in add_features

I'm running the following (locally, not docker) with the latest from mainline:  python -m deepform.data.add_features data/3_year_manifest.csv
And I'm getting this traceback:
Traceback (most recent call last):
  File<EMAIL_ADDRESS>line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File<EMAIL_ADDRESS>line 87, in _run_code
    exec(code, run_globals)
  File "/Users/evan/deepform/deepform/data/add_features.py", line 263, in <module>
    extend_and_write_docs(
  File "/Users/evan/deepform/deepform/data/add_features.py", line 98, in extend_and_write_docs
    doc_index.to_parquet(pq_index)
  File "/Users/evan/Library/Caches/pypoetry/virtualenvs/deepform-kruIrF0o-py3.8/lib/python3.8/site-packages/pandas/util/_decorators.py", line 199, in wrapper
    return func(*args, **kwargs)
  File "/Users/evan/Library/Caches/pypoetry/virtualenvs/deepform-kruIrF0o-py3.8/lib/python3.8/site-packages/pandas/core/frame.py", line 2365, in to_parquet
    to_parquet(
  File "/Users/evan/Library/Caches/pypoetry/virtualenvs/deepform-kruIrF0o-py3.8/lib/python3.8/site-packages/pandas/io/parquet.py", line 270, in to_parquet
    return impl.write(
  File "/Users/evan/Library/Caches/pypoetry/virtualenvs/deepform-kruIrF0o-py3.8/lib/python3.8/site-packages/pandas/io/parquet.py", line 101, in write
    table = self.api.Table.from_pandas(df, **from_pandas_kwargs)
  File "pyarrow/table.pxi", line 1376, in pyarrow.lib.Table.from_pandas
  File "/Users/evan/Library/Caches/pypoetry/virtualenvs/deepform-kruIrF0o-py3.8/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 593, in dataframe_to_arrays
    arrays[i] = maybe_fut.result()
  File<EMAIL_ADDRESS>line 432, in result
    return self.__get_result()
  File<EMAIL_ADDRESS>line 388, in __get_result
    raise self._exception
  File<EMAIL_ADDRESS>line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/evan/Library/Caches/pypoetry/virtualenvs/deepform-kruIrF0o-py3.8/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 565, in convert_column
    raise e
  File "/Users/evan/Library/Caches/pypoetry/virtualenvs/deepform-kruIrF0o-py3.8/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 559, in convert_column
    result = pa.array(col, type=type_, from_pandas=True, safe=safe)
  File "pyarrow/array.pxi", line 265, in pyarrow.lib.array
  File "pyarrow/array.pxi", line 80, in pyarrow.lib._ndarray_to_array
  File "pyarrow/error.pxi", line 84, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: ('Could not convert 0.0 with type str: tried to convert to double', 'Conversion failed for column gross_amount with type object')

I inspected the DataFrame, and the issue appears to be that the document with slug 499480-cancel-68803-13518579030793-_-pdf has 0.0 for gross_amount, which prevents conversion to double.
One solution might be to do:
doc_index['gross_amount'] = doc_index.gross_amount.apply(pd.to_numeric, errors='coerce')
before exporting to parquet format, but I wanted to confirm with you all that this field is supposed to be float, and that the 0.0 amount isn't a mistake. I'm also not sure why no one else has run into this, so maybe something else is up.

I think this is happening because of mixed dtypes in dollar_amount (both str and float) which doesn't vibe with pyarrow (or at least version 1.0.1 for me). I got the pd.to_numeric solution working, but it required too many other changes around the codebase, so instead I'm casting to str in my PR


COMPAT: Expand compatibility fromnumeric.py

Addresses issues in #5987 by fixing signature incompatibilities between functions defined in numpy.core.fromnumeric and similarly defined ones for scipy.sparse matrices by expanding the signatures in scipy.sparse to accommodate (and also implement) some of the functionality offered in numpy, in particular, these four functions:

sum
mean
reshape
transpose

cc @perimosocordiae

Why isn't reshape implemented for the other sparse matrices?
np.diagonal is another one in which there is a signature incompatibility, but it doesn't surface because numpy casts everything to ndarray (which is disastrous for a sparse matrix but no other Exception is thrown at the very least).  Is it worthwhile to match signatures even though it technically not an issue (aside from the casting part)?

@@            master   #6002   diff @@
======================================
  Files          238     238
  Stmts        43803   43845    +42
  Branches      8211    8227    +16
  Methods          0       0
======================================
+ Hit          34230   34278    +48
+ Partial       2603    2602     -1
+ Missed        6970    6965     -5

Review entire Coverage Diff as of fea493d

Powered by Codecov. Updated on successful CI builds.

I'm not sure if pointing to the numpy functions in the "See Also" block is worthwhile. I would rather point to the corresponding ndarray method, or just omit that section entirely.
There have been efforts to implement a reshape method for other sparse formats (see #3957) but motion has stalled. In theory, it should be implemented more broadly.
If np.diagonal doesn't attempt to call spmatrix.diagonal, then I'd prefer to leave it as-is.

Okay, very well.  I'll leave spmatrix.diagonal as is then.  I think actually it would better on second thought to point to the np.matrix methods since that is what we return for the methods that I mentioned.
With regards to reshape, yikes!  In my mind, it felt like it should be quite simple given the abundance of conversion functions we have lying around that we use quite freely for transpose for example.  I am perfectly okay with not implementing them for the other methods given that that was not the original purpose of my PR.  Some feedback on my implementation decision for order would be nice, since that could have consequences on future implementations of the method for other sparse matrix types.

xref <a href="https://github.com/numpy/numpy/issues/7460>#7460 in numpy.  It looks like some more incompatibilities to be squashed, both on the numpy and scipy ends.

@gfyoung: It's pretty hard to follow this PR because it includes a lot of PEP8/whitespace changes mixed in with the  actual code changes. Could you hold off on squashing for a while to help me follow the progress better?

@perimosocordiae : Oh goodness!  I must have done the squashing without noticing!  So sorry!

To give you a hand at what the most important parts of the PR are, search for any of the following terms:
def sum
def min
def max
def mean
def reshape
def transpose
Again, my bad about throwing all of those PEP8 changes along with it.

@perimosocordiae : Any updates on this PR?  Again, sorry that I mixed the PEP8 changes with the actual changes, so if that is impeding your ability to review this, I perfectly understand.

@perimosocordiae : Any updates on this PR?  Again, sorry that I mixed the PEP8 changes with the actual changes, so if that is impeding your ability to review this, I perfectly understand.

@perimosocordiae : Any updates on this PR?  Again, sorry that I mixed the PEP8 changes with the actual changes, so if that is impeding your ability to review this, I perfectly understand.

@perimosocordiae : Any updates on this PR?  Again, sorry that I mixed the PEP8 changes with the actual changes, so if that is impeding your ability to review this, I perfectly understand.

@perimosocordiae : Any updates on this PR?  Again, sorry that I mixed the PEP8 changes with the actual changes, so if that is impeding your ability to review this, I perfectly understand.

[Current coverage][cc-pull] is 78.21%

Merging [#6002][cc-pull] into [master][cc-base-branch] will increase coverage by +<.01%

@@             master      #6002   diff @@
==========================================
  Files           238        238
  Lines         43878      43886     +8
  Methods           0          0
  Messages          0          0
  Branches       8214       8224    +10
==========================================
+ Hits          34307      34323    +16
+ Misses         6968       6962     -6
+ Partials       2603       2601     -2

3 files (not in diff) in scipy were modified. more

Misses +1
Partials +1
Hits -6

4 files in scipy/sparse were modified. more

Misses +3
Partials +2
Hits -26

[][cc-pull]

Powered by Codecov. Last updated by e54beb6
[cc-base-branch]: https://codecov.io/gh/scipy/scipy/branch/master?src=pr
[cc-pull]: https://codecov.io/gh/scipy/scipy/pull/6002?src=pr

Can somebody take a look at this?  It seems like @perimosocordiae is very busy ATM, but I do see other people with merge-powers actively merging things.  It would be great if someone could take a look at this, as it is a compatibility issue with numpy.

Looks reasonable to me, but I'll let someone more familiar with scipy.sparse have a look and merge when they're happy.

@Eric89GXL : Do you know of anyone (with merging powers) in particular?  If so, you could perhaps ping them directly in this PR.

I suspect most, if not all, merge-power folks have repo notifications on or look through PRs/issues when they have time. I've tagged this as scipy.sparse so that when sorting by submodule it can be found easily.

@Eric89GXL : Oh, what I meant was do you know anyone in particular who is very familiar with scipy.sparse! :smile:

I made a few doc-related comments, but once those are resolved I'm +1 to merge.
Thanks for your patience, @gfyoung!

@perimosocordiae @Eric89GXL : Travis has given the green light.  Ready to merge if there is nothing else.

Merged!


Missing Python version marker in uv export output for dependency in a group

I have a pyproject.toml with a dependency group (thanks for adding support for this!) that contains a dependency that should only be installed for Python >=3.9 while our project support Python >=3.8 , here's the simplified version:
[project]
name = "foo"
version = "0.1.0"
requires-python = ">=3.8"
dependencies = [
    "packaging",
]

[dependency-groups]
foo = [
    "rucio-clients>=33.6.0 ; python_version>='3.9'",
]

When I export the foo dependencies to the requirements.txt format, the python_full_version >= '3.9' marker is missing for rucio-clients:
$ uv export --no-hashes --only-group=foo
Using CPython 3.12.7 interpreter at: /usr/bin/python3.12
Resolved 19 packages in 1ms
# This file was autogenerated by uv via the following command:
#    uv export --no-hashes --only-group=foo
attrs==24.2.0 ; python_full_version >= '3.9'
certifi==2024.8.30 ; python_full_version >= '3.9'
charset-normalizer==3.4.0 ; python_full_version >= '3.9'
decorator==5.1.1 ; python_full_version >= '3.9'
dogpile-cache==1.3.3 ; python_full_version >= '3.9'
idna==3.10 ; python_full_version >= '3.9'
jsonschema==4.23.0 ; python_full_version >= '3.9'
jsonschema-specifications==2024.10.1 ; python_full_version >= '3.9'
pbr==6.1.0 ; python_full_version >= '3.9'
referencing==0.35.1 ; python_full_version >= '3.9'
requests==2.32.3 ; python_full_version >= '3.9'
rpds-py==0.20.0 ; python_full_version >= '3.9'
rucio-clients==35.4.1
stevedore==5.3.0 ; python_full_version >= '3.9'
tabulate==0.9.0 ; python_full_version >= '3.9'
typing-extensions==4.12.2 ; python_full_version >= '3.9' and python_full_version < '3.11'
urllib3==2.2.3 ; python_full_version >= '3.9'

Note that if I move rucio-clients from the foo dependency group into the main list of dependencies, the marker is exported as expected:
[project]
name = "foo"
version = "0.1.0"
requires-python = ">=3.8"
dependencies = [
    "packaging",
    "rucio-clients>=33.6.0 ; python_version>='3.9'",
]

$ uv export --no-hashes
Using CPython 3.12.7 interpreter at: /usr/bin/python3.12
Resolved 19 packages in 142ms
# This file was autogenerated by uv via the following command:
#    uv export --no-hashes
attrs==24.2.0 ; python_full_version >= '3.9'
certifi==2024.8.30 ; python_full_version >= '3.9'
charset-normalizer==3.4.0 ; python_full_version >= '3.9'
decorator==5.1.1 ; python_full_version >= '3.9'
dogpile-cache==1.3.3 ; python_full_version >= '3.9'
idna==3.10 ; python_full_version >= '3.9'
jsonschema==4.23.0 ; python_full_version >= '3.9'
jsonschema-specifications==2024.10.1 ; python_full_version >= '3.9'
packaging==24.1
pbr==6.1.0 ; python_full_version >= '3.9'
referencing==0.35.1 ; python_full_version >= '3.9'
requests==2.32.3 ; python_full_version >= '3.9'
rpds-py==0.20.0 ; python_full_version >= '3.9'
rucio-clients==35.4.1 ; python_full_version >= '3.9'
stevedore==5.3.0 ; python_full_version >= '3.9'
tabulate==0.9.0 ; python_full_version >= '3.9'
typing-extensions==4.12.2 ; python_full_version >= '3.9' and python_full_version < '3.11'
urllib3==2.2.3 ; python_full_version >= '3.9'

Thanks, that does look like a bug.


🌍 : Add Hebrew Translation

Summary
Add Hebrew translation
Change Type

[X] New feature (non-breaking change which adds functionality)
[X] This change requires a documentation update
[X] Documentation update
[X] Translation update

Testing
TEsted locally
Test Configuration:
Checklist

[X] My code adheres to this project's style guidelines
[X] I have performed a self-review of my own code
[ ] I have commented in any complex areas of my code
[X] I have made pertinent documentation changes
[X] My changes do not introduce new warnings
[ ] I have written tests demonstrating that my changes are effective or that my feature works
[X] Local unit tests pass with my changes
[ ] Any changes dependent on mine have been merged and published in downstream modules.

Well, based on your comments, (and my initial version...) I started to think about the following.
There are actually 2 different needs for order.

For a developer the order should be the easiest for developing needs.  In other words, in a key-value pairs file like the above, it might be sorted keys.
For the end user the only order that matters is the UI, where sorted keys might work but a better one will probably be automatic language detection or common languages first.

AFAIU your criteria for the order is the order of adding to the code base, it is correct? however, this criteria while being logical to keep the history, is less appealing to a newcomer :-), and might not work for the long run.
What do you think?

Well, based on your comments, (and my initial version...) I started to think about the following.
There are actually 2 different needs for order.

For a developer the order should be the easiest for developing needs.  In other words, in a key-value pairs file like the above, it might be sorted keys.

For the end user the only order that matters is the UI, where sorted keys might work but a better one will probably be automatic language detection or common languages first.

AFAIU your criteria for the order is the order of adding to the code base, it is correct? however, this criteria while being logical to keep the history, is less appealing to a newcomer :-), and might not work for the long run.
What do you think?

Yes, there isn’t an official criteria for this yet. We’re considering automation due to numerous missing translations. For now, either move them down or leave them as is. Place your language in the general settings at the bottom, as it’s not a commonly used language. I’ll consider a more organized approach later. Thank you for your suggestions

I already changed the code per what you asked, it was only thoughts...
The only difference between us is the line
com_nav_language: 'שפה',
which you suggested to erase - and I asked about it above,  So you want me to erase it?


504 on gene-namedthing

This query from NCATS (https://github.com/NCATSTranslator/testing/issues/12):
{
"message": {
"query_graph": {
"nodes": {
"n0": {
"id": "HGNC:3236",
"category":"biolink:Gene"
},
"n1": {
"category": "biolink:NamedThing"
}
},
"edges": {
"e01": {
"subject": "n0",
"object": "n1"
}
}
}
}
}

Is causing a 504 error in strider

We have been working on some performance improvements for queries with named things and related-to edges, on the planning end. However KP calls can still be slow and we don't like to enforce timeouts on them. Probably we should do a little profiling to see if there are particular KPs holding us up or something.


Clicking the Short URL generated by Branch.io  is not opening my iOS App

Hello,
I have integrated the Branch SDK through Cocoapods in my iOS App.
Branch.io settings:
Settings -> Link Settings -> I have checked these three options,
1. Always try to open app
2. I have an iOSApp -> Updated the values of "iOS URI Scheme" & "Custom URL"
3. Enable Universal Links -> Updated the values of "Bundle Identifier" & "Apple App Prefix"
Finally, I got a short URL generated by Branch.io
iOS App configuration:

Added a branch_key in Info.Plist
Added a URL scheme in the Info.plist as,
Enabled the Associated Domains in the Capabilities tab & added the "applinks:bnc.lt" as an entitlement
Also enabled the Associated Domains in the Apple Developer portal

I have followed those above steps for integrating the branch.io in my iOS App for the deep linking to work. I have tried deployed the App through Xcode & also tried deployed the App through Adhoc provisioning. But that short URL is not opening My iOS App
Please do the needful on this issue.
Thanks,
Govindakarthikeyan

@Govindakarthikeyan could you please send us your branch key? If you prefer, you can also send it to<EMAIL_ADDRESS>and we'll take it from there.

Hi ahmednawar,
Here is my branch key "key_live_iom1TfqjE2ONMpUY9EocahkcCtnckLq2"

@Govindakarthikeyan I see that you sent a ticket to support with the same issue, let's continue the discussion there since there is more info.


Fixes #317, inlining bugs

Stop removing side-effecting init of unused locals.

Stop removing unassigned-but-used locals. This is accomplished by moving the unassigned removal from the "simple" inlining to the "safe" inlining.

tests/unit/inline.frag lines 144-153 can be restored.


QUESTION: How is Laravel 5.5 support

I am comparing chart packages, wondering how your L5.5 support is

We are currently upgrading all of our packages to L 5.5. The charts are already upgraded, check the 1.3.0 tag.


Permissions to allow shops and wand upgrades?

Hi, I disabled spells in my whole world, but I wanna enable wand shop by enchantment table to let players buy new enchantments on /warp enchant, which is on spawn with disabled magic (due to mines). I also maybe wanna allow some spells or just block list some of them. Is it possible too?

Your two options are WorldGuard flags or per-world-permissions. You just need to deny access to all spells except the progressmenu, spellshop and anything else in the progressmenu you want them to have access to.

And what are exact permission nodes for progressmenu and spellshop? I didn't find it in plugin.yml. I don't use worldguard, so only per-world permissions, if you give me them.

Spell permissions are always Magic.cast.<spell> so Magic.cast.progressmenu. You'll need to -Magic.cast.*, I assume you've done this already, then add back in Magic.cast.progressmenu.
Depending on your permissions plugin this may get tricky, PermissionEx at least has a bug where negating and re-adding within the same permission group does not work, so you have to negate in some base group but then add the permissions back in using an inherited group. Hopefully your perm plugin is better than that, but I thought it worth mentioning.

And about permissions... do you suggest allowing just some of them (but there is so much of them... so don't know), or do you suggest me to enable all and just disable some?
If the second variant, which permissions do you disable in spawn world, where is also one area (mines), where is destroy true? I already noticed that there was problem with mine spell, so probably it, but do you suggest something else too?
Or will be better to disable every of them and enable just few of them? But then I won't give players whole freedom, but maybe it will be more safe.... to enable just for ex. beginner's spells and progressmenu.

I am not sure why you've got magic disabled in that mine world in the first place, nor why players need to be able to buy spells there if they can't use them- so I think this is all something you'll have to decide on your own!
If you do allow the progressmenu I would suggest allowing everything in it, and yes it'll  probably be like a dozen spells if you include all of the branches of the buff menu and all that.

OK, and are there some of spells that you suggest me to disable in spawn world? And btw it's not mine world, on spawn are just MINES, created using Blockregen plugin... where blocks are regenerating, there has to be destroy:true in Residence, so if they can't grief this area using some of spells? I already discovered that they can do with mine spell, but any other idea what to disable?

Ah, I see- and WorldGuard is not an option? It's strange that blockregen can't modify protected Residences.
Personally, my opinion is if you have to disable casting of all spells in that world, there isn't really a reason for players to purchase spells there. But if your players are asking for it, for whatever reason, then perhaps just enable progressmenu and spellshop for now and see how that goes.

Yeah I also think it's strange, but author doesn't want to change it, already wrote him about it. They have own protection, but it's not a good solution. And if I enable all spells, is somewhere list that can cause destroy damage to area?

No, almost all spells can change blocks. It'd be pretty pointless to try to sort through it. If your players ask for specific ones (like maybe heal, boon?) you could add a whitelist but that's really up to you.

Yeah, I can enable some of spells... but now I tried to add spellshop and progressmenu permission and players can't even open categories to buy new spells :/ So it's useless probably?

Maybe you could way to just purchase them without permission on using them?

Haha- whoops, yeah I guess the spellshop filters out any spells a player does not have permission to cast. I could add a parameter to bypass that if that'd help?

spellshop:
  parameters:
     show_no_permission: true

In spells.yml, using the latest dev build.
Also note that dev build has some hopeful improvements to MySQL connection robustness, but do please test it out first if you can since I don't have a setup to test that myself.

Ok, solved yesterday, thanks. Will test new MYSQL connection robustness.


Video not playing in Android which was taken from iOS vice versa

Hi,
I am using Nativescript-Videoreorder & Nativescript-Videoplayer
"nativescript-videoplayer": "^3.1.0",
"nativescript-videorecorder": "^3.0.0-alpha.6",
Video which was recorded using Android is not Playing in iOS. Same thing happens when we record video in iOS it is not playing in Android. In both the scenario videos are playing in same OS.
Please help us to resolve this issue.
Regards
Madhan

What are the video formats? Knowing that, we can check if the native media players support the file extension. If it does, then we can check the player to add the code necessary to support it.

Closing as this is more OS and recording related and not playback.
See this SO post. Changing the encoding of the recording if you are recording inside your app would be a quick fix, or modifying your backend to handle the change.


Provide typescript declarations for tronweb package

We will estimate the work we need to do to support typescript. Please stay tuned to our progress. By the way, your code is appreciating.

is there a roadmap for typescripts support?


Mango test receiving 404/401?

https://builds.apache.org/blue/organizations/jenkins/CouchDB/detail/master/127/pipeline
./test/build/test-run-couch-for-mango.sh \

[ * ] Setup environment ... ok

[ * ] Ensure CouchDB is built ... ok

[ * ] Prepare configuration files ... ok

[ * ] Start node node1 ... ok

[ * ] Check node at http://<IP_ADDRESS>:15984/ ... {"couchdb":"Welcome","version":"2.1.1","features":["scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

ok

[ * ] Running cluster setup ... ok

[ * ] Developers cluster is set up at http://<IP_ADDRESS>:15984.

Admin username: testuser

Password: testpass

Time to hack! ... ESSS................................................................SSSSSSSSSSSSSSSSSSSSSSSSSS...........SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS.SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS....................................SS......SSSSSS..........

======================================================================

ERROR: test suite for <class '01-index-crud-test.IndexCrudTests'>

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/usr/lib/python3.4/site-packages/nose/suite.py", line 210, in run

    self.setUp()

  File "/usr/lib/python3.4/site-packages/nose/suite.py", line 293, in setUp

    self.setupContext(ancestor)

  File "/usr/lib/python3.4/site-packages/nose/suite.py", line 316, in setupContext

    try_run(context, names)

  File "/usr/lib/python3.4/site-packages/nose/util.py", line 471, in try_run

    return func()

  File "/tmp/tmp.qSlopPxdoL/apache-couchdb-2.1.1-28fae4d/src/mango/test/mango.py", line 248, in setUpClass

    klass.db.create(q=1, n=3)

  File "/tmp/tmp.qSlopPxdoL/apache-couchdb-2.1.1-28fae4d/src/mango/test/mango.py", line 78, in create

    r.raise_for_status()

  File "/usr/lib/python3.4/site-packages/requests/models.py", line 935, in raise_for_status

    raise HTTPError(http_error_msg, response=self)

nose.proxy.HTTPError: 401 Client Error: Unauthorized for url: http://<IP_ADDRESS>:15984/mango_test_7499828ea3ff4db7a2eb7ea91d4570e3?q=1&n=3

-------------------- >> begin captured logging << --------------------

urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): <IP_ADDRESS>

urllib3.connectionpool: DEBUG: http://<IP_ADDRESS>:15984 "GET /mango_test_7499828ea3ff4db7a2eb7ea91d4570e3 HTTP/1.1" 404 58

urllib3.connectionpool: DEBUG: http://<IP_ADDRESS>:15984 "PUT /mango_test_7499828ea3ff4db7a2eb7ea91d4570e3?q=1&n=3 HTTP/1.1" 401 67

--------------------- >> end captured logging << ---------------------

This is especially odd, given Couch was running just fine for a bunch of the tests...then suddenly rejects access.
Is something in the mango test suite changing auth credentials for validation and perhaps not waiting for those changes to take effect?
/cc @nickva @tonysun83

Very odd. The Mango tests use the same credentials for all requests (and
the user must be a cluster admin). For CI, this is hard coded to
testuser:testpass.
On 30 Oct 2017 7:53 pm, "Joan Touzet"<EMAIL_ADDRESS>wrote:

https://builds.apache.org/blue/organizations/jenkins/
CouchDB/detail/master/127/pipeline
./test/build/test-run-couch-for-mango.sh \
[ * ] Setup environment ... ok
[ * ] Ensure CouchDB is built ... ok
[ * ] Prepare configuration files ... ok
[ * ] Start node node1 ... ok
[ * ] Check node at http://<IP_ADDRESS>:15984/ ... {"couchdb":"Welcome","version":"2.1.1","features":["scheduler"],"vendor":{"name":"The Apache Software Foundation"}}
ok
[ * ] Running cluster setup ... ok
[ * ] Developers cluster is set up at http://<IP_ADDRESS>:15984.
Admin username: testuser
Password: testpass
Time to hack! ... ESSS................................................................SSSSSSSSSSSSSSSSSSSSSSSSSS...........SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS.SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS....................................SS......SSSSSS..........
======================================================================
ERROR: test suite for <class '01-index-crud-test.IndexCrudTests'>

Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/nose/suite.py", line 210, in run
self.setUp()

File "/usr/lib/python3.4/site-packages/nose/suite.py", line 293, in setUp
self.setupContext(ancestor)

File "/usr/lib/python3.4/site-packages/nose/suite.py", line 316, in setupContext
try_run(context, names)

File "/usr/lib/python3.4/site-packages/nose/util.py", line 471, in try_run
return func()

File "/tmp/tmp.qSlopPxdoL/apache-couchdb-2.1.1-28fae4d/src/mango/test/mango.py", line 248, in setUpClass
klass.db.create(q=1, n=3)

File "/tmp/tmp.qSlopPxdoL/apache-couchdb-2.1.1-28fae4d/src/mango/test/mango.py", line 78, in create
r.raise_for_status()

File "/usr/lib/python3.4/site-packages/requests/models.py", line 935, in raise_for_status
raise HTTPError(http_error_msg, response=self)

nose.proxy.HTTPError: 401 Client Error: Unauthorized for url: http://<IP_ADDRESS>:15984/mango_test_7499828ea3ff4db7a2eb7ea91d4570e3?q=1&n=3
-------------------- >> begin captured logging << --------------------
urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): <IP_ADDRESS>
urllib3.connectionpool: DEBUG: http://<IP_ADDRESS>:15984 "GET /mango_test_7499828ea3ff4db7a2eb7ea91d4570e3 HTTP/1.1" 404 58
urllib3.connectionpool: DEBUG: http://<IP_ADDRESS>:15984 "PUT /mango_test_7499828ea3ff4db7a2eb7ea91d4570e3?q=1&n=3 HTTP/1.1" 401 67
--------------------- >> end captured logging << ---------------------
This is especially odd, given Couch was running just fine for a bunch of the tests...then suddenly rejects access.
Is something in the mango test suite changing auth credentials for validation and perhaps not waiting for those changes to take effect?
/cc @nickva @tonysun83
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/couchdb/issues/935, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAF9-yXF0UX_7ppGfuUUM3AFjEevLfbxks5sxikbgaJpZM4QLwjO
.

Haven't seen this in months - closing.


pushing commits with tags takes a very long time

The server is stuck and not responding if commit includes tags. Normal clone with this repository takes 30seconds; push with tags takes between 20-30 minutes.
bonobo version: 6.0.3
running on windows 7

on bonobo, I see warnings like this:
2017-09-19 13:58:47.005 -07:00 [Warning] GitAuth: No auth header, anon operations not allowed

Here is what creating a tag and pushing it looks like:

then the server hangs: there is no response from the server.


Tidy up GitLab YMLs

Remove runner examples.

Should be good to go now, you can view all the commits in a squashed view via the 'Files Changed' tabbed for easier review.


Google Maps navigation not working

It seems that providing the saddr param isn't working on latest google maps. I just left it out and then navigation from my current location worked fine.
https://developers.google.com/maps/documentation/ios/urlscheme
For example:
NSString *dAddrVal = [NSString stringWithFormat:@"%f,%f", end.coordinate.latitude, end.coordinate.longitude];
NSString *url = [NSString stringWithFormat:@"comgooglemaps://?daddr=%@&directionsmode=driving",
dAddrVal
];

I had the same problem but I found that the issue was the format of the daddr param. I removed the destination map point's name and it worked fine. I don't think the latest Google Maps supports the name of the destination in the format that it is currently being created in googleMapsStringForMapPoint: mapPoint.
My quick fix was just this (Google Maps guessed the location name based on the lat/long, so that won't always be perfect, in my case it was fine):
[CMMapLauncher launchMapApp:CMMapAppGoogleMaps
                        forDirectionsTo:[CMMapPoint mapPointWithName:nil
                                                          coordinate:myCoordinate]];


Two gateways with phoscon/deConz - How to "ui" into the second?

Following situation after migrating from homebridge-hue to homebridge-deconz:
I have two gateways (raspberry's) with raspbee/deconz running. Only one of them has Homebridge / homebridge-deconz installed (same way as it was previously with homebridge-hue). Devices from both gateways are being properly exposed in homekit.
But:
I can perfectly "-ui" into the gateway with the homebridge installed to change loglevels etc. but I dont seem to be able to access the other gateway (the one without homebridge).
Any way t oget around this? I dont really want to install homebridge on the second device.
Many thanks

ui get /gateways lists the gateways.  Then use ui get /gateways/id/devices to list the devices for a given gateway.

Many thanks. That gave me the accessories of the second gateway. I cn update them.
Now trying to put LogLevel 0 against the second gateway.  I can update devices themselfs
Based on your example in dynamic config, I use ui-D put /gateways/ID/ '{"logLevel": 0}
However this does not return nything.

The /settings is only needed when calling the API directly through curl or Postman.   It’s just ui -D put /gateways/id ’{"logLlevel": 0}’

I tried that as well. this gets me stuck on that little "arrow" out of which I can only exit via CTRL+C
'ui -D put /gateways/id ’{"logLevel": 0}’.  Response: '>'

Your s8ngle quotes don’t match.  Make sure they’re not beautified.

Oh, dear...   Corrected that one, but different Error message now.  Typed manually into terminal to avoid further beautifications.:

You put a space in between the resource, causing ui to see the second part as the (invalid json) body.

Here we go!  Thank you very much!

Again issues with ui, I am afraid.  After I test that everything is working, I moced from my old raspi3/raspBee1 to a new raspi4/raspBee2 setup.  All is working fine.  Zigbee devices all showing up. But I want to stope that the new gateway (raspbee2) throws so many log entries.   Did a ping against the gateway and received a "pong" back.
But trying to set loglevel via "ui" gets me this error. I checked for beautifications, but all looks fine to me:

Any idea whats wrong? Could there be a rights issue?


Blank preview in settings

Instead of thousand words

Windows' part and screensaver itself seem to work just fine, thanks for your work

Hi @Lyapsus.
Thanks for your post.
While I am not exactly sure, I think you might be referring to the large box in the top screenshot? If you select the World Times radio button, that box is enabled and is used to configure the cities that you want displayed on your World Times board. If you don't add any cities, a default set is displayed.
I can see how that might be confusing though and will try to think of a way to make it clearer.

 All right then, thanks again for your creation.
Dunno if you prefer to leave it open not to forget so I'll better leave it as is.


[Feature] Change order of properties

Needs: (skip)

Problems: Can't change order of properties! Whenever I want to insert a new property between properties, I have to use 'alter query'.

Solutions: I don't expect that like 'drag and drop'. I just want to change my properties's orde by GUI.

Examples: Below is HeidiSQL which offers buttons to change properties's order.

Hi @555cider,
its already possible by drag and drop. The draggable area is the "::::" icons before field number.

Sorry, I didn't notice it. And thanks for making a great sevice.


[ENHANCEMENT]   Add Digital Ocean (once they publish some meaningful data)

Is your feature request related to a problem? Please describe.
We have a number of providers who use Digital Ocean, and want their sites to be marked as green, and have in good faith chosen regions that you might expect to be 'green', or running on green energy.
Describe the solution you'd like
We'd hear directly from Digital Ocean about which of their regions are cited in datacentres where the emissions from the use of the digital infra is accounted for. We outline what we accept below:
https://www.thegreenwebfoundation.org/what-we-accept-as-evidence-of-green-power/
Describe alternatives you've considered
We offer a mechanism for the downstream provider to assume responsibility suppliers 'upstream' to them, if they can quantify the emissions and demonstrate they are taking steps to account for them, as we outline above.
https://www.thegreenwebfoundation.org/support/im-using-a-cloud-provider-why-is-my-site-showing-as-grey/
We have a very high level estimation tool in the form of CO2.js to get an idea of carbon emissions, but the better scenario would be to extend something like Cloud Carbon Footprint to support Digital Ocean's line of services.
This would allow any customer of DO to quantify and account for the emissions based on their actual usage data, rather than a top down model that might be making very different assumptions to the reality of how DO's services are used.
Additional context
Here's the response passed to us from the Digital Ocean support representative when the customer asked this:
Hey there,

Thank you for taking the time to raise a case with us!
I understand you'd like to know if it is possible to green list your domains.
Upon reviewing your account, I see that all your Droplets are in the FRA1 region. We do not have official documentation on our end. However, I can give you the details of our DC in the FRA1 region.
Our DC provider for FRA1 is Interxion [REDACTED ADDRESS]. Here are the details of their sustainability: https://www.interxion.com/why-interxion/sustainability
I hope this helps! Let us know if you need any further assistance from us. We're here to help!
Kind Regards,
REDACTED SUPPORT STAFF NAME

If we know which regions are in regions where the DC operators are already accounting for the emissions, we can mark them as green, but we need this from DO themselves, so we know we're not misrepresenting them.

Hi,
Digital Ocean has been listed successfully, but right now it appears again as not whitelisted? Could you take a look?


Why i cannot get the first dimension of the features' shape?

Thanks your work!
In the D3Feat.py, i print the shape,
def assemble_FCNN_blocks(inputs, config, dropout_prob):
    # First get features from CNN
    F = assemble_CNN_blocks(inputs, config, dropout_prob)
    features = F[-1]
    for f in F:
        print("the feature shape ", f.shape)

but get the below:

but i see the dimension of the features in paper are : 64 128<PHONE_NUMBER>

it is the feature of TensorFlow. you can use tf.Print to printout the real value of features

Thanks，i get it


ed_pixel.h: The file or directory does not exist

Hi I'm trying to implement this proyect but I have a issue, the library ed_pixel.h isn't in the proyect, please can you help me?

https://github.com/kinap/canny-edge-detector/blob/master/inc/ed_pixel.h
Gonna need more information about your build environment.


Small imrovements

Added possibility to set a separator for HTTP groups header (i.e. Authentik uses '|')
Fixed wrong selection of headers into username/email
Switched condition checking for user to look into username instead of uid
P.S. Ideally, admin_userids should be admin_users and use username too

Added possibility to set a separator for HTTP groups header (i.e. Authentik uses '|')

good idea

Fixed wrong selection of headers into username/email

well spotted!

Switched condition checking for user to look into username instead of uid

I don't agree with this one. Why would you like to use the username as identifier?
A userID is supposed to be unique, but multiple users can share the same username. Sometimes users can even change their own username. So in my opinion the username should never be used to control permissions.

I don't agree with this one. Why would you like to use the username as identifier? A userID is supposed to be unique, but multiple users can share the same username. Sometimes users can even change their own username. So in my opinion the username should never be used to control permissions.

Yes, I agree that UID is a unique identifier and should be used to store information about the user. But I'm not sure that writing something like allow: [user: b8688e276996ae4686ed3794f80ca5324cadfd13a40059124dbef1825815455e] is user friendly option

Maybe it could have a switch?
If this were used in large organizations, no, we can't allow usernames to identify allowed options. But as it's geared towards home users, who have in most cases 1 admin who manages either Authelia or Authentik or whatever else option is out there, it's probably safe to assume, that it would be more easier to supply usernames instead of uid for options

I still think that this is not the right usage of the username. But I totally see your usability concern. The admin can just use the mail addr or group membership instead:
allow: [user:b8688e276996ae4686ed3794f80ca5324cadfd13a40059124dbef1825815455e]  # not so nice
allow<EMAIL_ADDRESS># better
allow: [group:admins]  # even better (using groups is the preferred way)

Maybe we can add allow: [username: mycoolusername] ?

The username: selector is  a good compromise, I've added it

Sweet, thanks!


accessing properties on types causes assembly loads

i would expect that once i've loaded an assembly, it's resolved all dependent assemblies and ensured that they are resolvable. the lazy loading of the assemblies makes it easy to run into issues further down the road when using this API. it also seems to require all APIs to be marked with throws which makes it cumbersome to use

That is a sensible expectation, but the decision is by design for this implementation. I think there are two fundamental ways to design this library:

Load everything ahead of time in an in-memory representation that is fully validated, such that no errors can happen when accessing members.
Load everything lazily and raise errors as they are encountered.

1's pros is the API has a single throwing point and the assembly file handle can be closed after reading. 1's cons is that it is only suitable for batch operation since it incurs a large upfront cost, especially for assemblies like Windows.winmd or mscorlib.dll. It is also difficult to report multiple issues rather than only the first encountered, and it might require implementing larger chunks of the file format at once to be able to return a value.
Conversely, 2's pros is that it supports building more targeted tools that don't need to load everything without incurring a large up-front cost, it allows handling assemblies having some bad data somewhere and reporting errors piecemeal as they are encountered, makes it easier to implement part of the file format. 2's cons are that all members become throwing functions and the assembly must be kept open (but it can be memory-mapped).
I wanted to make this library usable in streaming scenarios, with recoverable just-in-time errors so I went with option 2. The design also mimics System.Reflection.
I don't think hybrid approaches would make sense since it would be difficult to isolate throwing functions to just a few relevant places.


Relationship Fields are empty when editing in drawer

Bug Report
When editing a relationship field inside a collection the fields are empty.
Steps to Reproduce

Add a relationship field inside a collection
try to edit the relationship collection inside of the parent

Other Details
Payload 1.6.6

@raffaeleparisi Thanks for the report, I was able to reproduce 👍

Hey @raffaeleparisi — this is fixed and will be released shortly.


C#: Specify a runtime rollforward for OpenVisualStudio

We use LatestMajor in order to allow users that only have .NET superior to 6 installed to still run the project.
Fixes #91638

Thanks!


Spike - Collect all the issues related to environment issues

For #8584
https://github.com/Microsoft/vscode-python/blob/b4499f03d9ace2a6b2c309ebcf5f746f6db39e99/src/client/debugger/extension/configuration/resolvers/helper.ts#L53
While resolving a debug configuration, this line in add code merges all of the process environment variables with the custom vars. So all of environment variables are sent to the Debug console.
New debug adapter also uses debug resolver to resolve configuration, so it also runs into this issue.

For the issue #8584 , this is caused by a change in what we used to consider as internalConsole.  Previously, none meant internalConsole, and console type had ["none", "integratedTerminal", "externalTerminal"] , but some time this year, that changed to ["internalConsole", "integratedTerminal", "externalTerminal"]. There is no "none" console type.
https://github.com/Microsoft/vscode-python/blob/b4499f03d9ace2a6b2c309ebcf5f746f6db39e99/src/client/debugger/extension/configuration/resolvers/helper.ts#L49
When we see console is not defined, we assume internalConsole in the debugger. But the extension  assumes absence of console as integratedTerminal, and overwrites the console value. So debugger receives full set of environment variables (instead of just the diff), and uses it in integratedTerminal.
https://github.com/microsoft/vscode-python/blob/b4499f03d9ace2a6b2c309ebcf5f746f6db39e99/src/client/debugger/extension/configuration/resolvers/launch.ts#L76-L78

couple of other issues with that code:

It should not be setting utf-8 encoding. At least for the new debugger.
gwent support will be set in the new debugger.
The extension should not merge PATH and PYTHONPATH. This limits usage since they will always be additive. Instead they should override, that way users can override or provide combined paths as needed.

@int19h /cc


[RFC] Building a GitHub Automation App for OpenSearch GitHub Org

[RFC] Building a GitHub Automation App for OpenSearch GitHub Org
Introduction
As the OpenSearch organization continues its journey towards a more open and transparent future, we have faced several operational challenges that require manual steps by GitHub Org Admins. As OpenSearch is moving towards to a foundation model that focused on scalability, efficiency, and transparency, we want to solve these challenges with a permanent solution. This RFC proposes developing a GitHub Automation App specifically for the OpenSearch GitHub Organization to handle automation tasks on behalf of admins going forward.
Motivation
The OpenSearch project wants to create a transparent, open-source, and community focused development model. However, the manual processes we apply to manage the repositories in our GitHub Orgnization are not efficient enough to scale. For instance, if an issue is opened in the wrong repository, the repo maintainers must tag the opensearch-project/admins group in a comment and wait for someone to manually transfer the issue. Similarly, if PR authors forget to add a label that triggers a specific GitHub Action, a repo maintainer must step in, which further delays the review process. These kind of issues happen regularly. In order to address the challenges, we have created a detailed Problem Statement section that outlines the specific issues we face and proposed solutions based on the GitHub Automation App. By automating key tasks through the App, we can enhance the efficiency of the repository management, reduce the dependencies on admins/maintainers, and build a more seamless collaboration environment.
Tenets

Open-source: Engage community and ensure source code is publicly accessible and reviewed during development
Security: Have appropriate authentication and authorization mechanism for each user request
Usability: Clear documentations and SOPs for users to follow, with user-friendly interface
Efficiency: Automate repetitive / complicated tasks on behalf of admins
Scalability: Apply modular design to handle different automation tasks and operations
Performance: Ensure responsive operations to support increasing amount of demands
Availability: Provide users with reliable service and abilities to inform user about potential maintenance
Transparency: Provide clear visibility into the talks that the app is processing
Integration: Integrate with GitHub and other tools used by the organization natively

Problem Statement
1. Issue Management Automation

(1.1) Manual Issue Transfers: Transferring issues between repositories is currently a manual process, where users must tag the @opensearch-project/admin group, and an admin must manually transfer the issue. This process is time-consuming and can lead to delays in issue resolution.

https://github.com/opensearch-project/opensearch-build/issues/4505

(1.2) Add RFC/Meta Issue to Roadmap: Assigning new RFCs or Meta issues to the appropriate roadmap project board and categorizing them is currently a manual process. Several issues are missed on the new OpenSearch Roadmap because of this.

https://github.com/opensearch-project/.github/issues/196

(1.3) Merging Backport PRs and Remove Merged Branches: The existing Backport workflow can occasionally cause friction when a maintainer forgets to merge a PR. Moreover, repositories rules require that all PRs be "approved", including Backports, which means an extra-step to merge a Backport PR even when that PR has passed all checks.

https://github.com/opensearch-project/opensearch-build/issues/4608
A proof of concept of using github actions to achieve auto-merge with a followup branch removal  has been implemented since, but still requires changes to all the repos to take effects:

https://github.com/opensearch-project/documentation-website/pull/6893

The App will automate issue management tasks, such as transferring issues between repositories, assigning RFC/Meta issues to the roadmap project board, auto-merging backport PRs when GitHub Checks pass, etc. Make sure issues are efficiently and consistently handled.
2. Label Management and Documentation Support

(2.1) Enforce Documentation: There is no current enforcement mechanism to ensure that developers to open a doc issue on documentation-website repo before release, leading to documentation gaps and delayed involvement from tech writers.

https://github.com/opensearch-project/documentation-website/issues/6365
https://github.com/opensearch-project/opensearch-build/issues/4455

(2.2) Automate Labels: Ensuring that the appropriate labels are added to issues and PRs or get created on demand.

The App will enforce the use of labels, such as need-documentation, and automate the addition of labels based on user requirements. The label will then trigger issue creations on documentation-website repository. This will ensure that documentation requirements are raised early for quick follow-up before the release process happens.
3. Permission and Access Control Management

(3.1) Manage Repo Access: When team members leave, or new maintainers are added, their GitHub access must be manually updated by other maintainers. This activity can be missed or delayed overtime. Also, verifying a potential new maintainer's contributions and organize community voting process is cumbersome at the moment.

The App will automate access control management by moving departing members to an Emeritus section, removing their access, and making PRs and announcements. When a new maintainer gets nominated, the app will assess their contributions and start a community voting thread, ensuring that access control is properly managed.
4. Executor of Metrics Project

(4.1) Metrics Dashboard Integration: We currently have a public metrics dashboard that uses a metrics cluster as backend. However, there is no integrated way for users to interact with these metrics from within GitHub, nor is there an easy way to display important information during release phases, such as build status, test reports, or health status of each plugins as part of the release progress.

The App will serve as the frontend executor of the metrics cluster, acting as a bridge between users and the backend metrics cluster. It will provide an interface to display useful metrics on demand and showcase important information during release phase. We can think of other use cases as well.
5. Bulk Operations Across Multiple Repositories

(5.1) Multi-Repository Updates: Making the same changes across multiple repositories requires either asking each repository’s maintainer to implement the changes, or assigning one person to manually open PRs across all repositories. Using the GH client is also not efficient as only admins with access to multiple repositories can do so.

The App will enable bulk PR creation across multiple repositories. This will simplify the process of implementing organization-wide changes, ensuring consistency and reducing the time required to manage multi-repo updates.
Proof of Concept
In the past few months, we have made a few proof of concept GitHub Apps to tackle issue management automation.
Auto Issue Transfer:

A user creates an issue on Repo A
Repo A maintainer discovers that the issue should belong to Repo B
Repo A maintainer adds a label called Transfer to Repo B to the issue
The App detects issues.labeled event on the issue, verify label content, and identify that Repo B as destination
The App verifies that the user who added the label is a maintainer of Repo A
The App transfers the issue to Repo B, and comments on the issue about the transfer
Example: https://github.com/opensearch-ci-bot/test-docs-repository/issues/17

Add RFC/Meta Issues to Roadmap Project:

A user creates a RFC/Meta issue on Repo A with RFC/Meta issue template
The template enforce user to add RFC or Meta label upon issue creation
The App detects issues.labeled event on the issue, verify label content as either RFC or Meta
The App adds the issue onto the OpenSearch Roadmap Project
Repo A maintainer add a follow-up label Roadmap:Security since the issue is related to Security

Note: We will enforce issue creator to select the roadmap entry from a drop-down list upon issue creation later

The App detects issues.labeled event on issue, verify label content, and identify that Security as field entry
The App assigns the issue with OpenSearch Roadmap field, with the value of the field being Security
Test Issues:

https://github.com/peterzhu-organization/github-test-1/issues/4
https://github.com/opensearch-project/opensearch-build/issues/4936

In Practice:

https://github.com/opensearch-project/OpenSearch/issues/12602

Opportunities
While the RFC outlines several key places in which the App can help improve the repo management, we think that there could be additional opportunities for enhancement. We invite the community to propose more use cases and features that could be added into the GitHub Automation App.
Things to consider:

Are there any other manual processes that could benefit from the automation?
What additional features would make this app be more useful to the community?
Are there specific metrics that could be displayed or easily accessible with this app?

We encourage the community to provide feedbacks and suggestions by commenting on this RFC issue. Let us know what you think.
Next Steps
We will go ahead and create Meta issues and Design Proposals to the public, and start working on the App based on the aforementioned Proof of Concept.
Conclusion
By automating key tasks with the GitHub Automation App, we could further reduce manual intervention, formalize process, and improve transparency. The app will create a more efficient, scalable, and contributor-friendly environment.
Thanks for reading.

Wondering if it would be too many labels to transfer issues from one repo to another. How about somethings like @app please transfer this issue to security repo Only when the app is mentioned, the event can be triggered making it push based model? We don't have to keep monitoring the label events. Also most maintainers currently tag admins in similar fashion for the transfer. So wouldn't be much of a change from user perspective.

Wondering if it would be too many labels to transfer issues from one repo to another. How about somethings like @app please transfer this issue to security repo Only when the app is mentioned, the event can be triggered making it push based model? We don't have to keep monitoring the label events. Also most maintainers currently tag admins in similar fashion for the transfer. So wouldn't be much of a change from user perspective.

Thanks for comment.
Note that the label transfer method is just a POC and we have since reviewed it and decided to use other methods.
The section above is to show case what the app is capable of, will definitely improve the functions later.

agree with @gaiksaya on too many labels. I have a few questions:
1. Is the code logic written in app? Where can I see the code for PoC?

2. Is it always listening to all the github events across all the repos?

3. wrt auto merge of backport prs, would it still require approval or not?

Hi @rishabh6788 ,
Thanks for commenting.

The POC code is currently in a private fork, and I will publish it in a new repo when ready.
We can define exactly which event to listen to on which repo. As of now it is only listening to issues.labeled event on all repos of opensearch-project for adding issues to roadmap.
The app will backport then approve the PR. Once the checks pass as green it should merge it and remove the branch as well. As of now that behavior is created using github actions, which is cumbersome. By using the app to achieve this it is more intuitive and easy.

Thanks.

I am really excited to see a bot that does all of this!
A few things that are important.

One should be able to contribute workflows/tasks to the bot reliably. So code organization needs to have a single workflow that can be authored independently from other workflows.
The blast ratio of this bot are huge. It will have r/w admin-level access to the org. So we need a robust set of tests.

@peterzhuamazon I recommend moving whatever you have in a private repo to public as early as possible and starting with a very simple workflow so we can ensure that (1) and (2) are robust enough from the beginning.

I am really excited to see a bot that does all of this!
A few things that are important.
1. One should be able to contribute workflows/tasks to the bot easily. So code organization needs to have a single workflow that can be authored independently from other workflows.

2. The blast ratio of this bot are huge. It will have r/w admin-level access to the org. So we need a robust set of tests.

@peterzhuamazon I recommend moving whatever you have in a private repo to public as early as possible and starting with a very simple workflow so we can ensure that (1) and (2) are robust enough from the beginning.

Thanks @dblock for the suggestions here.
Yeah, we plan to get the source code out in a new repo soon.
As of now, the app is running with the poc code, and only focus on adding issues to project.
We should soon break the code into different layers of abstractions/objects so it can be easily maintained and extended on.
Thanks!

Thanks for your reply @peterzhuamazon, looks promising and excited to see what this app can do for us.
Not sure if it has already been considered, can we have an app that listens to all the major events, push, pull_request, issue_comment and label. The ones that are not in-scope of this project can be a no-op and we can implement logic for the ones we want. For e.g. for transferring we have an action performed when an admin or repo-maintainer adds a comment, such as transfer: <destination-repo> and the app just transfers the issue.
The same can be extended in future to probably add CHANGELOG entries to pull_requests across all repos that opt-in and other actions that you have already mentioned.
Also, it would be great if before the code PoC we can have a high-level design review on how the app design would look like and what all components are involved in its functioning. I have basic queries like, where this app will be running, will it be pull based or webhook based implementation etc.

Agree to @rishabh6788's suggestion. A global app that does everything. I believe more than listening (pull based model), push based model would be great where we tag the app and then the action item to carry.
For example:
@opensearch-ci-infra Transfer this issue to foo repo
@opensearch-ci-infra Add Changelog entry
@opensearch-ci-infra Run the performance test on this PR

In this way, you don't have to keep listening but just act on mention based events. We can make it simple by just keeping it comment based. All action items can follow the same pattern. It would be great to take the scaling and increasing scope into consideration. Starting small and incrementally increasing the AI would be the way to go.

Hi @rishabh6788 @gaiksaya ,
The current framework I am building is already taking care of the listen step, we do not need to worry about that implementation.
We can, specify exactly what we are listening so we are not overwhelmed by events.
Also, this is a global app framework, but each action would have its own listener, so they are not conflict and step over each other, so it can do all the things at the same time, while not compromise the performance here.
We do, however, would be limited by the github app quota, if all the actions are run under the same app id. If needed, we can create more github app entities, and wrap all under the same umbrella, and they can act coordinately together on our defined actions.
Thanks.

My recommendation was contradictory to this. Listening eventually will become overwhelming and result in lots of API calls across org. Was suggesting to use push based model where an app will act only when tagged (mentioned in GitHub terms).
We do however need to have robust checks as to who is mentioning the app but that is doable as well. Might need some more research on this front but would be much easier as comments are easy than labels/events which are permission based and might need time to be adapted by the community.
Based on the requirements and asks, it would be great to see the overall design with pros and cons so that we can make a call accordingly.

My recommendation was contradictory to this. Listening eventually will become overwhelming and result in lots of API calls across org. Was suggesting to use push based model where an app will act only when tagged (mentioned in GitHub terms). We do however need to have robust checks as to who is mentioning the app but that is doable as well. Might need some more research on this front but would be much easier as comments are easy than labels/events which are permission based and might need time to be adapted by the community. Based on the requirements and asks, it would be great to see the overall design with pros and cons so that we can make a call accordingly.

We are not just based on labels, all the event we can monitor.
The probot framework provide multiple ways to interact so we dont need to implement it ourselves.
As the framework is based on different actions, each action can be its own instance so there is no overwhelm cases. If the action is too big just split it into more fine-grained actions.

The code is now public for review:
https://github.com/opensearch-project/automation-app
Thanks.


[ QUESTION ] Resetting User Data

Question about resetting Player Data:
How do I reset all player data for the Magic plugin? I am trying to reset my world files and such so me and my server members can start fresh, but for some reason, we keep our magic info (that being our spells unlocked and sp and such)

Delete /plugins/Magic/data/players


Invite user modal fixes

Closes TryGhost/Ghost#7132

Adds tabindex -1 to the select span to allow the tab to go straight
from input to select
Adds a focus to the new user input on open

Needs the run later otherwise it is triggering too fast and not
getting the input

@dbalders another way you could go about this is to convert the gh-input into a gh-trim-focus-input. Those already have the focus functionality built in. See the signin form template for an example 😄

@acburdine kk, thanks, ill look into that

@acburdine the trim isn't liking the email at the beginning. Getting Cannot read property 'trim' of undefined in the sanitizeInput function. Any idea what to change that to?

so, i fix the problem from the last comment, but the field is still not taking focus without some sort of delay built in. The code for that works now, but obviously shouldn't stay the way it is. Any thoughts on this?

@dbalders ah, I see :)
So the trim-focus-input is doing what it's designed to do I believe, the issue is that it's trying to be focused before the component is fully rendered (I think).
Regardless, switching back to the regular input's probably good.

So I played around with this and did some googling, and it seems to me that because of the transitions that occur as part of the liquid-tether modal setup, setting the focus of the input using scheduleOnce('afterRender') just isn't possible (the input is technically in the DOM but not visible, so focusing fails). So - it appears using run.later is the better way to go about this 😄 /cc @kevinansfield
There is one issue with focusing the input on render (regardless of the way it's done) - when the user clicks on the 'X' close button - there is a brief flash of validation error on the field before the modal closes. It shouldn't be hard to stop that (see the closeModal method on the base modal and the current willDestroyElement of the invite user modal).
Also - would you be willing to write an acceptance/integration test for this?

@acburdine yep! all sounds good :). I will ping when its done

Oh @acburdine, regarding the validation error showing up, that is because of the onfocusout being run. Talked about that on my last comment on TryGhost/Ghost#7131. Waiting to see what to do for that. So, if its cool, may just do separate PRs. One for this, then one to fix whatever is decided in the other one? Does that sound like a good idea? Or, we can just solve the issue here and I can do both.

@acburdine how does this look?

Original issue resolved via https://github.com/TryGhost/Ghost-Admin/commit/4bf815dba4f8c803f4ffadb97c7d15a1ee87dc1d and https://github.com/TryGhost/Ghost-Admin/pull/1020


Examples don't work

I've tried to run the spectrogram and the plot input examples (Ubuntu) and they yielded similar errors:
  File "plot_input.py", line 48
    print(status, flush=True)
                       ^
SyntaxError: invalid syntax

------------------
(program exited with code: 1)
Press return to continue

and
  File "spectrogram.py", line 70
    print(*line, sep='', end='\x1b[0m\n', flush=True)
          ^
SyntaxError: invalid syntax

------------------
(program exited with code: 1)
Press return to continue

What am I missing here? What'#s the deal with 'Flush'?

Ok, I figured it out: I need to launch them with python3 . However I still get errors with the examples. Apparently numpy can't handle floating point values anymore. see this:
https://github.com/tomopy/tomopy/issues/252
Indeed, downgrading numpy helped helped: sudo pip install -U numpy==1.11.0

thanks for the report. To clarify, the issue is that numpy doesnt accept floating point values as an index

Yes, some examples don't work with Python 2, the sounddevice module itself should work on Python 2.6 and 2.7, though.
NumPy not accepting float indices is not a bug, this is an intentional change.
@alkopop79 Can you please provide the error message?

I've solved the original problems by downgrading numpy and running the examples with python3. I still get errors but I believe that has to do with settings on my system (Ubuntu running as VM on Mac OS X). I will give it a try on Mac OS X instead. Thank you for the kind help @mgeier and @tgarc, please feel free to close the issue!


Cannot authenticate to start or join a collaboration session

Whenever I try to use liveshare (through the left-side menu or F1 menu), it asks me to authenticate with github or microsoft, and once I've done it, I get the message
Starting collaboration session: The user code is invalid or expired. Try signing in again.
Using the code manually doesn't work as well. Tested on Chrome and Firefox.
What was your system configuration?
Product and Version [VS/VSCode]: VSCode for Windows
Version: 1.47.1 (user setup)
Commit: 485c41f9460bdb830c4da12c102daff275415b53
OS: Windows 10 Home
OS Version[macOS/Windows]: version 1909
Live Share Extension Version:  v1.0.2427
Target Platform or Language [e.g. Node.js]: Not relevant
Steps to Reproduce / Scenario:

Try to start/join a collaboration session
choose either github/microsoft auth
validate on your browser the auth part
See error

Please attach logs to this issue:
My logs are empty, cant upload empty files :/

Hi Vincent! Sorry you are experiencing this problem. Did you try restarting VS Code? also, can you check out some workarounds discussed in issue #3216 and see if you can get unblocked? thanks,

this seems to a duplicate of #3728


Does conversation transcription support python?

Does conversation transcription support python? Or if python speech sdk includes conversation transcription feature? Because here I only see javascript and C# examples. Thanks.

@hs79hs Thank you for the question. In order to best address your question, could you provide us with the URL of the MS Doc that you were following, if any?

@BhargaviAnnadevara-MSFT  Oh, sorry, forgot to provide that, it's
https://docs.microsoft.com/en-us/azure/cognitive-services/speech-service/how-to-use-conversation-transcription

@hs79hs Thank you for the response. I've routed this to the appropriate Team to investigate and update.

@hs79hs The documentation in this case displays samples for C# and JS. The SDK for python though supports it and the methods are documented in the SDK.
ConversationTranscriber
Conversation Class
Hope this helps to use the same for python with this sample. Thanks!!


test: avoid potential output conflict

Summary
Test files are parallelly, the build and inspect test file will conflict with each other. Not occurred on CI yet, But in theory it will and on my device locally now.
Related Links

Checklist

[x] Tests updated (or not required).
[ ] Documentation updated (or not required).

I think we can create unique folder for test cases, now it shared one folder to run.

I think we can create unique folder for test cases, now it shared one folder to run.

Damn, I'm wondering why I'm not doing this way.


Nice

Nice, but I hope you can add a search function to find videos based on keywords and then collect comments. Thank you.

noted

noted

Thanks for your reply, I look forward to seeing it soon.
And I hope that commenting on the list of related videos after searching for videos will be your next topic!
Thank you very much for your efforts!


app is not starting in emulator and showing this on console Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=com.tadashidate/.MainActivity }

Description
When I hit command npm run android . its saying app keep stoping
React Native Version
0.72.1
Output of npx react-native info
System:
OS: Windows 10 10.0.22621
CPU: "(16) x64 AMD Ryzen 7 5700U with Radeon Graphics         "
Memory: 1.96 GB / 13.86 GB
Binaries:
Node:
version: 18.12.0
path: C:\Program Files\nodejs\node.EXE
Yarn:
version: 1.22.19
path: C:\Program Files\nodejs\yarn.CMD
npm:
version: 8.19.2
path: C:\Program Files\nodejs\npm.CMD
Watchman: Not Found
SDKs:
Android SDK: Not Found
Windows SDK: Not Found
IDEs:
Android Studio: Not Found
Visual Studio: Not Found
Languages:
Java: 1.8.0_372
Ruby:
version: 2.7.5
path: C:\Ruby27-x64\bin\ruby.EXE
npmPackages:
"@react-native-community/cli": Not Found
react:
installed: 18.2.0
wanted: 18.2.0
react-native:
installed: 0.72.2
wanted: 0.72.2
react-native-windows: Not Found
npmGlobalPackages:
"react-native": Not Found
Android:
hermesEnabled: true
newArchEnabled: false
iOS:
hermesEnabled: Not found
newArchEnabled: Not found
Steps to reproduce
npm run android
Snack, code example, screenshot, or link to a repository

When I hit command npm run android . its saying app keep stoping

Can you clarify? Who is "it"?
Can you provide a screenshot?

terminal

emulator

We would need your app source code, as there is something going on over there.

here is the repo
https://github.com/hrutik7/Tadashi

I seem to have the same exact problem  @hrutik7  could you post how you solved this

hey @francis-Paul-code
the problem was in gradle files . I did a lot of changes in that that's why overall code was messed up
I will suggest look carefully on those android/ios files

@hrutik7 alright thanks for the feedback I'll try your suggestion


Preserving viewBox

Hi,
First of all, thank you for this amazing library, it is great!
With version 1.0.0, viewBox is dropped. In my case I have some illustrations that I inline, they are not icons but I still need viewBox to scale them.
I would be happy to contribute and help, but in order to understand where you come from: what is the rationale behind removing viewBox for "non-icons"?

Hi, after #17 I think adding an option « viewBox » and defaulting it to true is the best thing. I think viewBox doesn’t add a lot of data and give ability to resize, it is pretty common with SVG. @troch feel free to make a PR, else I will do it as soon as I have a moment.

For future googlers (like me). It seems the information in this issue is outdated, viewbox settings are now delegated to the svgoConfig. See https://github.com/smooth-code/svgr/issues/142
Webpack snippet:
{
        test: /(icons|images)\/.*?.svg$/,
        use: [{
          loader: '@svgr/webpack',
          options: {
            svgoConfig: {
              plugins: {
                removeViewBox: false
              }
            }
          }
        }, 'file-loader']
      }

@edorivai yes you are right, viewBox is svgo job!

Hi,
How to handle this in @svgr/rollup, I mean how to write this config in rollup

Hi,
How to handle this in @svgr/rollup, I mean how to write this config in rollup

For me this one works:
svgr({
  svgoConfig: {
    plugins: {
      removeViewBox: false
    }
  }
}),

https://github.com/webpack-contrib/image-minimizer-webpack-plugin/issues/190
{
    plugins: [
      {
        name: 'removeViewBox',
        active: false
      },
    ]
  }

Give a look to: https://react-svgr.com/docs/migrate/#svgo

Note, that in order to work now { removeViewBox: false } should be passed at array:
{
    test: /(icons|images)\/.*?.svg$/,
    use: [{
        loader: '@svgr/webpack',
        options: {
            svgoConfig: {
                plugins: [{removeViewBox: false}]
            }
        }
    }, 'file-loader']
}

New update on this one. If you encounter this error: Error: Plugin name should be specified, you need to update to the ff:
{
    test: /(icons|images)\/.*?.svg$/,
    use: [{
        loader: '@svgr/webpack',
        options: {
            svgoConfig: {
                plugins: [
                  {
                    name: 'preset-default',
                    params: {
                      overrides: {
                        removeViewBox: false,
                      },
                    },
                  },
                ],
            }
        }
    }, 'file-loader']
}

Source: https://stackoverflow.com/a/73089859/9608615


It seems that this project do not support window

Thanks for the question!


Add CephFS volume type

CephFS was missing, so lets add it.

@saad-ali assigning to you since it is Volumes

LGTM.


Error when comparison contains both Search terms and Topics

The error is "TypeError: Cannot read property 'requestOptions' of undefined"
This stems from the handling of`the parsed results.
The reason is because in Google's UI, and correspondingly in widgets JSON, there's a message at the top: "Note: This comparison contains both Search terms and Topics, which are measured differently."
In JSON, it looks like this
[
  {
    "text": {
      "text": "This comparison contains both Search terms and Topics, which are measured differently"
    },
    "url": "https://support.google.com/trends/answer/4359550",
    "id": "topics_note",
    "type": "fe_text",
    "title": "",
    "template": "fe_explore_note",
    "embedTemplate": "fe_embed",
    "version": "1",
    "isLong": true,
    "isCurated": false
  },
  {
    "request": {
      "time": "2016-08-26 2017-08-26",
      "resolution": "WEEK",
      "locale": "en-US",
      "comparisonItem": [
        {
          "geo": {},
          "complexKeywordsRestriction": {
            "keyword": [
              {
                "type": "ENTITY",
                "value": "/m/0128447t"
              }
            ]
          }
        },
...

As a result, the rest of the widgets get shifted by 1, so pos needs to be incremented as well.
A possible fix is to modify this try caluse of parseResults() to
var widgets = JSON.parse(e.slice(4)).widgets;
if (!widgets[0].request) { widgets.shift(); }
return widgets;

Oh boy. What method/request are you making to get this error so I can replicate it?

On Aug 25, 2017, at 22:17, Amit Sch<EMAIL_ADDRESS>wrote:
The error is "TypeError: Cannot read property 'requestOptions' of undefined"
This stems from the handling of`the parsed results.
The reason is because in Google's UI, and correspondingly in widgets JSON, there's a message at the top: "Note: This comparison contains both Search terms and Topics, which are measured differently."
In JSON, it looks like this
[
{
"text": {
"text": "This comparison contains both Search terms and Topics, which are measured differently"
},
"url": "https://support.google.com/trends/answer/4359550",
"id": "topics_note",
"type": "fe_text",
"title": "",
"template": "fe_explore_note",
"embedTemplate": "fe_embed",
"version": "1",
"isLong": true,
"isCurated": false
},
{
"request": {
"time": "2016-08-26 2017-08-26",
"resolution": "WEEK",
"locale": "en-US",
"comparisonItem": [
{
"geo": {},
"complexKeywordsRestriction": {
"keyword": [
{
"type": "ENTITY",
"value": "/m/0128447t"
}
]
}
},
As a result, the rest of the widgets get shifted by 1, so pos needs to be incremented as well.
A possible fix is to modify this try caluse of parseResults() to
var widgets = JSON.parse(e.slice(4)).widgets;
if (!widgets[0].request) { widgets.shift(); }
return widgets;
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Search Term (Amazon Fire TV) vs Topic ('/m/0h65vcz')
    api.interestOverTime({ keyword: ['/m/0h65vcz', 'Amazon Fire TV'] }

@meetamit I made a PR in #76 to correct this issue if you want to check it out

Thank you! I will check it out as soon as Google unblocks my IP address :( Yep, I managed to get myself blocked for making (not a crazy amount but still) too many concurrent queries to their API. I've since changed my code to throttle the requests, but the block still remains. In case you've been through this, I wonder whether you know how long the block takes to expire.


intergrate Coupons to admin app and allow for the app to

intergrate Coupons to admin app and allow for the app to
generate and manage them
https://github.com/freelancing-solutions/GCP-Based-Database-as-a-Service/blob/ac31d7222e383fbe8bfed71c77f093a8d1fbbb2f/data_service/api/coupons/routes.py#L32

        return coupons_view_instance.get_expired_coupons()
    else:
        pass

    # TODO intergrate Coupons to admin app and allow for the app to
    # generate and manage them

b53afb1e462e48c3867743f4953c46039c809e64

Closed in ba12a5850a168355f95be142798b20a7566e2b7b


Has Okta dependency

Bug Template
Title: ModuleNotFoundError: No module named 'okta.framework'
Description:

I've just installed Cartography and I'm running it for the first time, and it dies with an error about not finding the okta.framework module.

To Reproduce:

Install Neo4j (brew cask install neo4j)
Start Neo4j
Set up Python3 virtual environment (virtualenv -p $(which python3) python; source python/bin/activate; pip install --upgrade pip)
Install cartography (pip install cartography)
Run cartography:

cartography --neo4j-uri bolt://localhost:7687

Logs:

If applicable, copy and paste your console log with the failing stack trace.

$ cartography --neo4j-uri bolt://localhost:7687
Traceback (most recent call last):
  File "/Users/me/work/cartography/python/bin/cartography", line 5, in <module>
    from cartography.cli import main
  File "/Users/me/work/cartography/python/lib/python3.8/site-packages/cartography/cli.py", line 7, in <module>
    import cartography.sync
  File "/Users/me/work/cartography/python/lib/python3.8/site-packages/cartography/sync.py", line 16, in <module>
    import cartography.intel.okta
  File "/Users/me/work/cartography/python/lib/python3.8/site-packages/cartography/intel/okta/__init__.py", line 3, in <module>
    from okta.framework.OktaError import OktaError
ModuleNotFoundError: No module named 'okta.framework'

Screenshots:

If applicable, add screenshots to help explain your problem.

Please complete the following information::

Cartography release version or commit hash: cartography-0.28.0
Python version: 3.8.5
OS (feel free to omit this if you don't think it's relevant to your issue): OSX Catalina 10.15.7

Additional context:

Our company does use Okta, but not for AWS.

Hi, sorry for the trouble. See #424 and #425 for more details about this problem.
As a temporary work-around, you can try running pip install okta==0.0.4 in your environment and then trying to run cartography again.
We fixed this issue in our 0.29.0 release, but we've been having trouble getting that  pushed to pypi because of (yet another) dependency problem.
Sorry again for the trouble, there's lots of different moving pieces that we're fixing at the moment! 😅

Ok!  I apologize for opening an issue for this, I searched for 'okta' issues but didn't see any descriptions that matched in the open issues.

Next error:
$ cartography --neo4j-uri bolt://<IP_ADDRESS>:7687
Traceback (most recent call last):
  File "/Users/me/work/cartography/python/bin/cartography", line 8, in <module>
    sys.exit(main())
  File "/Users/me/work/cartography/python/lib/python3.8/site-packages/cartography/cli.py", line 348, in main
    return CLI(default_sync, prog='cartography').main(argv)
  File "/Users/me/work/cartography/python/lib/python3.8/site-packages/cartography/cli.py", line 328, in main
    return cartography.sync.run_with_config(self.sync, config)
  File "/Users/me/work/cartography/python/lib/python3.8/site-packages/cartography/sync.py", line 106, in run_with_config
    neo4j_driver = GraphDatabase.driver(
  File "/Users/me/work/cartography/python/lib/python3.8/site-packages/neo4j/__init__.py", line 120, in driver
    return Driver(uri, **config)
  File "/Users/me/work/cartography/python/lib/python3.8/site-packages/neo4j/__init__.py", line 161, in __new__
    return subclass(uri, **config)
  File "/Users/me/work/cartography/python/lib/python3.8/site-packages/neo4j/__init__.py", line 235, in __new__
    pool.release(pool.acquire())
  File "/Users/me/work/cartography/python/lib/python3.8/site-packages/neobolt/direct.py", line 715, in acquire
    return self.acquire_direct(self.address)
  File "/Users/me/work/cartography/python/lib/python3.8/site-packages/neobolt/direct.py", line 608, in acquire_direct
    connection = self.connector(address, error_handler=self.connection_error_handler)
  File "/Users/me/work/cartography/python/lib/python3.8/site-packages/neo4j/__init__.py", line 232, in connector
    return connect(address, **dict(config, **kwargs))
  File "/Users/me/work/cartography/python/lib/python3.8/site-packages/neobolt/direct.py", line 972, in connect
    raise last_error
  File "/Users/me/work/cartography/python/lib/python3.8/site-packages/neobolt/direct.py", line 963, in connect
    s, der_encoded_server_certificate = _secure(s, host, security_plan.ssl_context, **config)
  File "/Users/me/work/cartography/python/lib/python3.8/site-packages/neobolt/direct.py", line 854, in _secure
    s = ssl_context.wrap_socket(s, server_hostname=host if HAS_SNI and host else None)
  File<EMAIL_ADDRESS>line 500, in wrap_socket
    return self.sslsocket_class._create(
  File<EMAIL_ADDRESS>line 1040, in _create
    self.do_handshake()
  File<EMAIL_ADDRESS>line 1309, in do_handshake
    self._sslobj.do_handshake()
OSError: [Errno 0] Error

Isn't the zero error = "success"?

Apologies again; I should just get the 0.29.0 version from github.  Sorry!

Double check that you have the Neo4j 3.5.19 database (not 4.x!), and the neo4j 1.7.6 python driver.

It's Neo4j Desktop Version 1.3.10 (<IP_ADDRESS>), which is whatever version brew cask install neo4j decided to download.

Apparently there's a neo4j formula and a neo4j cask.  I'm downloading the formula now.  Sigh.

Aaaaand apparently Brew is downloading neo4j-community-4.1.3-unix.tar.gz :-(

Haha yeah, you'll want to follow these steps: https://github.com/lyft/cartography/blob/master/docs/setup/install.md#cartography-installation
In case you get stuck on that, I put together a setup video for devs of Cartography, but a lot of the steps probably apply to users too: https://www.youtube.com/watch?v=4-Ps7c_BmjE

From watching the video, TIL pip install -e .  😄
Also, Neo4j.com has redesigned their website, making it harder to find Neo4j 3.5.x.  👎🏻
Ok, it seems to be running now! ... and running... and running... Hmm, needs a progress bar.  :-)
Thanks!

Alright, I released 0.29.1 and that should be in Pypi now which should not be affected by the okta.framework issue so you don't need to run an editable install anymore unless you want to be running tests and deving cartography :-)

Thanks!
On November 13, 2020 11:44:43 a.m. PST, Alex Chantavy<EMAIL_ADDRESS>wrote:

Alright, I released 0.29.1 and that should be in Pypi now which should not be affected by the okta.framework issue so you don't need to run an editable install anymore unless you want to be running tests and deving cartography :-)
--
You are receiving this because you modified the open/close state.
Reply to this email directly or view it on GitHub:
https://github.com/lyft/cartography/issues/459#issuecomment-726996271
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Feature 1

I work on my first project 'Hello Microverse' by testing all the tools needed in the program.

Add index.html
Add styles.css for styling
Add linters also to avoid errors and to make my code dry.
And all other tools also to make my code look professional.

Thanks so much for the review it is perfectly nice, just have to amend a slice mistake of not adding my styles.css link into my html file but have done it now immediately i realize it.
Thank so much for the review @frank1738


Update stubbing to support any objects, not only classes

Now we can stub class/module methods only via fixture syntax:
---
- class: Foo
  chain:
    - bar
    - baz
    - qux
  # ...

It would be nice to support the following syntax as well:
---
- object: Foo.bar.baz
  chain:
    - qux
  # ...

The feature was implemented in the v0.0.7


New-PBIGroupUser error when using -groupUserAccessRight Member

Command:
New-PBIGroupUser -authToken $token -groupId $groupId -emailAddress $email -groupUserAccessRight Member
Error:
Invoke-PBIRequest : The remote server returned an error: (400) Bad Request. -
'{"error":{"code":"UnsupportedAccessRightError","pbi.error":{"code":"UnsupportedAccessRightError","parameters":{},"details":[]}}}'
At C:\Program Files\WindowsPowerShell\Modules\PowerBIPS\<IP_ADDRESS>\PowerBIPS.psm1:501 char:12

...   $result = Invoke-PBIRequest -authToken $authToken -method Post -res ...

            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : NotSpecified: (:) [Write-Error], WebException
FullyQualifiedErrorId : System.Net.WebException,Invoke-PBIRequest

Mind the "Note:" on https://docs.microsoft.com/en-us/rest/api/power-bi/groups/addgroupuser
API Version: v1.0
Grants the specified user permissions to the specified workspace.
Note: Only Admin access right is supported.

Why does this restriction exist? Is there any work around?

Not tested yet, but they told me that using workspace V2 should work:
https://docs.microsoft.com/en-us/rest/api/power-bi/groups/creategroup
POST https://api.powerbi.com/v1.0/myorg/groups?workspaceV2={workspaceV2}


How to get a release?

How can we get/generate a release that contains the following commit?
https://github.com/cjohansen/Sinon.JS/commit/79d3ee14aeb1fe07155c58f3469b31eff3950d23
couldn't find instructions and seems like the sinon-web repo is not what its beign used?

@Asgaroth I've poked @cjohansen a couple of days ago about a new release, and about releasing more often. It shouldn't be long now :)

Awesome :smiley:

Sorry, I've been traveling since the ping. I'll get something together shortly.

I believe this is done, can the issue be closed?

Yes thanks!