Feature: Read checkboxes in Word forms

I am trying to programmatically extract form data from a large number of Word documents. I am able to extract the text data, but not the checkboxes. Is it possible to determine if a checkbox has been checked using python-docx? If so, could someone show me some sample code? Otherwise, what tool would you recommend? Thanks!

I solved the problem by dropping python-docx and using lxml and zipfile directly. I hadn't realized that a Microsoft Word document is just a zipped archive of XML files! But it would be nice if python-docx had support for reading form inputs.

@Radcliffe for this sort of thing most folks use python-docx to do the heavy lifting of getting you close in the XML hierarchy, then going to lxml calls for any unimplemented bits. How and where do checkboxes show up in the XML? It could be handy for other folks to know who come across this requirement :)

Having the same issue here... My checkboxes are in a table cell. I have the cell object now. Can you get the raw data from the cell object?

I have the following code using the lxml lib:
import zipfile
from lxml import etree

def get_word_xml(docx_filename):
    with open(docx_filename) as f:
        zip = zipfile.ZipFile(f)
        xml_content = zip.read('word/document.xml')
    return xml_content

# def _check_element_is(self, element, type_char):
def _check_element_is(element, type_char):
    word_schema = 'http://schemas.openxmlformats.org/wordprocessingml/2006/main'
    return element.tag == '{%s}%s' % (word_schema, type_char)

xml_content = get_word_xml('./example.docx')
xml_tree = etree.fromstring(xml_content)

def isChecked(checkbox):
    val = False
    for child in checkbox:
        if _check_element_is(child, 'checked'):
            val = True
            return val
    return val

def checkboxValuesInElement(el):
    retVal = {}
    i = 0
    for child in el:
        if _check_element_is(child, 'checkBox'):
            retVal[i] = isChecked(child)
            i += 1
    return retVal

I tried to fork this project and make a pull request, but I couldn't figure out how to contribute. Is there docs on that?
Otherwise, a checkbox generally looks like this:
<w:checkBox>
   <w:sizeAuto />
   <w:default w:val="0">
   <w:checked/>
</w:checkBox>

and it only has the <w:checked/> child if it is checked. Otherwise it is unchecked. Mine happen to be in a table, but it could be in a body as well.
Other possible enhancements I thought might be nice would be to get the path to an element to pass that into lxml if that is more suited for it. It would be nice to be able to start with these objects though, like, Table.raw would give the xml for the Table and all it's child elements.

Ok cool. So the only issue then would be marrying the checkbox to a string value somewhere. Do these  nodes have a .children .parent or .siblings property?

Each element is a subclass of lxml.etree._Element, so all the members on that class are available to you: http://lxml.de/3.7/api/index.html
I might start by understanding the ancestors a bit better, like maybe printing out the XML for a paragraph that contains one:
document = Document('example.docx')
for paragraph in document.paragraphs:
    p = paragraph._element
    checkBoxes = p.xpath('.//w:checkBox')
    if checkBoxes:
        print(p.xml)
        break

Hi, Scanny,
Thanks for your great work. I try the checkbox code.  But I notice that my document.xml does not use w:checkBox. Instead, it uses w14:checkbox. I try p.xpath('.//w14:checkbox') but it complains about
File "/home/wailoktam/.local/lib/python3.5/site-packages/docx/oxml/xmlchemy.py", line 751, in xpath
xpath_str, namespaces=nsmap
File "src/lxml/etree.pyx", line 1577, in lxml.etree._Element.xpath
File "src/lxml/xpath.pxi", line 307, in lxml.etree.XPathElementEvaluator.call
File "src/lxml/xpath.pxi", line 227, in lxml.etree._XPathEvaluatorBase._handle_result
lxml.etree.XPathEvalError: Undefined namespace prefix
I try
checkBoxes = p.xpath('.//w14:checkbox', namespaces={'w14':'http://schemas.microsoft.com/office/word/2010/wordml'})
But then it complains about having the extra namespaces keyword argument.
What have I done wrong?
Many thanks in advance.

Just wanted to leave some comments after I got my program working for anyone else who ends up here.

For me lxml was the best module to make this work
To interact with the "w14" tags, i needed to modify the xml file prior to creating the xml tree using lxml.  I used the following commands to read the file, modify the xml, and create the XML tree.  The key line of code that allows lxml to interact with the w14 tag is where I removed the attribute "mc:Ignorable="w14 w15 wp14" "
`
from zipfile import ZipFile
from lxml import etree
import tempfile

tmp_dir = tempfile.mkdtemp()
with ZipFile(self.template_file_name) as myzip:
self.filenames = myzip.namelist()
myzip.extractall(self.tmp_dir)
file_name = tmp_dir+'\word\document.xml'
xml_content = open(file_name,'r', encoding="utf8").read()
xml_content = xml_content.replace('mc:Ignorable="w14 w15 wp14"', '').encode('utf-8')
parser = etree.XMLParser(recover=True, encoding='utf-8')
self.tree = etree.parse(io.BytesIO(xml_content), parser)
self.root = self.tree.getroot()
`
3.  The tags identified with the w14 prefix is only part of the xml tags associated with these controls.  The check boxes start with <w:sdt> which is a standard tag for a custom function in the OOXML tag reference.  It stands for "structured document tags".  You can find out more about sdt tags here:

4.  There are 2 methods that can be used to read the check boxes as their value is stored in 2 locations. For one of them you don't need to worry about reading tags prefixed with "w14".  The value is stored in the following 2 locations (using XPATH notation, you'll likely want to learn how XPATH works).
a) \w:sdt\w:sdtPr\w14:checkbox\w14:checked[@w14:val="1"]
b) \w:sdt\w:sdtContent\w:r\w:t
For further clairty I'll past an example tag below and bold the key tags where the value can be extracted:
<w:sdt>
<w:sdtPr>
<w:rPr>
<w:rFonts w:ascii="Arial" w:hAnsi="Arial" w:cs="Arial"/>
<w:sz w:val="22"/>
<w:szCs w:val="22"/>
</w:rPr>
<w:alias w:val="user defined checkbox name"/>
<w:tag w:val="user_defined_tag_name"/>
<w:id w:val="163048189"/>
<w:lock w:val="sdtLocked"/>
w14:checkbox
<w14:checked w14:val="1"/>
<w14:checkedState w14:val="2612" w14:font="MS Gothic"/>
<w14:uncheckedState w14:val="2610" w14:font="MS Gothic"/>
</w14:checkbox>
</w:sdtPr>
<w:sdtEndPr/>
<w:sdtContent>
<w:r>
<w:rPr>
<w:rFonts w:ascii="MS Gothic" w:eastAsia="MS Gothic" w:hAnsi="MS Gothic" w:cs="Arial" w:hint="eastAsia"/>
<w:sz w:val="22"/>
<w:szCs w:val="22"/>
</w:rPr>
<w:t>☒</w:t>
</w:r>
</w:sdtContent>
</w:sdt>
5.  This is likely obvious but to those wanting to control the checkbox you need to to modify both tags highlighted above.  As one would expect, 1 equals checked ('☒'), 0 equals unchecked ('☐').
I hope the above is helpful to someone.

I read a simple docx, where I put 1 checkbox in the text, 2 checkboxes in the table's cells. however I found the xml file does not match what have been discussed above. So there is no universal solution for different version DOCX, isn't it?
<w:control r:id="rId6" w:name="CheckBox2" w:shapeid="_x0000_i1026"/>
</w:object></w:r></w:p></w:tc><w:tc><w:tcPr><w:tcW w:w="1380" w:type="dxa"/></w:tcPr><w:p><w:r><w:object><v:shape id="_x0000_i1027" o:spt="201" alt="" type="#_x0000_t201" style="height:35.4pt;width:58.2pt;" o:ole="t" filled="f" o:preferrelative="t" stroked="f" coordsize="21600,21600"><v:path/><v:fill on="f" focussize="0,0"/><v:stroke on="f"/><v:imagedata r:id="rId9" o:title=""/><o:lock v:ext="edit" aspectratio="f"/><w10:wrap type="none"/><w10:anchorlock/></v:shape>

<w:control r:id="rId8" w:name="CheckBox3" w:shapeid="_x0000_i1027"/>
</w:object></w:r></w:p></w:tc></w:tr></w:tbl><w:p><w:bookmarkStart w:id="0" w:name="_GoBack"/><w:bookmarkEnd w:id="0"/></w:p><w:sectPr><w:pgSz w:w="11906" w:h="16838"/><w:pgMar w:top="1440" w:right="1800" w:bottom="1440" w:left="1800" w:header="851" w:footer="992" w:gutter="0"/><w:cols w:space="425" w:num=[checkbox.docx](https://github.com/python-openxml/python-docx/files/7776019/checkbox.docx)

+1

There are 2 methods that can be used to read the check boxes as their value is stored in 2 locations. For one of them you don't need to worry about reading tags prefixed with "w14".  The value is stored in the following 2 locations (using XPATH notation, you'll likely want to learn how XPATH works).
a) \w:sdt\w:sdtPr\w14:checkbox\w14:checked[@w14:val="1"]
b) \w:sdt\w:sdtContent\w:r\w:t
For further clairty I'll paste example xml code below of the sdt element:

<w:sdt>
    <w:sdtPr>
        <w:rPr>
            <w:rFonts w:ascii="Arial" w:hAnsi="Arial" w:cs="Arial"/>
            <w:sz w:val="22"/>
            <w:szCs w:val="22"/>
        </w:rPr>
        <w:alias w:val="user defined checkbox name"/>
        <w:tag w:val="user_defined_tag_name"/>
        <w:id w:val="163048189"/>
        <w:lock w:val="sdtLocked"/>
        <w14:checkbox>
            <w14:checked w14:val="1"/>
            <w14:checkedState w14:val="2612" w14:font="MS Gothic"/>
            <w14:uncheckedState w14:val="2610" w14:font="MS Gothic"/>
        </w14:checkbox>
    </w:sdtPr>
    <w:sdtEndPr/>
    <w:sdtContent>
        <w:r>
            <w:rPr>
                <w:rFonts w:ascii="MS Gothic" w:eastAsia="MS Gothic" w:hAnsi="MS Gothic" w:cs="Arial" w:hint="eastAsia"/>
                <w:sz w:val="22"/>
                <w:szCs w:val="22"/>
            </w:rPr>
            <w:t>☒</w:t>
        </w:r>
    </w:sdtContent>
</w:sdt>

This is likely obvious but to those wanting to control the checkbox you need to to modify both tags highlighted above.  As one would expect, 1 equals checked ('☒'), 0 equals unchecked ('☐').
If you are modifying documents, I found the simplest method was to unzip everything to a temporary folder, modify the document.xml file, and then rezip everything together.  The code above takes care of the uzipping into a temporary file.  If you're just reading the word document, the code above by @Radcliffe  to access the document.xml file works great (thanks jdell64, that's what I started with).  The code associated with rezipping and then cleaning up the temporary files is as follows:

        import os
        import shutil

        with open(os.path.join(self.tmp_dir, 'word/document.xml'), 'w') as f:
            xmlstr = etree.tostring(self.tree, pretty_print=True)
            f.write(xmlstr.decode('utf-8'))

        # Create the new zip file and add all the filex into the archive
        zip_copy_filename = self.output_file_name
        with ZipFile(zip_copy_filename, "w") as docx_file:
            for filename in self.filenames:
                docx_file.write(os.path.join(self.tmp_dir, filename), filename)

        # Clean up the temp dir
        shutil.rmtree(self.tmp_dir)

I hope the above is helpful to someone. I apologize if the code doesn't copy and paste and simply work. I inferred the necessary code form my own trying to eliminate unnecessary detail. Let me know if it doesn't and I'll see if I can figure out where any error is.

Each element is a subclass of lxml.etree._Element, so all the members on that class are available to you: http://lxml.de/3.7/api/index.html
I might start by understanding the ancestors a bit better, like maybe printing out the XML for a paragraph that contains one:
document = Document('example.docx')
for paragraph in document.paragraphs:
    p = paragraph._element
    checkBoxes = p.xpath('.//w:checkBox')
    if checkBoxes:
        print(p.xml)
        break

Hi Scanny,
Could you teach me how to check the checkbox?
I set w14:val to 1, but the checkbox is still unchecked.
Anything else I need to do?
Thanks


[in_app_purchase] How to check for refunds on android ?

Hello folks,
I am currently writing an app with flutter that will have in app products to buy. Everything is ready and I tested the whole purchase process. But there is still one problem I can't find a solution for :
Scenario :
Someone downloads the app and tries out the feature. Now he upgrades to the PRO version by bying the corresponding in app product. A few days later, the customer is having some problems with the app and want to have the money back. I create a refund in the play console.
Now the problem is, that the queryPastPurchases still fetches the purchase. I waited a few days ( around 7 days ) and no changes. Also the PurchaseStatus does not seem to have the status refunded.
How would I check if a purchase was refunded ?
I use the following version of the in_app_purchase plugin  :
in_app_purchase: ^0.3.0+3

Duplicate of #51296

Please reopen, due to the linked issue is the same as this one ...

@Tomucha
How would I revoke a purchase ? There is only a "refund" button in the Play Console interface. Also the documentation at https://support.google.com/googleplay/android-developer/answer/2741495?hl=en does not list a "revoked" status.
The documentation only mentions "revoke" for subscriptions. But I only offer a one time purchase to upgrade to the "PRO" version of the app.

"Revoke" checkbox is part of "refund" dialog.

@Tomucha
Thanks for the good information. Now I am trying to revoke a purchase after I refunded it via the UI. There is no way to revoke it via the UI after I refunded it without checking the checkbox. Currently I am trying to revoke it via API but I can't get the OAUTH thing working.
{
    "error": {
        "errors": [
            {
                "domain": "androidpublisher",
                "reason": "projectNotLinked",
                "message": "The project id used to call the Google Play Developer API has not been linked in the Google Play Developer Console."
            }
        ],
        "code": 403,
        "message": "The project id used to call the Google Play Developer API has not been linked in the Google Play Developer Console."
    }
}

Is the API accessible via API Key or something like that ?

Sorry, I don't have any experience with that. Even the "revoke" suggestion was just a lucky guess :-)

I have encountered the same issue, nor refund or (pending->denied) features work.

@Tomucha
Still the same problem. I created a new app and added a new in app product. I ordered the product and after that, closed the app and refunded the purchase. I also clicked the checkbox for "revoke". Afer the refund is done, I started the app again and I am still receiving the pruchase with status purchased.
Will now wait a few hours / days to check again. Maybe there is some caching.

@Tomucha after around 24 hours the queryPastPurchases did not return any purchases any more after refunding and revoking. So this seems to work now! Thanks for the advice.
@TheWCKD i am not sure if this information helps you to fix your problem. I will close this issue.

As @Tomucha had the right hint. This issue can now be closed.


Check if excel file has any macro

Hi,
This is not an issue, I just want to know is there a way to find out if the excel file has any macro using this library.

xlsm files are the only ones allowed to have macros, maybe check the file extension?

@wildeyes xls and xlsb are also permitted to have records.
For VBA macros, passing the option bookVBA:true to the read functions will set the vbaraw property of the workbook to the VBA storage if it is available.
Pre-95 Macrosheets are currently not persisted.

Reading the file with the option bookVBA:true will create a key vbaraw if VBA is in the workbook.  To find macrosheets, check the worksheet !type key.  The full test is:
const wb_has_macro = (wb) => !!wb.vbaraw || wb.SheetNames.map(n => wb.Sheets[n]).some(ws => ws && ws['!type'] == 'macro');


Nutch-2676

This is a patch containing some updates to the selenium plugin:

headless mode for Chrome/Firefox
code for Selenium HUB
customers rotation to WebDriver in remote mode
examples of use with Docker (specifically with Docker Compose) as well with Docker Selenium nodes, Docker Selenium Hub, Docker Apache Solr
deprecated code removed

@sebastian-nagel Ready to be merged:

duplicated readme removed
eclipse formatting applied.

For the squashing commits, it is possible to do it right from the GitHub
https://help.github.com/articles/about-pull-request-merges/#squash-and-merge-your-pull-request-commits

Yes, of course, I can let git/github do the squash. But writing a comprehensive commit message is still manual work. :)   I'll need 1-2 days to test the code, then I'll merge it. Thanks a lot, @sbatururimi!

My proposition:
- upgrade of Selenium plugin related packages
- added the use of headless mode when using Selenium nodes(chrome & firefox)
- obsolete code for Selenium plugin removed
- fix of a bug occurring during the build of  the Nutch docker container
- added possibility to use a Selenium Hub orchestrator in multi-containers docker mode
- added several examples of using Nutch+Solr+Selenium Hub+Selenium Nodes in a network of Docker containers

Rebased, squashed and merged PR. Thanks, @sbatururimi!


Update for v3.4.6, update theme and fonts

Updated docs according to v3.4.6.
Added new fonts and theme to docz.
Moved material-ui tips to a new menu section: 'Guides & tips'.

Preview can be seen here: https://pr-4.d284iy6qg8hpbt.amplifyapp.com


HDDS-4913. Refine the native authorizer parent context right check.

What changes were proposed in this pull request?
Change the parent context right check based on READ/WRITE.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-4913
How was this patch tested?
Draft patch based on current UT and acceptance test. Will add more test with followup changes.

@xiaoyuyao ,  currently parent context check of a key applies to both bucket and volume.  While the volume is the parent of bucket, not the parent of key,  I think we need to have seperate parent context for bucket and volume.  Say, If I want to create key1 under volume1/bucket1,  write permission on bucket1 is required, but write permission on volume1 is not necessary.

@xiaoyuyao , currently parent context check of a key applies to both bucket and volume. While the volume is the parent of bucket, not the parent of key, so shall we consider to have seperate parent context for bucket and volume? Say, If I want to create key1 under volume1/bucket1, write permission on bucket1 is required, but write permission on volume1 seems not necessary.

@xiaoyuyao , currently parent context check of a key applies to both bucket and volume. While the volume is the parent of bucket, not the parent of key, so shall we consider to have seperate parent context for bucket and volume? Say, If I want to create key1 under volume1/bucket1, write permission on bucket1 is required, but write permission on volume1 seems not necessary.

Based on offline discussion with @arp7, he raised a good point that the requirement of write permission on volume gives admin a quick way to prevent user access to all the resources (buckets/keys) in one change at volume level instead of individual changes at buckets/keys level.

For @arp7 's requirement, we can have a tool or CLI command to achieve this.  If we think of volume/bucket/key as a FileSystem tree, it's not very make sense that write a grand-child node need the write permission of it's grand-parent.

Thanks @ChenSammi  for the review. This JIRA fix the issue with the parent context of READ_ACL/WRITE_ACL/CREATE right. We can address that in followup discussions. Can you take another look? Thanks!

@rakeshadr, can you help reviewing? Thanks!

@rakeshadr, thanks for the review. Please find the UT added in this commit a3527b9.  The related tasks are captured under umbrella JIRA HDDS-4655.

Thanks @rakeshadr for the review. PR has been merged.


docker-resin-supervisor-disk: add armv7va support

When switching from jethro to morty, the architecture for the rpi3
changes from "armv7a" to "armv7va", which is armv7a + virtualization.

connect https://github.com/resin-os/resin-raspberrypi/pull/76

Changelog please.

Needs rebase.


DRILL-3749: Upgrade Hadoop to 2.7.1

Upgrades Hadoop dependency from 2.4.1 to 2.7.1. This allows us to take
advantage of the latest S3 filesystem support.

Can you check if we need to update (override new methods) the DrillFileSystem and/or DrillFsDataInputStream which are decorators around hadoop FileSystem and FsDataInputStream?

LGTM +1.
Let's get this merged.


Random failure when starting swarm on linux

There's a random failure when starting amp on linux:
 bq@ubuntu  ~/go/src/github.com/appcelerator/amp   accounts2  amp start
copying source files from /usr/local/bootstrap to /root/.config/infrakit/infrakit from inside a container
preparing a Docker image with the configuration data
creating Docker volume amp-bootstrap with the configuration data
the 5 nodes will be distributed as 3 managers and 2 workers
cluster size added to /infrakit/env.ikt
c66e17d40b28694d3579c43b76f6bb7fa167c6c884377b5566c2cdc1199b8b71
0.4.1: Pulling from infrakit/devbundle
0a8490d0dfd3: Already exists
6848525df145: Already exists
b9654bb99a97: Already exists
c9b4e777ebbe: Already exists
Digest: sha256:f0903639c11ba5449b061d321197fc67cde9711c1a410c6c9909122bcc9fe5ce
Status: Image is up to date for infrakit/devbundle:0.4.1
1.0.0: Pulling from appcelerator/infrakit-docker
Digest: sha256:af057538ad2c32b3d6dcf3abaad6d153010bb8a12032dd945fae873af16a5fc5
Status: Image is up to date for appcelerator/infrakit-docker:1.0.0
making sure the network hostnet is created
starting up InfraKit
9202c7a007fbd10f00397f398b9a03577d9c15de64b2285312a532be2d79596f
starting up InfraKit docker plugin (image appcelerator/infrakit-docker:1.0.0)...
7520314253aa734aed36e0912a67bd140ad4b2fe1fcdab26e432ef4961ed1734
waiting for plugins to be available....
prepare the InfraKit configuration file...
time="2017-03-23T22:28:17Z" level=info msg="Using file:///infrakit/config.docker.tpl for reading template\n"
deploy the configuration...
time="2017-03-23T22:28:17Z" level=info msg="Found manager group is leader =  false"
time="2017-03-23T22:28:17Z" level=error msg="no plugin"
done

the issue (leader = false) is known and a fix is in progress on branch ikt-terraform-aws


Check assets folder on start

I think it will be nice to check assets or just assets folder on start to exit gracefully with hint run make assets before cargo run instead of:
thread '<main>' panicked at 'Can`t open file 'assets/DroidSerif-Regular.ttf' (No such file or directory (os error 2))', src/visualizer/src/fs.rs:14
note: Run with `RUST_BACKTRACE=1` for a backtrace.
error: Process didn't exit successfully: `target/debug/zoc` (exit code: 101)

@Vinatorul Is it ok now? :)

Great, thanks!


Use of #define LWIP_SINGLE_NETIF 1 by default on Pico W

For minimalism reasons it might be useful to enable by default the LWIP_SINGLE_NETIF in lwipopts.h.
At the moment the SDK does not compile with that option.

I can't see any issues in the SDK. It seems to be a problem in pico-examples and the cyw43-driver.
https://github.com/raspberrypi/pico-examples/issues/514
https://github.com/georgerobotics/cyw43-driver/issues/124


move folders to a new folder?

is it possible yet to move a folder to another folder, or is the same as #10?

same as #10, it's not possible yet.
will be added soon.

This has been implemented in the latest release v0.9.0: https://github.com/sakethpathike/Linkora/releases/tag/release-v0.9.0


Update excel.py

Fix https://github.com/pydata/pandas/issues/9930
read_excel with sheetname=None or sheetname=[0,1,”Sheet5”] returns an OrderedDict instead of a dict

This still needs tests :). Take a look at the excel tests and update some or all of them to require sheets to appear in the original order.

How can I run only this test ?
(because I don't want to run all tests using tests.sh)
I tried
$ python pandas/io/tests/test_excel.py
cannot import name hashtable
Traceback (most recent call last):
  File "pandas/io/tests/test_excel.py", line 3, in <module>
    from pandas.compat import u, range, map, openpyxl_compat
  File "/Users/scls/github/scls19fr/pandas/pandas/__init__.py", line 7, in <module>
    from . import hashtable, tslib, lib
ImportError: cannot import name hashtable

I also tried
bash test_fast.sh
E
======================================================================
ERROR: Failure: ImportError (cannot import name hashtable)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "//anaconda/lib/python2.7/site-packages/nose/loader.py", line 414, in loadTestsFromName
    addr.filename, addr.module)
  File "//anaconda/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "//anaconda/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/Users/scls/github/scls19fr/pandas/pandas/__init__.py", line 7, in <module>
    from . import hashtable, tslib, lib
ImportError: cannot import name hashtable

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)

@scls19fr Take a look at these sections of the contributing guidelines (might be worth reading other sections, too!):
http://pandas-docs.github.io/pandas-docs-travis/contributing.html#making-changes
http://pandas-docs.github.io/pandas-docs-travis/contributing.html#running-the-test-suite

I think this test could be good
def test_reading_all_sheets(self):
    # Test reading all sheetnames by setting sheetname to None,
    # Ensure a dict is returned.
    # See PR #9450
    # Ensure sheet order is preserved.
    # See issue #9930

    _skip_if_no_xlrd()

    dfs = read_excel(self.multisheet,sheetname=None)
    expected_keys = ['Alpha','Beta','Charlie']
    tm.assert_contains_all(expected_keys,dfs.keys())
    for i, sheetname in enumerate(dfs):
        expected_sheetname = expected_keys[i]
        tm.assert_equal(sheetname, expected_sheetname)

But I won't send a new PR until I can run test on my side.
Any idea ?

I also wonder what should happen when a list is given as sheetname parameter. Should list order be preserved or should excel sheetname order be preserved ?

I would preserve the excel sheetname order. The user already supplied a list of desired sheets, so they can access them in that order if desired. But if we reorder them to the supplied order, there is no way for them to access them in the original order.
This rule is also simpler to explain.

Preserving order defined by user will be (in my mind) both much easier to explain and much easier to implement. I don't see how we can preserve excel sheetname order easily.
see ExcelFile._parse_excel
    if isinstance(sheetname, list):
        sheets = sheetname
        ret_dict = True
    elif sheetname is None:
        sheets = self.sheet_names
        ret_dict = True
    else:
        sheets = [sheetname]

....

for asheetname in sheets:
    ...

@scls19fr you don't need to open a new pull requests each time you update. just push to the same one.

Sorry I still need to learn Git and GitHub

To preserve original sheet order, you could use something like this:
    if isinstance(sheetname, list):
        requested_sheets = set(sheetname)
        if not requested_sheets <= set(self.sheet_names):
            raise ValueError('...')
        sheets = [s for s in self.sheet_names if s in requested_sheets]
        ret_dict = True

@scls19fr pls add some tests.

I'm sorry but I'm quite busy these days

closing if/when updated, pls reopen


Can I set rasa-webchat live chat with the website host username as sender_id?

Hi, that's maybe a change request/info request:
I want to integrate a live chat into an existing website that users can navigate with or without logging-in.
Optimally I want to manage two different scenarios:

user could chat with a RASA backend assistant through the live chat, even if not (yet) logged for general info, etc.

But the basic requirement is that user must log in the website with his credentials (username/password) and afterward he can chat with the RASA assistant backend as a logged-in user with a single user_id (say the username == his email). In that way the assistant can profile/personalize answers.

Reading the RASA connectors guide: https://rasa.com/docs/rasa/user-guide/connectors/your-own-website/ I understand that I can configure RASA to accept socketio channel and a good candidate for the websocket live chat integrations seems the project:
https://github.com/botfront/rasa-webchat
My question is about how to possibly simply configure the sender_id equal to the username with rasa-webchat.
The integration seems seamless in case of an "impersonal" chat , and as I well understood, in this case the sender_id is set with a random value. That's ok for the case (1)
But how can I manage the case (2)? Reading https://github.com/botfront/rasa-webchat#rasa, seems to me that te solution is to build a custom connector to manage customData={{sender_id: 'aUserName'}}. Is it so?
I wish that when user is logged in, his username is used as sender_id could be used utomatically in the WebChat.default.initjavascript request.
Any idea?
Thanksgiorgio

A solution is to use the initPayload of the webchat and to parse an entity to it.
initPayload: "/get_started{"username"= "foo", "password"="bar"}
Then you can affect slots "username" and "password" in rasa and use them.

thanks @n2718281, I was referring to sender_id, but my final goal is to identify logged user, so your proposal makes sense!

Hi, I think @n2718281 is the best idea here. We don't have a feature to add your own id in the webchat yet, it sounds like a good idea though. I'll discuss it with the team.

Ok, so you either can pass the sender_id to rasa via the custom data, but as you stated in your initial messages, this will necessitate you writing your own rasa channel.
Or you can do as suggested and the data to the initial payload.
We're not planning to add a sender_id props as it will provide no additional feature compared to just adding it in the custom_data.
Thank you


roles/nv_gpu: allow installing an older version

This PR allows installing an older version of the GPU Operator from OperatorHub, but requested a specific ClusterServiceVersion and marking the as Manual.
The version is currently hardcoded:
gpu_operator_operatorhub_version: "1.4.0"

I'll modify it to be passed as a parameter to toolbox/gpu-operator/deploy_from_operatorhub.sh.
The deployment of 1.4.0 passed successfully the tests from toolbox/gpu-operator/run_ci_checks.sh on my local cluster;
I'll see how we integrate that into the CI, certainly at first as a on-demand test.

/test gpu-operator-e2e

/test gpu-operator-e2e

/opt/ci-artifacts/src/playbooks/roles/nv_gpu/tasks/install_nv.yml:38
Wednesday 03 March 2021  11:15:02 +0000 (0:00:00.759)       0:00:04.188 *******
TASK: nv_gpu : Find the GPU Operator OperatorHub InstallPlan
TASK: nv_gpu : Find the GPU Operator OperatorHub InstallPlan
changed: True

<command> oc get installplan -oname -loperators.coreos.com/gpu-operator-certified.openshift-operators

==> FAILED attempt #1/10

==> FAILED attempt #2/10

==> FAILED attempt #3/10

==> FAILED attempt #4/10

==> FAILED attempt #5/10

==> FAILED attempt #6/10

==> FAILED attempt #7/10

==> FAILED attempt #8/10

==> FAILED attempt #9/10

==> FAILED attempt #10/10

TASK: nv_gpu : Find the GPU Operator OperatorHub InstallPlan
changed: True

<command> oc get installplan -oname -loperators.coreos.com/gpu-operator-certified.openshift-operators

I don't understand why this is failing, it worked as expected in my cluster :(
I'll try to manually investigate
/test gpu-operator-e2e

the issue was caused by the namespace being missing while listing the InstallPlans ...
/test gpu-operator-e2e

@kpouget: The following test failed, say /retest to rerun all failed tests:
Test name 	Commit 	Details 	Rerun command
ci/prow/gpu-operator-e2e 	5311ab1 link /test gpu-operator-e2e
Full PR test history. Your PR dashboard.

failed because of an infrastructure crash, retrying
/test gpu-operator-e2e

deployment was failing when no version was specified (=deploy the latest version available)
/test gpu-operator-e2e

/test gpu-operator-e2e

I don't know what's going on here, I can't get the old versions to deploy anymore, the CVS never gets automatically created like 2 days ago ...

/test gpu-operator-e2e

/ok-to-test

/ok-to-test

/ok-to-test

/test yamllint

/ok-to-test
/test gpu-operator-e2e

/hold

/test gpu-operator-e2e

/test gpu-operator-e2e
/ok-to-test

/hold cancel

/hold

/unhold (I had to remove the branch protection)

/hold cancel
(I had to remove the branch protection)


Allow a CA to be provided to bootkube render

Bare metal examples could be simpler if a pre-existing CA cert could be provided to bootkube render so that only manifests and other credentials are generated.
Currently, I generate the CA with bootkube, paste it into different machine metadata declarations, then provision nodes to be bootkube-ready. It might also be valuable to pass a desired token to bootkube, depending on how long we're going to be using tokens.

This is needed for the automated installer.


Migrate models error

migrate models path throws  error TypeError: strapi.elastic.migrateModel.migrateModels is not a function
it should be strapi.elastic.migrateModels instead at strapi-plugin-elasticsearch/controllers/elasticsearch.js:12:33

thanks for your report


typedoc getting a random README.md file as entry point document?

Search terms
path entrypoint entrydocument entrypointstrategy  readme.md
Question
I've been wrapping my head around this for two days, and i don't really get what is happening here.
Whenever typedoc generates a new document, it creates a README.md file (name can be changed with the entryDocument option). But even though i specified a specific entryPoints, wheter it's a file or a folder, typedoc grabs a random README.md file that is positioned way upper in the folder structure and away from any specified entrypoint.
this is the folder structure:
[...][...][...]
├
├── 09-Data-Structures-Operators
│   ├── Coding challenge 1
│   │   ├── build
│   │   │   └── js
│   │   │       └── main.js
│   │   ├── docs
│   │   │   ├── assets
│   │   │   │   ├── highlight.css
│   │   │   │   ├── main.js
│   │   │   │   ├── search.js
│   │   │   │   └── style.css
│   │   │   ├── index.html
│   │   │   ├── modules.html
│   │   │   └── variables
│   │   │       └── a.html
│   │   ├── src
│   │   │   └── main.ts
│   │   └── tsconfig.json
[...][...][...]
└── README.md
This is my tsconfig.json
{
{
  "compilerOptions": {
"target": "es2016"
"module": "commonjs"
"rootDir": "./src"
"outDir": "./build/js"
"noEmitOnError": true
"esModuleInterop": true
"forceConsistentCasingInFileNames": true
 "strict": true
"skipLibCheck": true
},

"typedocOptions": {
    "entryPoints": ["./src/main.ts"],
    "entryPointStrategy": "resolve",
    "name": "Coding challenge 1.ts"
  },

  "include": ["./src"]
}

The entryDocument file (README.md) i get is that README.md in the very root folder, outside of my current workspace and distant from my entryPoints, i don't really know what may be causing this behaviour.
Another weird thing is that all the content in main.ts actually get rendered into modules, it's just the entry document that's showing this weird behaviour.
Things i tried:

changing the entryDocument name to something like: "entryDocument": "Conding challenge 1.md"
different entry points strategies: "entryPointStrategy": "resolve", "entryPointStrategy": "expand"
changing the entryPoints to containing directory or to the file:     "entryPoints": ["./src/main.ts"],  "entryPoints": ["./src"]

This sounds like a question for the markdown plugin repo -- typedoc doesn't have an entryDocument option, and I have no idea how it works.

This sounds like a question for the markdown plugin repo -- typedoc doesn't have an entryDocument option, and I have no idea how it works.

Even without the entryDocument option and without the typedoc markdown plugin active, the documentation gets rendered in HTML with exactly the same issue.

Ah, that is working as intended. README discovery starts at the base directory of your entry points, and traverses up the tree looking for a readme. This is so that common project structures with everything in a src or src/lib folder (like typedoc) can still have their readme discovered. If you don't want typedoc to include a readme, then --readme none is available.


InvalidResourceType when requesting quota usage list

Bug Report

Module: github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/quota/armquota v0.5.0

SDK version : v65.0.0

commit : 7393edcd0c85ff6a71dc7b30e28d48629bc69f6b

go version go1.18.3 linux/amd64

What happened?

A quota NewUsagesClient was created and the NewListPager method is called, which uses the listCreateRequest  method. The following error is produced :
2022/07/09 19:45:59 failed to advance page: GET https://management.azure.com/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx/providers/Microsoft.Compute/locations/westeurope/providers/Microsoft.Quota/usages
--------------------------------------------------------------------------------
RESPONSE 404: 404 Not Found
ERROR CODE: InvalidResourceType
--------------------------------------------------------------------------------
{
  "error": {
    "code": "InvalidResourceType",
    "message": "The resource type 'locations' could not be found in the namespace 'Microsoft.Compute' for api version '2021-03-15-preview'. The supported api-versions are '2015-05-01-preview,2015-06-15,2016-03-30,2016-04-30-preview,2016-08-30,2017-03-30,2017-12-01,2018-04-01,2018-06-01,2018-10-01,2019-03-01,2019-07-01,2019-12-01,2020-06-01,2020-12-01,2021-03-01,2021-04-01,2021-07-01,2021-11-01,2022-03-01'."
  }
}
--------------------------------------------------------------------------------
exit status 1

What did you expect or want to happen?

Get a list of current usage for all resources for the scope specified, same as provided here

How can we reproduce it?

Create an NewUsagesClient and call the NewListPager method :
func getQuota(location, quotaName, limit string) {

	credentials, err := azidentity.NewClientSecretCredential(tenantId, clientId, clientSecret, nil)

	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()

	client, err := armquota.NewUsagesClient(credentials, nil)

	if err != nil {
		log.Fatal(err)
	}

	pager := client.NewListPager("subscriptions/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx/providers/Microsoft.Compute/locations/westeurope",
	nil)

		for pager.More() {
		nextResult, err := pager.NextPage(ctx)
		if err != nil {
			log.Fatalf("failed to advance page: %v", err)
		}
		for _, v := range nextResult.Value {
			// TODO: use page item
			_ = v
		}
	}
}

Anything we should know about your environment.

Azure Public Cloud

Thank you,
Erlis

This seems to be a service issue.  @tadelesh, could you help to take a look?

@erlisb Thanks for your feedback. It seems the service use the wrong api-version for the resource query. I'll involve service team to have a look.

@erlisb Apologies for the late reply. Thanks for reaching out to us and reporting this issue. We are looking into this issue and we will provide an update.

@erlisb The QuotaRP is in private preview, and the Compute RP is not supported, only network RP is supported.

@erlisb The Compute RP will be supported for Quota API in a couple of weeks tentatively. Hope this helps.

@erlisb I wanted to do quick follow-up to check if you had a chance to look at my above comment. Please let us know if you need any further assistance on this. Awaiting your reply.

@navba-MSFT , I will check it again and provide you a feedback. Thanks.

@erlisb Thanks for getting back. Like I mentioned above The Compute RP will be supported for Quota API in a couple of weeks tentatively. So you might have to try it then.

Hi @navba-MSFT, it seems that neither of RPs (Compute, Network & Storage) is working in the current armquota version :
Microsoft.Compute :
{
  "error": {
    "code": "InvalidResourceType",
    "message": "The resource type 'locations' could not be found in the namespace 'Microsoft.Compute' for api version '2021-03-15-preview'. The supported api-versions are '2015-05-01-preview,2015-06-15,2016-03-30,2016-04-30-preview,2016-08-30,2017-03-30,2017-12-01,2018-04-01,2018-06-01,2018-10-01,2019-03-01,2019-07-01,2019-12-01,2020-06-01,2020-12-01,2021-03-01,2021-04-01,2021-07-01,2021-11-01,2022-03-01,2022-08-01'."
  }
}

Microsoft.Network :
{
  "error": {
    "code": "InvalidResourceType",
    "message": "The resource type 'locations' could not be found in the namespace 'Microsoft.Network' for api version '2021-03-15-preview'. The supported api-versions are '2014-12-01-preview,2015-05-01-preview,2015-06-15,2016-03-30,2016-06-01,2016-07-01,2016-08-01,2016-09-01,2016-10-01,2016-11-01,2016-12-01,2017-03-01,2017-04-01,2017-06-01,2017-08-01,2017-09-01,2017-10-01,2017-11-01,2018-01-01,2018-02-01,2018-03-01,2018-04-01,2018-05-01,2018-06-01,2018-07-01,2018-08-01,2018-10-01,2018-11-01,2018-12-01,2019-02-01,2019-04-01,2019-06-01,2019-07-01,2019-08-01,2019-09-01,2019-11-01,2019-12-01,2020-01-01,2020-03-01,2020-04-01,2020-05-01,2020-06-01,2020-07-01,2020-08-01,2020-11-01,2021-01-01,2021-02-01,2021-03-01,2021-04-01,2021-05-01,2021-06-01,2021-08-01,2021-12-01,2022-01-01'."
  }
}

Microsoft.Storage :
{
  "error": {
    "code": "InvalidResourceType",
    "message": "The resource type 'locations' could not be found in the namespace 'Microsoft.Storage' for api version '2021-03-15-preview'. The supported api-versions are '2021-09-01,2021-08-01,2021-06-01,2021-05-01,2021-04-01,2021-02-01,2021-01-01,2020-08-01-preview,2019-06-01,2019-04-01,2018-11-01,2018-07-01,2018-03-01-preview,2018-02-01,2017-10-01,2017-06-01,2016-12-01,2016-07-01,2016-01-01'."
  }
}

Do you know if there is an actual ETA for this fix ?  A couple of weeks doesn't seem very optimistic.
Thank you,
Erlis

@erlisb Thanks for getting back. As mentioned in my above comment, Compute RP is yet to be supported. So the above error is expected now. There are a few more RPs along with Compute RP which will be supported. I didn't get a fixed ETA from the Product Owners. So I can only provide the tentative ETA as of now which is in next 2 - 4 weeks.

@erlisb I hope you had a chance to look at the above comment. We will proceed with archival of this GitHub thread. The fix will be released in couple of weeks, post that if you see the same issue, feel free to reopen this thread. We would be happy to help.

Hi @navba-MSFT,
is there any update on this issue ?
Thanks,
Erlis

@erlisb Are you still facing the same issue ? If Yes, let me know. I will follow-up with the product owners and update the status. Awaiting you reply.

Hi @navba-MSFT , I'm facing the same issue on both Microsoft.Network and Microsoft.Compute RP. Are any of those supported?
Using the SDK documentation example (with my subscription and region names), I also get similar failure messages: "The resource type 'locations' could not be found in the namespace 'Microsoft.Network'", "The resource type 'locations' could not be found in the namespace 'Microsoft.Compute'").


fix: adjusted execute order of 1.2.1, 1.2.2, 1.2.3 to prevent failure in AWS

Overall Review of Changes
Using Packer to create hardened AMI for Windows Server 2022 was failing due to the lockoutduration being less than the lockoutcounter.
Issue Fixes
Changing the order of execution to satisfy lockoutduration value set prerequisites. See https://github.com/ansible/ansible/issues/62594#issuecomment-533455130

@ChiefHolland
Thanks for the PR and good catch on the var in defaults/main. I'm trying to sort out the order change for 1.2.1, 1.2.2, and 1.2.3. The order they are in with the role are the only order that work for a physical box I have, VM's in vagrant, and our AWS accounts. It looks like that order change was something related to your packer build, can you add a bit more detail there?
-George


PacketInClientCacheStatus.java:47: Add blobCacheSupport...

The puzzle 1-929b23ba from #1 has to be resolved:
https://github.com/shiruka/shiruka/blob/a80bae3326a07eeda14f1428399b4deef5ac061d/src/main/java/io/github/shiruka/shiruka/network/packets/PacketInClientCacheStatus.java#L47-L47
The puzzle was created by Hasan Demirtaş on 04-Jan-21.
Estimate: 15 minutes,  role: DEV.
If you have any technical questions, don't ask me, submit new tickets instead. The task will be "done" when the problem is fixed and the text of the puzzle is removed from the source code. Here is more about PDD and about me.

The puzzle 1-929b23ba has disappeared from the source code, that's why I closed this issue.

The puzzle 1-929b23ba has disappeared from the source code, that's why I closed this issue.


DATETIME AND SMALLDATETIME 's corresponding JDBC types are wrong

Driver version or jar name
The Microsoft JDBC Drivers 6.0 and 4.2 and The Microsoft JDBC Driver 6.2
SQL Server version
SQL Server 2012
Client operating system
Windows and Linux
Java/JVM version
java 1.8.0_144
Table schema
TABLE_QUALIFIER                                     TABLE_OWNER                                         TABLE_NAME                                          COLUMN_NAME                                         DATA_TYPE  TYPE_NAME                                           PRECISION    LENGTH       SCALE   RADIX   NULLABLE  REMARKS                                             COLUMN_DEF                                          SQL_DATA_TYPE  SQL_DATETIME_SUB  CHAR_OCTET_LENGTH  ORDINAL_POSITION  IS_NULLABLE                                         SS_DATA_TYPE
Testmysql                                           dbo                                                 datetimetypes                                       date_type                                           11         datetime                                            23           16           3         1                                                        9              3                              1                 YES                                                 111
Problem description
Datatypes DATETIME AND SMALLDATETIME in the JDBC Drivers 6.0 and 4.2 and 6.2 all are defined as part of FQN microsoft.sql.Types and rather part of java.sql.Types , there is a difference in the values they represent microsoft.sql.Types.DATETIME and microsoft.sql.Types.SMALLDATETIME which is -151 and -150 respectively where as java.sql.Types.TIMESTAMP/TIME/DATE has values 93,92,91 . But here is the problem ,  https://docs.microsoft.com/en-us/sql/connect/jdbc/using-basic-data-types under Data Type Mappings represent datetime and smalldatetime having equivalent JDBC type as java.sql.Types.TimeStamp , which is not right , this assumption is being used in downstream applications ## Expected behavior and actual behavior
Either the doc https://docs.microsoft.com/en-us/sql/connect/jdbc/using-basic-data-types needs correction or DATETIME and SMALLDATETIME should be represented as java.sql.Types and not microsoft.sql.Types
Repro code
// scala code
val stmt = connection.prepareStatement("SELECT * FROM datetimetypes WHERE 1=0")
val resultSet = stmt.executeQuery()
val rsmd = resultSet.getMetaData
val ncols = rsmd.getColumnCount
var i = 0
while (i < ncols) {
val columnName = rsmd.getColumnLabel(i + 1)
val dataType = rsmd.getColumnType(i + 1)
val typeName = rsmd.getColumnTypeName(i + 1)
println("Datatype  --> ",datatype)
}

Hi @KenpachiRules,
Thank you for creating the issue. @peterbae , can we update the documentation? Maybe a note explaining the difference?

Hi @KenpachiRules. The document is correct in saying that DATETIME and SMALLDATETIME maps to java.sql.Timestamp. The "Data Type Mappings" table correctly shows that the user is supposed to provide a java.sql.Timestamp value to insert into a DATETIME or SMALLDATETIME column in SQL Server. The different type codes between the data types you mentioned are irrelevant to how the data types are mapped, and the microsoft.sql.Types.DATETIME and SMALLDATETIME can be used by the user along with getObject method to specify the data type for an encrypted table. I hope this clears things up.

Hi @peterbae  The ResultSet obtained on executing a schema query through JDBC driver in that case should return type code 93 which is the value for TimeStamp , types code are vital because that is what precisely java.sql.Types represent , if the resulting ResultSet.getMetdata.getColumnType returns a value other than that how can data type mappings be claimed correct , because the value returned is not 93 which java.sql.Types.TIMESTAMP represents.

Will spawn a new issue as it is critical.

Hi @KenpachiRules, I just tried an example myself and the getColumnType against a DATETIME column from SQL Server returns 93 for me. Could you provide me with a repro code that returns -150 for a DATETIME column?

import java.sql.DriverManager
import scala.collection.mutable.Set
import java.sql.ResultSet
import java.sql.SQLException
object JDBCMetadataTest {
def main(args: Array[String]): Unit = {
val connProp = "jdbc:sqlserver://hostname;databaseName=dbName"
val connection = DriverManager.getConnection(connProp, "username", "username")
val stmt = connection.prepareStatement(getSchemaQuery("datetimetest"))
getSchema(stmt.executeQuery())
}
def getSchemaQuery(table: String): String = {
s"SELECT * FROM $table WHERE 1=0"
}
def getSchema(
resultSet: ResultSet,
alwaysNullable: Boolean = false): Unit = {
val rsmd = resultSet.getMetaData
val ncols = rsmd.getColumnCount
var i = 0
while (i < ncols) {
val columnName = rsmd.getColumnLabel(i + 1)
val dataType = rsmd.getColumnType(i + 1)
val typeName = rsmd.getColumnTypeName(i + 1)
val fieldSize = rsmd.getPrecision(i + 1)
val fieldScale = rsmd.getScale(i + 1)
val isSigned = {
try {
rsmd.isSigned(i + 1)
} catch {
case e: SQLException => e.printStackTrace
}
}
val nullable = if (alwaysNullable) {
true
} else {
}
i = i + 1
}
}
}
//  val dataType = rsmd.getColumnType(i + 1) for datetime would return -150 , tested against server 2012 SP1 and with JDBC driver 6.0 client with both Sqljdbc41.jar and Sqljdbc42.jar

As an additional info common types like float , int , varchar all return types compliant with documented values , this issue seems to occur for all types defined in microsoft.sql.Types , tested for money and smallmoney , the same issue , it returns values -148 and -146 and not 3 which is the value for java.sql.Types.DECIMAL

Re-opening issue so we can track it better.

Hi @KenpachiRules, thanks for the detailed information. I tested again with the same settings as you (JDBC v6.0, with SQL Server 2012). For my repro case, the SQL Server does return the SSType as DATETIME, which gets converted to java.sql.TIMESTAMP in SQLServerResultSetMetadata::getColumnType method. Could you debug into that method and show me why the returned integer from the getColumnType method is -150?
Also, JDBC Driver 6.0 is an older release - it might help for you to use the latest JDBC Driver, v6.4.0.

Hi @KenpachiRules, have you had a chance to look into this issue a bit more? Thanks.


Disable bitcode for all targets

As far as I'm aware, bitcode is not included in the XCTest framework so compiling Kiwi with bitcode enabled just complicates things.

When would you like to merge it to master ?

Yep, agreed, let's get this in 👍

:tada:


e2e: OpenStack CPMS created automatically

We manage the OpenStack CPMS via the installer, so we can switch the e2e
in Full, so the framework won't create it for us.
Depends-on: https://github.com/openshift/installer/pull/7280

/cc pierreprinetti
since the installer PR is not merged yet, the e2e-openstack-operator should fail. It's a good first test to make sure it actually fails, and once we have the installer patch we'll retest and make sure the test passes.

/test e2e-openstack-operator

I don't understand how e2e-openstack-operator could pass, I'll investigate a bit.

/retest

/retest e2e-openstack-operator

/test e2e-openstack-operator

@damdo @JoelSpeed for review please

/approve
/lgtm

/retest-required

/retest-required

/retest-required

/retest-required


updtade branch smartc

Description
Concise description of proposed changes, We recommend using screenshots and videos for better description
Additional Information

[ ] I have read the contributing docs (if this is your first contribution)
[ ] This is not a duplicate of any existing pull request

Related Issues
Closes #{issue number}
Note: If your changes are small and straightforward, you may skip the creation of an issue beforehand and remove this section. However, for medium-to-large changes, it is recommended to have an open issue for discussion and approval prior to submitting a pull request.
Your ENS/address:

update branch smartc


Refactor sign-types to remove libsecp256k1 dep

Description
Refactoring that allows us to remove libsecp256k1 dependency altogether
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[x] Refactor

@roynalnaruto conflicts


Server Login to Rocket.Chat Cloud does not work

Ubuntu Snap server latest
Login to Rocket.Chat Cloud does not work
I get 400 oauth_workspace_client_different_account
also cannot sync
rocketchat-server.rocketchat-server[19427]: Failed to sync with Rocket.Chat Cloud.  Error: internal_server_error

sorry it's for rockechat server, I am going to change it.

I'm having the same issue trying to connect my self-hosted instance to the cloud.
In DevTools, after login,  it says:
400: oauth_workspace_client_different_account
Have you been able to solve this problem? How?

Sorry for reopening.... Did you solve the problem? I have the same issue

I have the same issue RocketChat 3.6.2 snap


Unable to recover from Network Error until reboot Gitify

🔍 Is there already an issue for your problem?

[X] I have checked older issues, open and closed

📝 Description
Since v5.12.0, an error screen that shows Network Error has appeared in my environment. Once in this state, Gitify does not seem to recover from this state until it is restarted.

Here, I enabled debug mode and tried reloading, which did not recover from the error screen despite everything returning 200 for each requests.

When Gitify was simply restarted in this state, everything worked correctly.
🪜 Steps To Reproduce

Open Gitify and confirm that it is fine.
Turn off Wi-Fi and/or disconnect from wired network.
Reload Gitify manually

Gitify Version
Other
Operating System
macOS
GitHub Account
Combination
📸 Screenshots
No response

Thank you @pddg for the detailed issue. Will investigate this today

Same but the error icon isnt even rendering for me (Windows)

@setchy what about the icon not rendering? any idea why is that?

@Araxeus - 🤷 we're using the 🛜 emoji.  Does that not exist on Windows? 🤔

I've never seen it before
I assumed it was a bug because the screenshot on mac has a WIFI icon
for me this is just an empty rectangle and doesn't look intentional

for me this is just an empty rectangle and doesn't look intentional

Certainly not intentional :)

oh LMAO i actually thought it was because even on browser it looks like that:

raised https://github.com/gitify-app/gitify/issues/1446

oh LMAO i actually thought it was because even on browser it looks like that:

oh man, who would have thought!

@setchy
I have confirmed that this issue has been fixed in v5.12.1. Thank you for your quick response!

@setchy
I have confirmed that this issue has been fixed in v5.12.1. Thank you for your quick response!

Thank you for reporting and confirming the fix. Greatly appreciated.


Verify eicr setId is being added to eCR Data Store

We need to ensure the eicr setId is being pulled over during FHIR Conversion and that it is also a column within the eCR data store extract, with the proper FHIR Path.

I'll need to add an extension into our FHIR Converter to grab and store the setId for eicr and then add a column to our ecr data store for that field.  I have a ticket for this work and will just add the detail to it.

According to the HL7 FHIR EICR Profile IG/Documentation (http://build.fhir.org/ig/HL7/case-reporting/StructureDefinition-eicr-composition-definitions.html#Composition.identifier) it states:  "Public Health Composition identifier. This is equivalent to ClinicalDocument.setId in CDA. The identifier of the containing Bundle is equivalent to ClinicalDocument.id in CDA and Composition.sliceExtensionVersionNumber is equivalent to ClinicalDocument.versionNumber in CDA."


Container not start after Host reboot (Exit code 127)

My config:

Windows 11 x64
WSL - Debian x64

Here my docker compose file :
  dnsmasq:
    image: dockurr/dnsmasq
    container_name: dnsmasq
    environment:
      DNS1: "<IP_ADDRESS>"
      DNS2: "<IP_ADDRESS>"
    ports:
      - 53:53/udp
      - 53:53/tcp
    cap_add:
      - NET_ADMIN
    restart: always
    volumes:
      - ./dnsmasq/etc/dnsmasq.d/:/etc/dnsmasq.d/
      - ./dnsmasq/etc/dnsmasq-hosts.conf:/etc/dnsmasq-hosts.conf
    networks:
      entrypoint:
        ipv4_address: <IP_ADDRESS>

All works perfectly... after host reboot... the container have exit code 127 :

If i start manually the container its works.
Have you an idea how to fix it please ?
Thanks

I have see in /var/log but the folder IS empty... Journal is it in another
folder ?
Le mar. 21 mai 2024, 17:10, Kroese @.***> a écrit :

Is there nothing more in the log than what is visible in the screenshot?
Because I would expect some message about why it is quitting?
—
Reply to this email directly, view it on GitHub
https://github.com/dockur/dnsmasq/issues/4#issuecomment-2122856053, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAA3TIMEAKC26TEDYAVH7ADZDNPU3AVCNFSM6AAAAABIBXPPTSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRSHA2TMMBVGM
.
You are receiving this because you authored the thread.Message ID:
@.***>

In docker logs there are no more than my screen shots :
docker container logs dnsmasq --details

Ok... i have take docker daemon log here :
-------------------------------------------------------------------------------->8
time="2024-05-22T07:42:19.303808653Z" level=info msg="Starting up"
time="2024-05-22T07:42:19.359921407Z" level=info msg="[graphdriver] using prior storage driver: overlay2"
time="2024-05-22T07:42:19.694401787Z" level=info msg="Loading containers: start."
time="2024-05-22T07:42:19.831159396Z" level=info msg="Removing stale sandbox a37e845ff746e09cfc8982ec3b879dcc77dcfc28db32095950f4bfc6e345a3b5 (408a51e8d552c42026d4cfd442593e35a7775cad68b07698a571d4de3a0f0624)"
time="2024-05-22T07:42:19.838763625Z" level=warning msg="Failed to clean up network resources on container maildev disconnect: failed to set gateway while updating gateway: route for the gateway <IP_ADDRESS> could not be found: network is unreachable"
time="2024-05-22T07:42:19.843670393Z" level=warning msg="Error (Unable to complete atomic operation, key modified) deleting object [endpoint 7ee46e9cf101b1a931dcbf830536a97f6a6ee53f7c46ce44babee638c7abacf3 c0144edcf3d3d8df4abf9677551cc41c29c906d14eb1069a19222232c3686053], retrying...."
time="2024-05-22T07:42:19.851874931Z" level=warning msg="Error (Unable to complete atomic operation, key modified) deleting object [endpoint 0f8bafc6e3d9a9fdcb364b051e343df7e1a2b9fee5c90be9fc2539a0e6ef025a d0fe3e6245beeaf86c9bb37fbdbb34c9ea603aa5b358ced0b0fcbdffc39a5c6b], retrying...."
time="2024-05-22T07:42:19.856336868Z" level=info msg="Removing stale sandbox a86d09fc66a709e357fc9816a50b5119d8d28295370afdd2ac5eedae6133303a (5f072549c3e7779fccdc1fcbbcec07a293ca9a9fcc5c5af62bc6eb8a86b0cabe)"
time="2024-05-22T07:42:19.864830501Z" level=warning msg="Error (Unable to complete atomic operation, key modified) deleting object [endpoint 7ee46e9cf101b1a931dcbf830536a97f6a6ee53f7c46ce44babee638c7abacf3 15ed2e8b3fbba763e7c0af1ea36084c91e81a3b37261461d7b10df5a51d4bdde], retrying...."
time="2024-05-22T07:42:19.869109472Z" level=info msg="Removing stale sandbox faabae5311a82025b1f9a83589d39600ec6e8f7c1db94a843b36d658982d43c9 (83e6075e221b35d57574d3aab2ad54ad4db82a9ece67d67dcdc201d7690deba8)"
time="2024-05-22T07:42:19.872990252Z" level=warning msg="Failed to clean up network resources on container php55 disconnect: failed to set gateway while updating gateway: route for the gateway <IP_ADDRESS> could not be found: network is unreachable"
time="2024-05-22T07:42:19.877685763Z" level=warning msg="Error (Unable to complete atomic operation, key modified) deleting object [endpoint 7ee46e9cf101b1a931dcbf830536a97f6a6ee53f7c46ce44babee638c7abacf3 2ba75aeb770050944462a3129f632789d6464f452280ae50438e8e044bcb591e], retrying...."
time="2024-05-22T07:42:19.886465506Z" level=warning msg="Error (Unable to complete atomic operation, key modified) deleting object [endpoint 0f8bafc6e3d9a9fdcb364b051e343df7e1a2b9fee5c90be9fc2539a0e6ef025a 5310ab3cf5a7b9aebc9ec3b5dca3d4d96464b8922b58fade93eaca1ae50f88a2], retrying...."
time="2024-05-22T07:42:19.890942793Z" level=info msg="Removing stale sandbox 1bc6d1c019dac3a91d01979fa6286f6cf3bf6c989d213e7497f4acdceea8dfa2 (1e5ae2448c050f60314d4df952a2daba68d8c794fd1fe0cb2f8c1f778d9ee632)"
time="2024-05-22T07:42:19.902447722Z" level=warning msg="Error (Unable to complete atomic operation, key modified) deleting object [endpoint f80731d77e30f19ce4f2388789b0180d59b40943a6dfba88e1ae2500d08c9964 4da376f8899fb446dac228f5937464a259d3bfe9fd5f17821257046bdc25b758], retrying...."
time="2024-05-22T07:42:19.906894817Z" level=info msg="Removing stale sandbox 547611f84708dc692d85d885c100acd90aae87fdfd1c7cd51e0665fa6b7a09d7 (44f9f522b840b76c54f9e8de897a4485449e95118aba6d1e8bc89942b335e7e6)"
time="2024-05-22T07:42:19.919765149Z" level=warning msg="Failed to clean up network resources on container nginxproxymanager disconnect: failed to set gateway while updating gateway: route for the gateway <IP_ADDRESS> could not be found: network is unreachable"
time="2024-05-22T07:42:19.924602848Z" level=warning msg="Error (Unable to complete atomic operation, key modified) deleting object [endpoint f80731d77e30f19ce4f2388789b0180d59b40943a6dfba88e1ae2500d08c9964 87fec57c75fd79051824b4e1be830346bd61c80a41ee8fe2851db7577a0fc91b], retrying...."
time="2024-05-22T07:42:19.932617781Z" level=warning msg="Error (Unable to complete atomic operation, key modified) deleting object [endpoint 0f8bafc6e3d9a9fdcb364b051e343df7e1a2b9fee5c90be9fc2539a0e6ef025a 49c9008388fa33a9b425162b3eecf397d2d403b3d0beb90532184344291fb9a1], retrying...."
time="2024-05-22T07:42:19.936894252Z" level=info msg="Removing stale sandbox 9d5e2d9f814bf62bb3890d2fc3f0aa3d2ba06e54a683cb3bea34f95763b5bbe0 (780c263d53e50cfc27d70dd7c0dea5660b261a2a483363bbc2056b1673e0866c)"
time="2024-05-22T07:42:19.940705328Z" level=warning msg="Failed to clean up network resources on container php82 disconnect: failed to set gateway while updating gateway: route for the gateway <IP_ADDRESS> could not be found: network is unreachable"
time="2024-05-22T07:42:19.945437811Z" level=warning msg="Error (Unable to complete atomic operation, key modified) deleting object [endpoint 7ee46e9cf101b1a931dcbf830536a97f6a6ee53f7c46ce44babee638c7abacf3 977b628face13b2b34a13945382143a7f7947781cff67c04e2ed5192d69ef9e0], retrying...."
time="2024-05-22T07:42:19.952923733Z" level=warning msg="Error (Unable to complete atomic operation, key modified) deleting object [endpoint 0f8bafc6e3d9a9fdcb364b051e343df7e1a2b9fee5c90be9fc2539a0e6ef025a 3f2ecaa1523740a7b83ae17011ec4a276d958615fd107ec36b79e3bc62395e0b], retrying...."
time="2024-05-22T07:42:19.969584988Z" level=info msg="Default bridge (docker0) is assigned with an IP address <IP_ADDRESS>/16. Daemon option --bip can be used to set a preferred IP address"
time="2024-05-22T07:42:20.180761625Z" level=error msg="stream copy error: reading from a closed fifo"
time="2024-05-22T07:42:20.180761767Z" level=error msg="stream copy error: reading from a closed fifo"
time="2024-05-22T07:42:20.188655881Z" level=error msg="1e5ae2448c050f60314d4df952a2daba68d8c794fd1fe0cb2f8c1f778d9ee632 cleanup: failed to delete container from containerd: container \"1e5ae2448c050f60314d4df952a2daba68d8c794fd1fe0cb2f8c1f778d9ee632\" in namespace \"moby\": not found"
time="2024-05-22T07:42:20.928765400Z" level=error msg="stream copy error: reading from a closed fifo"
time="2024-05-22T07:42:20.928850683Z" level=error msg="stream copy error: reading from a closed fifo"
time="2024-05-22T07:42:21.677627034Z" level=error msg="failed to start container" container=1e5ae2448c050f60314d4df952a2daba68d8c794fd1fe0cb2f8c1f778d9ee632 error="failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting \"/run/desktop/mnt/host/wsl/docker-desktop-bind-mounts/Debian/8180a849bfa4ee7b74b0b4ef8557a35fc8c707ec0d2e7af244e6c3668643806a\" to rootfs at \"/etc/dnsmasq-hosts.conf\": mount /run/desktop/mnt/host/wsl/docker-desktop-bind-mounts/Debian/8180a849bfa4ee7b74b0b4ef8557a35fc8c707ec0d2e7af244e6c3668643806a:/etc/dnsmasq-hosts.conf (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type"
time="2024-05-22T07:42:21.839189393Z" level=error msg="failed to start container" container=1172309c5b2381798be48f7d64f669d9976af9ef2abd4c26bd52a8713aa51bfe error="failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting \"/run/desktop/mnt/host/wsl/docker-desktop-bind-mounts/Debian/0f34a228a4cf409ef06b81b212be2f5c797ccd0bbe8c7d0cb167d6add969ad92\" to rootfs at \"/root/.gitconfig\": mount /run/desktop/mnt/host/wsl/docker-desktop-bind-mounts/Debian/0f34a228a4cf409ef06b81b212be2f5c797ccd0bbe8c7d0cb167d6add969ad92:/root/.gitconfig (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type"
time="2024-05-22T07:42:22.218046970Z" level=info msg="Loading containers: done."
time="2024-05-22T07:42:22.225903759Z" level=warning msg="WARNING: No blkio throttle.read_bps_device support"
time="2024-05-22T07:42:22.225927591Z" level=warning msg="WARNING: No blkio throttle.write_bps_device support"
time="2024-05-22T07:42:22.225930895Z" level=warning msg="WARNING: No blkio throttle.read_iops_device support"
time="2024-05-22T07:42:22.225933294Z" level=warning msg="WARNING: No blkio throttle.write_iops_device support"
time="2024-05-22T07:42:22.225936207Z" level=warning msg="WARNING: daemon is not using the default seccomp profile"
time="2024-05-22T07:42:22.225947676Z" level=info msg="Docker daemon" commit=ac2de55 containerd-snapshotter=false storage-driver=overlay2 version=26.1.1
time="2024-05-22T07:42:22.226203383Z" level=info msg="Daemon has completed initialization"
time="2024-05-22T07:42:22.627591302Z" level=info msg="API listen on /var/run/docker.sock"
time="2024-05-22T07:42:22.627608075Z" level=info msg="API listen on /run/guest-services/docker.sock"
time="2024-05-22T07:42:22.633793695Z" level=error msg="Handler for GET /v1.45/services returned error: This node is not a swarm manager. Use \"docker swarm init\" or \"docker swarm join\" to connect this node to swarm and try again." spanID=43dd6cfff28241f5 traceID=f862a314e8fd9db8fa2be85f36fa36e4
2024/05/22 07:42:22 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
2024/05/22 07:42:22 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
time="2024-05-22T07:42:22.983641237Z" level=error msg="Handler for GET /v1.45/services returned error: This node is not a swarm manager. Use \"docker swarm init\" or \"docker swarm join\" to connect this node to swarm and try again." spanID=41cbf63e670495a3 traceID=994b964566726eaaa6e929092e79d6bf
time="2024-05-22T07:42:23.625824434Z" level=error msg="Handler for GET /v1.45/services returned error: This node is not a swarm manager. Use \"docker swarm init\" or \"docker swarm join\" to connect this node to swarm and try again." spanID=395d6e205884f913 traceID=b05da93237eeaf577294ce9969a93c18
time="2024-05-22T07:42:29.215860954Z" level=error msg="[resolver] failed to query external DNS server" client-addr="udp:<IP_ADDRESS>:32956" dns-server="udp:<IP_ADDRESS>:53" error="read udp <IP_ADDRESS>:32956-><IP_ADDRESS>:53: i/o timeout" question=";ip-ranges.amazonaws.com.\tIN\t A" spanID=1da8e0592ec64ff1 traceID=98a5a293d220186c586438f8287f9b0c
2024/05/22 07:42:31 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
2024/05/22 07:42:31 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
2024/05/22 07:42:31 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
time="2024-05-22T07:42:33.217082919Z" level=error msg="[resolver] failed to query external DNS server" client-addr="udp:<IP_ADDRESS>:54285" dns-server="udp:<IP_ADDRESS>:53" error="read udp <IP_ADDRESS>:54285-><IP_ADDRESS>:53: i/o timeout" question=";ip-ranges.amazonaws.com.\tIN\t A" spanID=aaac436187d7a810 traceID=deee9fb571cb44a9e5f3c91239e0182b
2024/05/22 07:43:03 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
2024/05/22 07:43:03 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
2024/05/22 07:43:03 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
2024/05/22 07:43:03 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
2024/05/22 07:43:16 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)

I don't understand why it can't mount file... :S


Investigate this intermittent failure in Elixir build inside nix

https://github.com/build-trust/ockam/actions/runs/6056195233/job/16435982912?pr=5825#step:5:616

Another instance of this https://github.com/build-trust/ockam/actions/runs/6166649979/job/16736470350#step:5:605

Another instance https://github.com/build-trust/ockam/actions/runs/6181384279/job/16779253708#step:5:622

Two more:
https://github.com/build-trust/ockam/actions/runs/6181552961/job/16779719038
https://github.com/build-trust/ockam/actions/runs/6181552961/job/16779722345

@metaclips this one seems to happening frequently

@mrinalwadhwa I updated the Nix lockfile to take a stab at this issue, https://github.com/build-trust/ockam/pull/5973, please review

This is still re-occuring. Reopening now


Link to third-party pages from license list

This is not critical, and it would require API extentions, but it would make it easier for users to audit our assertions and/or discover additional details if the “FSF Free/Libre?” and “OSI Approved?” entries in resources/htmlTemplate/TocHTMLTemplate.html were also links to the third-party page.  For example, if the MIT entries linked to https://www.gnu.org/licenses/license-list.html#Expat and https://opensource.org/licenses/MIT:

Full name
Identifier
FSF Free/Libre?
OSI Approved?
Text

MIT License
MIT
Y
Y
License Text

Definitely not something to block a release on or anything, but if someone has the time to work this up, I think it would be a useful improvement.

This would be a bit of work - I'll can look into after the next release of the license list (which is keeping me pretty busy).
If anyone else wants to take this on - feel free, I can help provide pointers to the existing implementations

Moved to https://github.com/spdx/LicenseListPublisher/issues/19


chore: add queue to workflows config

Needed for workflows runners to pick up tasks.
Type of change

Chore (any other change that doesn't affect source or test files, such as configuration)

Not needed anymore. I was working off of an older main locally that didn't have this.


Bug: Short values are not recognized

What type of issues are you dealing with?
Incorrect highlighting
What happened?

Extension version: 0.6.0
Which text editors are you seeing the problem on?
VSCode
Include a code snippet
say -1L -1l -1S -1s -1 0 1 1b 1s 1S 1l 1L

fixed


Change List

- FontAwsome Added
- Template and CSS require system updated to angular2 from Commonjs for better usablity in Visual Studio (2015) error tracking
- "extract-text-webpack-plugin" version updated
- webpack.vendor file updated

Can you please check and if find this helpful, please merge?
I have tested myself everything and nothing broken

I am starting on signalR from now.
After that is done, will I create a pull request here like this?

Great! Yes just make a new PR for it :)
You can remove any of the old/dead WebSockets stuff as well if you'd like!


Rewrite Dub to PHP

Is your feature request related to a problem? Please describe.
I think a rewrite of Dub in PHP would be very beneficial to the speed, development experience and codebase maintainability for the developers.
Describe the solution you'd like
I really believe Dub being rewritten to PHP would be a great project, and would make everything better.
Describe alternatives you've considered
Dub first being in C# was okay, but TypeScript is soooo slow. PHP is objectively the best language for an osu! server hybrid.


Refactors ensembler, online-ensembler-forecaster and descendants

Reference : https://github.com/alan-turing-institute/sktime/issues/955
We need to slowly re-factor all forecasters to be compliant with the new interface (see PR #912). i.e., no overrides of fit etc and descendant classes only implementing core logic, while the BaseForecaster implements check etc logic.
Currently this is running due to various downwards compatibility workarounds, which should be removed once all forecasters are ready (in "no-override" state)
The forecasters that still need a downstream refactor are:

[x] Online Ensembler Forecaster , Descendant and Ensembler

Could you point me to where you are referring to @fkiraly

Could you point me to where you are referring to @fkiraly

the update/_update function, currently line 59.


CliRunner.invoke takes mix_stderr which does nothing

mix_stderr is a parameter of CliRunner.invoke.
This is not documented in the docstring as a :param.
Simply reading the code (which I did after trying this and seeing no output) shows that this parameter is not used.
mix_stderr works on CliRunner.__init__.
The mix_stderr parameter should be removed, or changed to work and get documented.

Fairly sure this is fixed in master

@davidism I believe that it is not.
See on master the parameter list includes mix_stderr:
https://github.com/pallets/click/blob/6944450c5331f2d652d342650b6d9b3b52404baa/click/testing.py#L280-L281
The parameter documentation does not include mix_stderr:
https://github.com/pallets/click/blob/6944450c5331f2d652d342650b6d9b3b52404baa/click/testing.py#L296-L311
And the function code does not include mix_stderr:
https://github.com/pallets/click/blob/6944450c5331f2d652d342650b6d9b3b52404baa/click/testing.py#L312-L357

I think this is at least partially fixed here:
https://github.com/pallets/click/pull/1194

I think this is at least partially fixed here:
#1194

@jcrotts I don't think that #1194 relates to this issue - this issue concerns CliRunner.invoke and that PR concerns CliRunner.__init__.

Some behavior is changed in CliRunner.invoke, the docstring isn't changed though.
https://github.com/pallets/click/pull/1194/files#diff-637edab3f8e0763a9874d17879193dc9R350

I still don't understand how that in any way addresses this issue, which is that the mix_stderr parameter to CliRunner.invoke does nothing.

Hi Adam, you seem to have a specific idea in mind that we're having trouble understanding. If this is important to you and you have the time, it would probably be easier to create a PR fixing the issue rather than going back and forth more.

Good idea @davidism . I have created #1442 . I'm happy to back port / change this if you are happy with the general idea.

@jcrotts @davidism - I hope that the linked PR makes it clear what this issue is.


Add the option to customize the "stats" attribute

There should be an option to customize webpack's stats attribute.
To solve this issue on mini-css-extract-plugin, it is required to change the property.

Hi @danilofuchs, is this still an issue for you? Have you found a workaround or created your own function for handling this change?

As I'm currently using Styled Components, not Sass, I haven't faced this issue in some time. But to my understanding, this is still a major issue for Sass users, but a simple customize-cra function is viable to those still needing to fix this.

If you think this is out of the scope of this project, I'm happy to close the issue and propose a user-made solution

I'm fine with that being added, if anyone would like to open a PR.

I'll work on it!

Thank you! Sorry for taking so long to say that 😆 looking forward to it!

I have been working on it, but I can't seem to make create-react-app respect the config. I think their implementation of error / warning handling is overriding webpack's actions.


Ensure Non-Conflicting Kustomization by default in Lifecycle-Manager config directories

In order to use consistent, non-conflicting labels in Kustomize, we need to adjust the default kubebuilder generated labels and kustomization configuration. This will allow us to run with other kubebuilder operators in the same namespace without custom patches.
AC:
change all manual occurrences in manager.yaml and all relevant components, e.g. istio / monitoring:
control-plane: controller-manager => app.kubernetes.io/component: kyma-project.io/lifecycle-manager
This can include labels and/or selectors.
in kustomization:
commonLabels:
  app.kubernetes.io/instance: kcp-lifecycle-manager-main
  app.kubernetes.io/name: kcp-lifecycle-manager
  app.kubernetes.io/created-by: kustomize
  app.kubernetes.io/managed-by: kustomize
  app.kubernetes.io/part-of: manual-deployment

https://github.com/kyma-project/lifecycle-manager/pull/166 fixes lifecycle-manager related issues


Fix PianorollWindow

Delete unnecessary Grid and Border
Toolbars and menus placed in front of note properties

Commit 1a264269d141744f11e4d0c195956c4a89d5dc4d makes this PR unnecessary.


Hierarchy creation error: The Copper level matches with more than one cube level

Steps to reproduce
I tried to follow tutorial to regroup my attributes in the same hierarchy but I encountered the error:
AtotiJavaException: The Copper level Level[level=coin_symbol] matches with more than one cube level: [Level(Hierarchies, cryptocurrency, coin_symbol), Level(Hierarchies, coin_symbol, coin_symbol)]

thereafter the atoti cube stopped running:
AtotiJavaException: Pivot Twitter_Cyptocurrency is not running

I can re-run the command to create the cube again, thereafter, I'm able to regroup the hierarchy without error. Please see the below gif:

Please see attached notebook.
Twitter and Cryptocurrency.zip
Environment

atoti: 0.4.2
Python: 3.8.5
Operating System: win32

Logs (if relevant)

Aug 27, 2020 10:30:52 AM com.activeviam.health.monitor.impl.LoggingActivePivotHealthEventHandler onActivePivotTransactionCommitted
INFO: [activepivot, transaction] INFO 2020-08-27T02:30:52.057Z uptime=11272ms com.quartetfs.biz.pivot.transaction.impl.ActivePivotSchemaTransactionManager.commit:343 thread=Thread-1 thread_id=20 event_type=ActivePivotTransactionCommittedEvent user=anonymous roles=[ROLE_USER, ROLE_ADMIN] ActivePivot transaction 2 was successfully committed on epoch 3.
Aug 27, 2020 10:30:52 AM com.quartetfs.biz.pivot.impl.ActivePivotManager doStop
INFO: [ActivePivotManager]: Stopped
Aug 27, 2020 10:30:52 AM com.qfs.multiversion.impl.EpochManager$DiscardOnGcTask run
INFO: Thread com.qfs.multiversion.impl.EpochManager$DiscardOnGcTask@7af39feb is interrupted!
Aug 27, 2020 10:30:52 AM py4j.reflection.MethodInvoker invoke
WARNING: Exception occurred in client code.
java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
	at py4j.Gateway.invoke(Gateway.java:282)
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
	at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
	at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.activeviam.copper.exception.UserErrorException: The Copper level Level[level=coin_symbol] matches with more than one cube level: [Level(Hierarchies, cryptocurrency, coin_symbol), Level(Hierarchies, coin_symbol, coin_symbol)]
	at com.activeviam.copper.CubeHelper$LevelInformationRetriever.findLevelInformation(CubeHelper.java:364)
	at com.activeviam.copper.CubeHelper$LevelInformationRetriever.lambda$getLevelInformation$0(CubeHelper.java:271)
	at com.activeviam.copper.CubeHelper$LevelInformationRetriever.computeIfAbsent(CubeHelper.java:308)
	at com.activeviam.copper.CubeHelper$LevelInformationRetriever.getLevelInformation(CubeHelper.java:271)
	at com.activeviam.copper.CubeHelper.getLevelInformation(CubeHelper.java:131)
	at com.activeviam.copper.CubeHelper.getLevelIdentifier(CubeHelper.java:109)
	at com.activeviam.copper.store.Mapping.createMappingForProcedure(Mapping.java:89)
	at com.activeviam.copper.JoinPublisher.createKey(JoinPublisher.java:448)
	at com.activeviam.copper.JoinPublisher.publish(JoinPublisher.java:212)
	at com.activeviam.copper.HierarchyPublisher.publish(HierarchyPublisher.java:139)
	at com.activeviam.copper.impl.DataCubeContext.publish(DataCubeContext.java:182)
	at com.activeviam.copper.hierarchy.BaseCopperSingleLevelAnalysisHierarchy.publish(BaseCopperSingleLevelAnalysisHierarchy.java:148)
	at com.activeviam.copper.hierarchy.BaseCopperSingleLevelAnalysisHierarchy.publish(BaseCopperSingleLevelAnalysisHierarchy.java:37)
	at com.activeviam.chouket.pivot.impl.SingleLevelCopperHierarchy.doPublish(SingleLevelCopperHierarchy.java:149)
	at com.activeviam.chouket.pivot.alambyk.impl.ACopperDefinition.publish(ACopperDefinition.java:67)
	at com.activeviam.chouket.pivot.impl.HierarchiesManager.lambda$publishCopper$7(HierarchiesManager.java:243)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
	at com.activeviam.chouket.pivot.impl.HierarchiesManager.publishCopper(HierarchiesManager.java:243)
	at com.activeviam.chouket.postprocesssor.impl.UserCopperMeasuresDescriptionPostProcessor.accept(UserCopperMeasuresDescriptionPostProcessor.java:36)
	at com.activeviam.chouket.postprocesssor.impl.UserCopperMeasuresDescriptionPostProcessor.accept(UserCopperMeasuresDescriptionPostProcessor.java:20)
	at com.activeviam.copper.CopperDescriptionPostProcessor.enrichDescriptionWithCopperCalculations(CopperDescriptionPostProcessor.java:90)
	at com.activeviam.chouket.pivot.impl.CubeImpl$ChouketCopperActivePivotDescriptionPostProcessor.enrichDescriptionWithCopperCalculations(CubeImpl.java:1242)
	at com.activeviam.copper.CopperDescriptionPostProcessor.postProcessActivePivotDescription(CopperDescriptionPostProcessor.java:57)
	at com.quartetfs.biz.pivot.impl.ActivePivotSchemaBuilder.postProcess(ActivePivotSchemaBuilder.java:964)
	at com.quartetfs.biz.pivot.impl.ActivePivotManagerBuilder.postProcess(ActivePivotManagerBuilder.java:267)
	at com.activeviam.chouket.pivot.impl.CubeManagerImpl.toManagerDesc(CubeManagerImpl.java:194)
	at com.activeviam.chouket.impl.SimpleActiveViamSession.resume(SimpleActiveViamSession.java:416)
	at com.activeviam.chouket.impl.SimpleActiveViamSession.refresh(SimpleActiveViamSession.java:505)
	at com.activeviam.chouket.impl.SimpleActiveViamSession.refreshPivot(SimpleActiveViamSession.java:537)
	at com.activeviam.chouket.api.impl.ActiveViamSessionApiImpl.refreshPivot(ActiveViamSessionApiImpl.java:133)
	... 12 more
Aug 27, 2020 10:34:34 AM py4j.reflection.MethodInvoker invoke
WARNING: Exception occurred in client code.
java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.NullPointerException: Pivot Twitter_Cyptocurrency is not running
at com.activeviam.chouket.api.impl.ActiveViamSessionApiImpl.retrieveHierarchies(ActiveViamSessionApiImpl.java:505)
... 12 more

Thanks for reporting.
I can understand the initial error, since a copper join relies on existing levels to match the mapping keys, and in this case there are 2 levels that match the same key (coin_symbol). We need to see what can be done for this.
I'm more surprised that it seems to work the second time. It's also worth looking at.
I'm assigning this to @a-manai for further investigation.
As as side note, after regrouping the levels under the same hierarchy, do you expect / want the single level coin_symbol hierarchy to still exist?

By the way, you can slightly re-arrange your code to avoid this issue:
# In[1]:

import atoti as tt
import pandas as pd
from atoti.config import create_config

# In[2]:

config = create_config(metadata_db="./metadata.db", sampling_mode=tt.sampling.FULL)
session = tt.create_session(config=config)

# In[3]:

cyptocurrency_price_store = session.read_csv(
    "crypto_prices.csv", keys=["date", "coin_symbol"], store_name="cyptocurrency price"
)

# In[4]:

tweet_store = session.read_csv(
    "currency_tweets.csv",
    keys=["tweet_id", "coin_symbol", "date"],
    store_name="tweet_currency",
)
tweet_sentiment = session.read_csv(
    "tweets_sentiments.csv", keys=["tweet_id"], store_name="tweet_sentiment"
)

# In[5]:

cube = session.create_cube(cyptocurrency_price_store, "Twitter_Cyptocurrency")

# In[6]:

cube.schema

# In[7]:

m = cube.measures
l = cube.levels
h = cube.hierarchies

# In[8]:

cube

# In[9]:

del h["coin_symbol"]
del h["currency_name"]
h["cryptocurrency"] = [cyptocurrency_price_store["coin_symbol"], cyptocurrency_price_store["currency_name"]]

# In[10]:

del l["date"]
del l["time"]
h["timeline"] = [cyptocurrency_price_store["date"], cyptocurrency_price_store["time"]]

# In[11]:

# Move the join after re-arranging the hierarchies.
cyptocurrency_price_store.join(tweet_store)
tweet_store.join(tweet_sentiment)

# In[12]:

cube.schema

# In[13]:

cube

I realize that creating the cube second time, I lose the joins. Hence the levels can be created.
Even if I tried to create the joins, they do not appear in the schema.
As for the side note, in the case of the currency, it would be nice to keep the second level hierarchy (currency_name). It's fine to delete the single level coin_symbol hierarchy as it can be collapsed in pivot table (no impact visually).

Hi @rcolle,
your propose solution does creates the hierarchy but the resulting join is incorrect and measures value becomes a little unpredictable.
I have data at the hour 2:

Price which is derived from value.VALUE has a value but value.VALUE itself has none.

I created the join as follows
date_store.join(cyptocurrency_price_store, mapping={"coin_symbol":"coin_symbol", "date":"date", "hour":"hour"})

So I should be able to filter the date. However, it is pulling out all the dates when I tried to filter to 2020-08-21:

Please see the attached notebook in case I have implemented something differently:
del_hierarchy.zip

I'm baffled. If I do not create the multi-level hierarchy, the filtering mentioned in the previous comment is fine.
However, where I have 2 records under the hour 2, the value.VALUE which is automatically created during cube creation is still not showing any value:

I'm baffled. If I do not create the multi-level hierarchy, the filtering mentioned in the previous comment is fine.
However, where I have 2 records under the hour 2, the value.VALUE which is automatically created during cube creation is still not showing any value:

This is answered by https://github.com/atoti/atoti/issues/143#issuecomment-682395750

So ... you've hit quite a few hair raising points :-)
First of all, you've found yourself in single-value "hell".
The first observation has been answered in #143, explaining why there are no values above certain levels for single values.
The second observation (why is there a price value then?) comes from the fact that we "help" people when creating aggregations on top of a single value.
When you write m["price"] = tt.agg.mean(m["value.VALUE"], scope=tt.scope.origin(l["hour"])), we actually add date and coin_symbol to the scope to make sure that value.VALUE is always defined.
Both these points are debatable, and we're planning to work on the automated VALUE aggregation to avoid all these confusions. See https://github.com/activeviam/atoti/issues/1430.
Finally, the last point (filter on date not working) is caused by a core bug:
https://github.com/activeviam/activepivot/blob/37d3dbe264f5cd92b28472ec5d1c35863e88272a/pivot/core/impl/src/main/java/com/quartetfs/biz/pivot/cube/hierarchy/axis/impl/LocationBuilder.java#L123
I'll file a core ticket for it.

@a-manai , regardless of all the raised points, the initial issue still exists:
AtotiJavaException: The Copper level Level[level=coin_symbol] matches with more than one cube level: [Level(Hierarchies, cryptocurrency, coin_symbol), Level(Hierarchies, coin_symbol, coin_symbol)]

@HuifangYeo can you still reproduce the issue ? If not can you close this issue ?
I don't manage to reproduce in 0.7.0 with:
import atoti as tt
session = tt.Session()
cryptocurrency_table = session.create_table(
    "cryptocurrency price",
    types={
        "date": tt.LOCAL_DATE,
        "coin_symbol": tt.STRING,
        "datetime": tt.STRING,
        "currency_name": tt.STRING,
    },
    keys=["date", "coin_symbol"]
)
tweet_currency = session.create_table(
    "tweet_currency",
    types={
        "tweet_id": tt.LONG,
        "coin_symbol": tt.STRING,
        "date": tt.LOCAL_DATE,
        "created_at": tt.STRING,
    },
    keys=["date", "coin_symbol", "tweet_id"]
)
tweet_sentiment = session.create_table(
    "tweet_sentiment",
    types={
        "tweet_id": tt.LONG,
        "retweet_count": tt.INT,
    },
    keys=["tweet_id"]
)
cryptocurrency_table.join(tweet_currency)
tweet_currency.join(tweet_sentiment)
cube = session.create_cube(cryptocurrency_table)
h, l, m = cube.hierarchies, cube.levels, cube.measures

h["cryptocurrency"] = [l["coin_symbol"], l["currency_name"]]


Add all errors to one file

[Replace this text by the issue key](Replace this text by the issue link)
Description
Currently, all error classes are stored under the lib/inquisition/errors folder. Need create a error.rb file where will be stored all our errors
Replace this text with a summary of the changes in your merge request
Before submitting the merge request make sure the following are checked

[ ] Followed the style guidelines of this project
[ ] Performed a self-review of own code
[ ] Wrote the tests that prove that fix is effective/that feature works

@rostikkkk2 your branch has conflicts, can you resolve them please,  thanks


docs(custom-resources): update acl KongCredential

To use the correct singular rather than an incorrect plural.

What this PR does / why we need it: Fixes a small error in the documentation of how to add a Consumer to an ACL group which would result in the consumer not being able to be added to the group.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


N805 ignores classmethod-decorators setting

ruff.toml
select = ['N805']

[pep8-naming]
classmethod-decorators = [
    'classmethod',
    'declared_attr',
    'strict_classproperty',
    'expression',
    'comparator',
]

ruff_sample.py
from sqlalchemy.ext.hybrid import hybrid_property
from indico.util.decorators import strict_classproperty

class SQLAlchemyModel:
    @hybrid_property
    def foo(self):
        return not self._foo

    @foo.expression
    def foo(cls):
        return ~cls._foo

    @strict_classproperty
    def bar(cls):
        return cls._bar

Output
$ ruff check ruff_sample.py
ruff_sample.py:11:13: N805 First argument of a method should be named `self`
ruff_sample.py:15:13: N805 First argument of a method should be named `self`
Found 2 errors.

Versions
flake8-to-ruff==0.0.233
ruff==0.0.269

Ah, you need to provide fully-qualified paths, e.g., classmethod-decorators = ["sqlalchemy.ext.hybrid.hybrid_property"]. We match on the binding, not the symbol name, which ensures that (e.g.) you can import symbols under aliases and everything works as expected.

This works fine for my strict_classproperty decorator but I don't see how this would work for the hybrids. The method decorated with hybrid_property itself does not become a classmethod. The foo.expression one does. And that's not something I can specify as a fully-qualified name...

@charliermarsh ping :) I think this issue was closed too soon.
I just tested again using ruff==0.1.1 and it still seems to be impossible to handle those classproperties...
from sqlalchemy.ext.hybrid import hybrid_property

class SQLAlchemyModel:
    @hybrid_property
    def foo(self):
        return not self._foo

    @foo.expression
    def foo(cls):
        return ~cls._foo

Basically I need ruff to treat either @*.expression as a classmethod decorator, or alternatively have a way of telling it that sqlalchemy.ext.hybrid.hybrid_property returns a normal property, but the method decorated with that property has a .expression attribute that is a classmethod decorator...
Considering that this is a very common SQLAlchemy feature, and larger projects may have plenty of those, it's still a blocker for using ruff :/


Fix loader compatibilty with OpalRB

See: https://github.com/opal/opal/issues/1024.  Iterating, here, ends up with an exception, because current becomes an instance of OpalClass.
Possible solutions:

Using for .. of
Calling continue if data[type][asset].hasOwnProperty(current) is false

I initially verified with the latter solution, but someone suggested that this works better.

Calling Crafty.load with an array of assets no longer works; see the docs for more details.

@mucaho  I know. I'm passing in an object with arrays. That doesn't fix my issue.
Sample input:
{ images: [ '1.png', '2.png'], audio: ['1.ogg'] }

Embarrassingly, my fix doesn't work/compile in Chrome. I'll submit an update in a sec.

@mucaho I don't think that will work. Hash/objects don't have a length. Using of didn't work, either. I think this is the best solution I have right now. (I don't know why the CI build failed.)

Ah ok so the data passed to Crafty.load can look like this
{
    "audio": {
        "beep": ["beep.wav", "beep.mp3", "beep.ogg"],
        "boop": "boop.wav",
        "slash": "slash.wav"
    },
    "images": ["badguy.bmp", "goodguy.png"],
    "sprites": {
        "animals.png": {
            "tile": 50,
            "tileh": 40,
            "map": { "ladybug": [0,0], "lazycat": [0,1], "ferociousdog": [0,2] }
            "paddingX": 5,
            "paddingY": 5,
            "paddingAroundBorder": 10
        },
        "vehicles.png": {
            "tile": 150,
            "tileh": 75,
            "map": { "car": [0,0], "truck": [0,1] }
        }
    }
}

Crafty.load iterates over this data like this:
for (type in data) {
    for(asset in data[type]) {
       foo(data[type][asset]);
    }
}

On the 2nd iteration level Crafty does not distinguish between an object or an array being iterated on and that seems to give you trouble in the case the for ... in ... statement iterates over an array, right?
Your 2nd proposed solution seems reasonable (ES6 compatibility is lightyears away :) ).  My core change is this addition:
if ((typeof current) != 'string') {
  continue;
 }

Those lines go here.
I also only tested it in Chrome.
I'm not sure how to "fix" my PR. If you want to reject it, and submit the change yourself, that's cool with me. Or I can close it and try to create a new one.

Your commit seems to fail because current is allowed to be an array.
e.g.
data = {
    ...,
    "images": ["badguy.bmp", "goodguy.png"],
    ...
}

Calling continue if data[type][asset].hasOwnProperty(current) is false

This should do the trick. Note that there may be other places in the library that need a similar adjustment.

Your suggestion doesn't seem to work --  it skips over the strings from the images/audio arrays.
Any other ideas?

Last thing that pops to my mind is to check whether data[type] is an array or an object data[type].constructor === Array.

@mucaho I don't understand exactly the change that you're proposing.  I can, however, test it and make sure it works with at least the one case I have.
Right now, your PR doesn't fix my problem.

I stand corrected. Thanks for fixing this so fast.

I have to admit, I'm confused now. What is the actual fix so we can merge it? :)

@mucaho your PR that you submitted fixed the problem. I have some other problem where I ran something which didn't include your fix. My problem is solved, with your commit.
 But anyway, I did a manual clone + apply of your change, and I'm good to go now.

That PR was closed without being merged, so this actually isn't fixed yet.

Ok, I used some rebase magic, maybe that messed up things. Gonna go ahead and merge the relevant PR.