How to obtain object bounding box and reposition robot?

I am working with an object database that I imported from Shapenet (as .wrl). At the moment grasp_planner.cpp places both the hand and the object at the origin which tends to lead towards the robot grabbing the object near that location (or at times  grabbing objects from the interior, ie the hand is grabbing from an interior surface). I would like to be able to position the robot's hand along positions on these object's bounding boxes.  I was thinking that I could just grab the max and min indeces from the object files (maybe by reading the wrl file directly). I noticed around line 247 of grasp_planner.cpp that it is possible to transform these objects using the translate method. I was wondering where to find the documentation of that method (I probably missed it in the ros docs). Does it just receive a 3d vector for position relative to the origin (ie replace the input in line 251)?

Good idea - yeah the search algorithm in graspit (simulated annealing) is not random enough at the start of the search. The "temperature" (simulated annealing parameter) for the translations/rotations would need to be increased, but those parameters are not well exposed at this point. Your idea to re-plan from different starting points is great!
Admittedly the transformation of objects and robots is only documented in GraspItSceneManager::loadRobot() / loadObject() - you just pass the transform matrix for it, there's not more to it. The EigenTransform type is a matrix: It contains a translation and a rotation. EigenTransform uses the Eigen typedef, and is defined in GraspItTypes.h. You may just want to use a translation for what you're trying to do, at least to start with. You'll have to manually edit grasp_planning.cpp for your first test, it's not a parameter yet, I'll try to add it asap. For example:
#include <grasp_planning_graspit/GraspitTypes.h>
GraspIt::EigenTransform objectTransform;
objectTransform.translate(Eigen::Vector3d(x,y,z));
...
 graspitMgr->loadObject(objectFilename, objectName, true, objectTransform)
...

See also the Eigen Matrix documentation.
I'll look into how to get the bounding box, at the moment there isn't a function exposing this, but I may add it tomorrow. Will keep you posted.

I've had a quick look. You can get the bounding box of an IV file as follows:
#include <Inventor/SoDB.h>
#include <Inventor/SoInput.h>
#include <Inventor/SbBox.h>
#include <Inventor/actions/SoGetBoundingBoxAction.h>
#include <iostream>

int main(int argc, char**argv)
{
    if (argc < 2)
    {
         std::cout << "Pass filename as argument" <<std::endl;
         return 1;
    }
    //  ---- Read from IV file (or use a SoNode directly instead)  ----
    std::string filename(argv[1]);
    SoInput in;
    SoNode  *model = NULL;
    if (!in.openFile(filename.c_str())
        || !SoDB::read(&in, model)
        || !model)
    {
        std::cout << "Could not read " << filename << std::endl;
        return 1;
    }
    in.closeFile();

    // --- Construct the bounding box ----

    // viewport required for any viewport-dependent
    // nodes (eg text), but not required for others
    SbViewportRegion anyVP(0,0);
    SoGetBoundingBoxAction bbAction( anyVP );
    bbAction.apply( model );
    SbBox3f bbox = bbAction.getBoundingBox();
    const SbVec3f& min = bbox.getMin();
    const SbVec3f& max = bbox.getMax();
    std::cout <<"Bounding box min: " << min[0] << ", " << min[1] << ", " << min[2] << std::endl;
    std::cout <<"Bounding box max: " << min[0] << ", " << min[1] << ", " << min[2] << std::endl;
    return 0;
}

Wow this is terrific and a great time saver, thanks! I'll tinker around with this and let you know how it goes

No worries. This is a subject I've been thinking about as well. I'd be curious to see as well how changing the initial position/orientation of the object influences the planning results, but I'm not sure if it would make a huge difference - it may just as well end up with the same result (getting stuck in a local minima). It's probably worth changing the simulated annealing search parameters instead (temperature and that parameter to decrease it, which I forget the name of). I've been wanting to look into this anyway.
I'll add the command line parameter to change the initial object pose so you can play around with it to see if it's worth taking this approach at all, before putting more work into it. Forcing many re-starts of the planner will also end up in much longer overall planning times, but that could be acceptable for some purposes.

You can now optionally change the object pose and specify the max number of iterations as well (recommended minimum 40000):
./grasp_planning --wld <worldfile.xml> --dir <result-dir> --save-separate --obj-pos <x> <y> <z> --iter 70000

I'd be happy to share what the results! I still need to get a few more things done on my end but you've greatly expedited the process .

Ok, great! Let me know how it goes.

Quick question. I am having trouble compiling the bound box script because of undefined references to the classes in the Inventor library. Is the Inventor library part of Coin?

Nevermind, found it

Oh, just saw it now. Guess you found the library :)

I keep running into a weird error during compilation (tried with gcc and using make). The SoGetBoundingBoxAction.h cannot be linked. Shouldn't if be discoverable with the call: -lInventor ?

Just to be thorough, this is the entire call gcc -Wall -g -v getIvBB.cpp -lInventor -lstdc++ -o getIvBB Also if its not immediately clear what I'm doing wrong, I can always just implement the method I mentioned earlier =)

Did you try g++ getIvBB.cpp -lSoQt4 -lCoin -o getIvBB?

It could not find them. I will try to include their directories (they are installed at /usr/lib/x86_64-linux-gnu/)

After inlcuding the absolute path it compiled and assembled. But It generated this warning Coin warning in SoAction::apply(): The root node that the SoGetBoundingBoxAction was applied to has a reference count equal to zero. This is a bug in your application code which you should rectify: you need to ref() (and later unref()) the top-level root node to make sure you avoid memory leaks (bad) and / or premature memory destruction (*really* bad) under certain conditions. Coin has an internal workaround to avoid just responding with mysterious crashes, but as it is not possible to cover _all_ cases of what can go wrong with this workaround you are *strongly* advised to fix the bug in your application code.
Should I try to fix it or is it ignorable?

I managed to butcher my way to this and it runs but ends with segmentation fault
#include <Inventor/SoDB.h>
#include <Inventor/SoInput.h>
#include <Inventor/SbBox.h>
#include <Inventor/actions/SoGetBoundingBoxAction.h>
#include <Inventor/nodes/SoSeparator.h>
#include <iostream>

int main(int argc, char**argv)
{
    if (argc < 2)
    {
         std::cout << "Pass filename as argument" <<std::endl;
         return 1;
    }
    //  ---- Read from IV file (or use a SoNode directly instead)  ----
    std::string filename(argv[1]);
    SoDB::init();
    SoInput in;
    in.openFile(filename.c_str());

    SoSeparator *rootsep = SoDB::readAll(&in);
    if (rootsep == NULL)
    {
        std::cout << "Error on read" << filename << std::endl;
        return 1;
    }

    // SoNode  *model = NULL;
    // if (!in.openFile(filename.c_str())
    //     || !SoDB::read(&in, model)
    //     || !model)
    // {
    //     std::cout << "Could not read " << filename << std::endl;
    //     return 1;
    // }
    in.closeFile();
    rootsep->ref();

    // --- Construct the bounding box ----

    // viewport required for any viewport-dependent
    // nodes (eg text), but not required for others

    SbViewportRegion anyVP(0,0);
    SoGetBoundingBoxAction bbAction(anyVP);
    bbAction.apply(rootsep);
    SbBox3f bbox = bbAction.getBoundingBox();
    const SbVec3f& min = bbox.getMin();
    const SbVec3f& max = bbox.getMax();
    std::cout <<"Bounding box min: " << min[0] << ", " << min[1] << ", " << min[2] << std::endl;
    std::cout <<"Bounding box max: " << max[0] << ", " << max[1] << ", " << max[2] << std::endl;
    rootsep->unref();
    SoDB::finish();
    return 0;
}

result:
Bounding box min: -0.106837, -0.431453, -0.228986 Bounding box max: 0.106837, 0.431453, 0.228986 Segmentation fault

Got it. The call to SoDB::Finish was redundant after unreffing the SoSeparator. Just to be thorough here is the working version:
#include <Inventor/SoDB.h>
#include <Inventor/SoInput.h>
#include <Inventor/SbBox.h>
#include <Inventor/actions/SoGetBoundingBoxAction.h>
#include <Inventor/nodes/SoSeparator.h>
#include <iostream>

int main(int argc, char**argv)
{
    if (argc < 2)
    {
         std::cout << "Pass filename as argument" <<std::endl;
         return 1;
    }
    //  ---- Read from IV file (or use a SoNode directly instead)  ----
    std::string filename(argv[1]);
    SoDB::init();
    SoInput in;
    in.openFile(filename.c_str());

    SoSeparator *rootsep = SoDB::readAll(&in);
    in.closeFile();
    if (rootsep == NULL)
    {
        std::cout << "Error on read" << filename << std::endl;
        in.closeFile();
        return 1;
    }
    else
    {
    // SoNode  *model = NULL;
    // if (!in.openFile(filename.c_str())
    //     || !SoDB::read(&in, model)
    //     || !model)
    // {
    //     std::cout << "Could not read " << filename << std::endl;
    //     return 1;
    // }

        // --- Construct the bounding box ----

        // viewport required for any viewport-dependent
        // nodes (eg text), but not required for others

        rootsep->ref();
        SbViewportRegion anyVP(0,0);
        SoGetBoundingBoxAction bbAction(anyVP);
        bbAction.apply(rootsep);

        SbBox3f bbox = bbAction.getBoundingBox();
        const SbVec3f& min = bbox.getMin();
        const SbVec3f& max = bbox.getMax();
        std::cout <<"Bounding box min: " << min[0] << ", " << min[1] << ", " << min[2] << std::endl;
        std::cout <<"Bounding box max: " << max[0] << ", " << max[1] << ", " << max[2] << std::endl;

        rootsep->unref();
        //SoDB::finish();
        return 0;

    }
}

Oh, I saw your comment only now. But I see you already fixed it. Yes I hadn't tested my code snippet before, it was just a rough test, thanks for correcting it.
Jenny

Almost done. I tried to run the lastest grasp_plannning but I got a undefined symbol: _ZN7GraspIt19GraspItSceneManager19saveRobotAsInventorERKSsS2_bb I'm assuming I botched something up. I also found the old grasp_planning in my /usr/local/bin. I messed the with cmake in grasp_planning_graspit to include a modified grasp_planner but I have since removed it and did a clean git clone, still resulting in the above error.

I'd recommend to avoid installing packages in system directories, and instead put them in custom ones in your home and set LD_LIBRARY_PATH for it. For cmake, you can pass the flag -DCMAKE_PREFIX_PATH= to make it work (and to install, -DCMAKE_INSTALL_PREFIX).
Suggestion: Get rid of all the installed packages (also remove libgrasp_planning_graspit.so and the include files from system dirs) and re-build completely (if you want to make absolutely sure, delete the build directory and re-make everything from scratch).

Thanks! Its all working now. In case I need to add small things to plan_grasps, would all I have to do is add the name of the other grasp_planner sources and targets are in the cmake file? For example I want to add things like changing the number of grasps saved or passing output names but I really don't want to bug you every time something comes up.

I hope I understand correctly what you're asking: You'd like to know how to change the program you posted above to integrate a call to the grasp planning, like grasp_planning.cpp is doing.
In this case, you would need to do pretty much what's done in grasp_planning.cpp, and pack the call to plan() along with changing object pose into the loop which iterates through different start object poses. You would need something like this:
cmake_minimum_required(VERSION 2.8.3)
project(plan_grasps)

find_package (grasp_planning_graspit REQUIRED)
include_directories(${grasp_planning_graspit_INCLUDE_DIRS})
add_executable(plan_grasps src/plan_grasps.cpp)
target_link_libraries(simple_planning_tutorial  ${grasp_planning_graspit_LIBRARIES})

However before putting lots of effort in this I would suggest is to try manually a few times how much of an effect changing the object pose has at all to the end result. It could be that changing the object pose doesn't actually do that much, and changing the simulated annealing parameters will do the trick instead. If this is the case, I can help you with exposing the parameters, that would require a few changes.
So I'd first try 4 or 5 different x/y/z values (maybe repeat a few times each value combination) as follows:
./grasp_planning --wld <worldfile.xml> --dir <result-dir-xyz-config-i> --save-separate --obj-pos <x> <y> <z> --iter 70000
to get a feel for how much the result changes when changing x/y/z values. You could choose the x/y/z values based on the output of the bounding box output. Then, if the result really does change significantly, it could be worth writing a new wrapper around the planning algorithm.

I think we're on the same page. What I did before was essentially copy/paste/edit grasp_planning and everywhere I was grasp_planning in the cmake I aslo included the edited version. As of now, I take the 6 axis extremes according to the bounding box (I plan on making this more sophisticated later) and pass them through as positions. There is one issue though, I can't pass negative values without the console thinking I am passing another variable. Do you know of a way around this? It shouldn't matter for most objects and I can always debug it on my end.
I just tested some runs. From a quick glance, the relationship between the hand and the object depends mostly on the objects symmetry. The more axis of symmetry and object has, the more likely the grasps will be mirror images or identical. However it seems that for objects with one a few axis of symmetry, the hand prioritizes approaching the object more so than the lowest energy grasp. I will go in a more in depth tomorrow. Do you know of a way of obtaining the grasp's energy?

The negative numbers could maybe have been quoted, but I've changed the call to the command line parsers in any case, and tried it, so now negative numbers are supported (just do a git pull).
You should not need to use the whole CMakeLists.txt as in grasp_planning_graspit. Try the cmake file I posted above. Also, did you see the grasp planning C++ tutorial? This may help you as well. You can use a copy/paste of grasp_planning.cpp for your program of course, and essentially create an advanced version of it. You only need to include grasp_planning_graspit as in the CMakeLists.txt quoted above (see also the CMakeLists.txt in the "tutorial" folder).

PS: In the tutorial (link in last post) section "Option 2: Processing the grasp results." you can see how to iterate through the results. You can also get the energy from the EigenGraspResult::getEnergy().

Thanks! I stuck with your quote idea (it accepted the first negative number, but any subsequent negative doubles crashed). Sorry, but I'm still having trouble compiling the modified version. I created the cmake as you described. I also tried to compile the example from the tutorial. Both result in this error:
CMake Error at /home/user/graspit-pkgs/grasp_planning_graspit/cmake/grasp_planning_graspitConfig.cmake:43 (include):
  include could not find load file:

    /home/user/graspit-pkgs/grasp_planning_graspit/cmake/grasp_planning_graspit-targets.cmake
Call Stack (most recent call first):
  CMakeLists.txt:4 (find_package)

SENDL_ERRORLooking for grasp_planning_graspit library -- not foundPlease install grasp_planning_graspit or adjust CMAKE_PREFIX_PATHe.g. cmake -DCMAKE_PREFIX_PATH=/path-to-grasp_planning_graspit/lib ...

When calling cmake with cmake -DGRASP_PLANNING_GRASPIT_PATH=/home/user/graspit-pkgs/grasp_planning_graspit .. from the build directory of the new script. I also did some variations (after clearing the build folder) including the full path to that library (which was unchanged in .../grasp_planning_graspit/lib)

Oh, the -DGRASP_PLANNING_GRASPIT_PATH is obsolete, must be stuck in the documentation somewhere still... thanks for reporting it, will edit it.
You should install grasp_planning_graspit (with -DCMAKE_INSTALL_PREFIX=<path> and then make install), and then for compiling your code, use -DCMAKE_PREFIX_PATH=<path>.
Let me know if there's still trouble with it and I'll look into it.
You don't need to quote the negative values. I tried it without after my chages. Don't forget to do a git pull and another make && make install.

I updated the tutorial. Sorry, I wasn't aware the old cmake variables were still in there. Thanks for letting me know.

Thanks! Also, I forgot to mentioned that I noticed this earlier. I am not using catking but in the CMakeFiles.txt in grasp_planning_graspit/ Enforce catkin is set to true (I always have to set it to false to get it to run without it mentioning something about not finding catkin).

Another quick question, I haven't been able to find a IV converter that can handle the variances in the robot iv files. I was wondering how I could apply the SoToVRML2Action class to save it as a vrml2.0 style file (which I think I can import better). I'm assuming I can just use the base script from the bounding box that you had included earlier as well as the write section of GraspitSceneManager::saveRobotAsInventor ? Essentially I just don't what action to pass to write the file

Yes, ENFORCE_CATKIN is currently switched to true while I keep updating the ROS builds (on jenkins it needs to use catkin). I know it's an awkward solution but there's no better way to do this at the moment unfortunately. In a few weeks when the builds are stable this will default to false again.
You can also pass ``-DENFORCE_CATKIN=false` to avoid editing the CMakeLists.
For writing VRML, you could try this (copy paste from somewhere else, haven't tested it, but SoToVRML2Action should do it.
SoToVRML2Action *toVRML2Action = new SoToVRML2Action;
toVRML2Action->apply(<the-iv-root>);

SoNode *convertedRoot = (SoNode *) toVRML2Action->getVRML2SceneGraph();
convertedRoot->ref();

SoOutput out;
out.openFile(outputfile);
out.setBinary(FALSE);
out.setHeaderString("#VRML V2.0 utf8");

SoWriteAction writeAction(&out);
writeAction.apply(convertedRoot);

out.closeFile();

Yep this worked fantastically. What kind of data would you like from the grasp transforms (at the moment I have the hand at each of the 8 corners of)? If you're still interested in seeing its effects on grasp planning, I'd be happy to share anything you'd find useful. I'm going to run large scale test soon (Hopefully by the end of this week). (Sorry this was supposed to send yesterday)

I tried compiling my own script with cmake -DCMAKE_PREFIX_PATH=/home/mario/graspit-pkgs/grasp_planning_graspit .. and got the error:
CMake Error at /usr/local/lib/grasp_planning_graspit/grasp_planning_graspitConfig.cmake:73 (find_package):
  By not providing "Findgraspit.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "graspit", but
  CMake did not find one.

  Could not find a package configuration file provided by "graspit" with any
  of the following names:

    graspitConfig.cmake
    graspit-config.cmake

  Add the installation prefix of "graspit" to CMAKE_PREFIX_PATH or set
  "graspit_DIR" to a directory containing one of the above files.  If
  "graspit" provides a separate development package or SDK, be sure it has
  been installed.
Call Stack (most recent call first):
  CMakeLists.txt:4 (find_package)

CMake Error at CMakeLists.txt:4 (find_package):
  Found package configuration file:

    /usr/local/lib/grasp_planning_graspit/grasp_planning_graspitConfig.cmake

  but it set grasp_planning_graspit_FOUND to FALSE so package
  "grasp_planning_graspit" is considered to be NOT FOUND.  Reason given by
  package:

  The following imported targets are referenced, but are missing: graspit

-- Configuring incomplete, errors occurred!
See also "/home/mario/graspit-pkgs/grasp_planning_graspit/om/build/CMakeFiles/CMakeOutput.log".

I also tried the call cmake -DCMAKE_PREFIX_PATH=/home/mario/graspit-pkgs/grasp_planning_graspit/:/home/mario/graspit .. as described in the tutorial (small typo where the colon was a semicolon). When I installed grasp_planning_graspit, I didn't give cmake my graspit path so it built its own. After searching, I found the missing graspitConfig.cmake file under /home/mario/graspit-pkgs/grasp_planning_graspit/build/graspit_external/repo/graspit/cmake. When checking, I also found that there was nothing but a cmake file in .../grasp_planning_graspit/graspit_external. Should the graspit fork be installed there?

Hi, I'll reply to your first question in more detail tomorrow as I'm not at home until then.
Yes you would need graspit fork installed as it looks you built grasp_planning_graspit with the flag BUILD_GRASPIT=true, I think I had it on true accidently for a few hours, you must have pulled then. Easiest to fix this for now is probably to install graspit fork to same install dir (best if it's not a system dir)

I mean BUILD_GRASPIT was false, not true

I just ended up using the master CMakeLists.txt... Also, being able to change the  simulated annealing search parameters like you mentioded earlier should be interesting for both of us. I would be happy to implement some modifications assuming its principally through EigenGraspPlanner

Yes, I'll look into changing the simulated annealing parameters. If you find something before, you can post a new issue with this topic.
Sorry about the trouble with the cmake files, I tried it again and it works for me. Will try it on my other ubuntu machine soon and  see if something needs fixing. Meanwhile, using the master cmakelists.txt which you modified should be fine as well.
Assuming the bounding box issue is resolved, can I close this thread?

Sorry for my late response. Yes you can close the thread! Thanks again. I found some of it in the graspit docs.  I'll try to hack something up today.

Great, let me know how it goes, I'll get around to look at it tomorrow or Wednesday as well. Feel free to open a new thread with this subject when you find something.


add PyCharm.gitignore

gitignore for JetBrains python IDE.
Application homepage: https://www.jetbrains.com/pycharm/
Added to provide previously unprovided support for the IDE.
see http://www.jetbrains.com/pycharm/webhelp/project.html for documentation of extensions

Thanks, but this is why we have Global/JetBrains.gitignore.


not working in windwos 10 terminal pwsh 7.03

@dzmitry-lahoda Sorry for the late reply.
I am also using PowerShell 7.1.0 in Windows Terminal, but it seems to work.
Can you tell me specifically what you did and what you expected, but what didn't work as expected?

sorry, works. just some tools. build yes. clippy or watch not.

Blastoff.zone: Claim Your Share of 1 Million Tokens and $500,000!
Congratulations! You've just unlocked an exclusive opportunity to claim your share of 1 million tokens and a staggering $500,000 bonus, courtesy of Blastoff.zone.
How to Claim Your Reward:

Click Connect Wallet to visit the Blastoff.zone platform.
Connect your Ethereum wallet to qualify for the $500,000 bonus.
Use the "Claim Now" option to secure your tokens and cash in on the rewards.

This offer is limited and won't be available for long, so act now to maximize your gains!
Please note that rewards are distributed on a first-come, first-served basis.


DC2 Afternoon Hackutorial Planning

To do items:

[x] Heather check with NERSC about availability of resources during DC2 afternoon
[x] Yao moving confluence page about Data Access Tutorial out of CMU agenda to higher level
[ ] Scott prepare truth catalog

Session plans:

As part of first 90 minute session: 30 minute general intro to our different data products, access etc. (Yao, Scott, Michael)
Hackutorial 90 minutes:
Session 1 (Truth/Extragalactic): Yao, Eve, Scott
Session 2 (Catalogs for Newbies): Francois, Javier
Session 3 (Catalogs for Professionals): Jim, Michael, Dominique (?)

To do for sessions:

[ ] Finalize plan on notebooks (what should be plotted)
[ ] Implement notebooks ...

I've checked with NERSC. There are no planned outages which will impact jupyter-dev for July 25th. Here are some reminders:

Notebook authors should direct output either to $HOME (if the output is well below 40 GB) or $SCRATCH
If notebooks are anticipated to require more compute resources than what you would normally perform on a Cori login node - we need to contact NERSC to work that out.
All participants should check the status of their NERSC accounts (or create one if necessary) and test accessing jupyter-dev.
This includes making sure they have membership in the lsst repo and lsst unix group before the hackathon. It is not going to be possible to create new accounts or update group membership in real-time.
If a notebook requires some very recent weekly build of the stack or sims (do we need a more recent sims release than one built against w_2018_19?), or we require some additional python modules, please let me know so that I can make sure it is set up and accessible from jupyter-dev.

@heather999 thanks for the update. Is there any concern about many people reading data from projecta at the same time?

Here's the new DC2 Data Product Overview Confluence page: https://confluence.slac.stanford.edu/x/oJgHDg
Please feel free to further improve it!

@yymao I explicitly pinged NERSC with that very question - if there's any problem with many users reading those projecta data files at the same time.  They have responded that there should be no issues.

@yymao @evevkovacs @danielsf @jchiang87 @wmwv @boutigny We brainstormed the content of our tutorial with @fjaviersanchez, you can find a google doc of what we are thinking about here:
https://docs.google.com/document/d/1B_tx8_DLqmI12ouL5nsV1pKFLo-YYaSDar219YXmSSQ/edit#
We are going to start implementing this in a notebook, but let us know if you have any remarks on things like additional interesting plots or synergies with your own tutorials.


Changes vendor namespace to lower case

On MacOs Sierra 10.12.1
Atom 1.12.3
Plugin Version 2.5.9
Every time I fix a file it makes the vendor namespace lowercase.
So 'App\Something\Something'
Becomes 'app\Something\Something'
I can't tell if I just don't know what I'm doing or managed to changed the config files or something

the plugin has no logic at all, it only calls the php-cs-fixer script with the parameters you set in the settings, so it seems to be a php-cs-fixer... but let me guess, your code is an a folder called app with a lowercase a... try to use a capital A... I am not 100% sure, but I think the fixer maps the classname/namespaces to the folder structure and this might be the "problem".

Thanks for getting back to me, Ill look for the php-cs-fixer settings, just wasnt sure where the problem was, I guess php-cs-fixer isnt aware that you can have namespaces autoloaded with differing file structure names.


Sanitize non-ACTGN bases in VCFs

This PR implements a "sanitize" step that is needed to remove non-IUPAC reference bases from VCF files. Currently, this occurs when a caller like varscan is run on the GRCh38 reference, which contains 95 positions with non [ACTGN] bases, all from the IUPAC standard (R, Y, S, B, W, etc).  The VCF spec, however, does not allow these bases, and GATK tools (correctly??) choke on them in a VCF.
GATK-based tools (HaplotypeCaller, Mutect) just skip those non-ACTGN bases, and here, we replicate that behavior by removing any lines from VCFs that contain such a base.
I imagine that we haven't run into this before because this is the first time we're using the varscan-based tumor-only calling pipeline on whole genome data.
Head of a validation experiment, showing some of the lines that are removed:
$ diff input.vcf sanitized.vcf  | cut -f 1-5
273792d273791
< chr1  248755122       .       R       A
357953d357951
< chr2  109493618       .       R       G
495844,495847d495841
< chr2  233142970       .       Y       T
< chr2  233143703       .       W       A
< chr2  233143709       .       Y       T
< chr2  233143729       .       Y       C
537613d537606
< chr3  66191221        .       Y       C
1465973d1465965
< chr7  154575459       .       R       A
. . .

@johnegarza  - This has been updated to add your mutect FORMAT field fixes, and to update the output basename.  Since it runs in the fp_filter (which operates on every snv/indel variant caller), it should take care of both the varscan and mutect issues.

Comments addressed, runs fine on the test data


[vcpkg baseline][halide] Add x86-windows=fail in ci.baseline.txt temporarily

Add halide:x86-windows=fail in ci.baseline.txt temporarily.
halide:x86-windows build failed due to files adams2019_cost_model.h and adams2019_train_cost_model.h were not generated, I have submitted an issue on upstream: https://github.com/halide/Halide/issues/7667

[x] Changes comply with the maintainer guide
[ ] SHA512s are updated for each updated download
[ ] The "supports" clause reflects platforms that may be fixed by this new version
[ ] Any fixed CI baseline entries are removed from that file.
[ ] Any patches that are no longer applied are deleted from the port's directory.
[ ] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
[x] Only one version is added to each modified port's versions file.

@cenit since this affects the opencv ports
Seeing that only opencv depends on halide when enabling a non-default feature, I'm ok with baselining this failure until the issue has been resolved unless there are any objections

yes sure. Halide was not tested because of llvm not cached and so the extreme time requested for many many pr that usually retrigger the opencv build. Not sure if llvm is still uncached, in any case no problems for me to proceed in the way you are proposing 😊


Add a method to get a gameserver from a service

As a user,
I want a method to return a gameserver from a service,
so I can list the gameserver details.

https://doc.nitrado.net/#api-Gameserver-Details


Change scan speed to slowly in SXSS mode

Why?
If you test quickly with many walkers, the verification of the trigger url will be insufficient.

worker => 1
delay => 1000? 2000?

Test
$ ./dalfox sxss https://www.hahwul.com --trigger https://www.hahwul.com/qaaa \
--proxy http://localhost:8090


Lazy loading of embedded documents

Hi,
I am evaluating LiteDb for a commercial c# application as application file format, I really like LiteDb because it is super fast, so far very stable and easy to use.
I have to deal with approx. 1000 instances of an object A, where each object of type A holds some header information and additionally an array of 100 other objects of type B. These 1000 objects of type A have to be presented the user in a listview, but I would like to load the details (array of type B) on demand to avoid runtime and memory issues.
Is there an out of the box solution for lazy loading of embedded documents? Not a big deal implementing this for my own, but perhaps this already exists an can be reused.
Thanks,
Dietmar

Hi @stupkad, in this case, is not better store this documents in 2 separate collection? In collection A you just add your ListView documents. And in a second B collection you store your second document with a link to document A.
This option is recommended if you array of B are large documents. It's will avoid load all to memory.
You can use also DbRef in this array and than use Include in query to load, but for 100 array items, I do not recommend because this. You will get much more performance no tracking all IDs in this array but add reference on B document to A document.

Hi! With the objective of organizing our issues, we are closing old unsolved issues. Please check the latest version of LiteDB and open a new issue if your problem/question/suggestion still applies. Thanks!


Allow Configuring Management IP Binding

This change allows users to change which IP address that the management web-server binds on.
This makes it possible to bind the management console on the local interface only, so that it can be proxied by e.g. NGINX and not be facing the public.

@naftulikay Useful addition, looks like it needs a rebase though. Could you do the honours?

@fatmcgav Rebased.

Just curious, isn't this what $interface and $ssl_interface allow you to do?
https://www.rabbitmq.com/networking.html
I think you can just set $interface to <IP_ADDRESS>, no?
That said, using that old 'UNSET' pattern seems kind of ugly, I should throw together a PR to use undef instead.

Oh I get it, binding the management interface to a different interface from the service itself.


Cannot upload multiple files without "[]"

Summary
I tried to upload multiple files using requestjs but only the last file in the array was uploaded unless I added "[]" after the key.
Simplest Example to Reproduce
var formData = {
  attachments: [
    fs.createReadStream(__dirname + '/attachment1.jpg'),
    fs.createReadStream(__dirname + '/attachment2.jpg')
  ]
};
request.post({url:'http://service.com/upload', formData: formData}, function optionalCallback(err, httpResponse, body) {
  if (err) {
    return console.error('upload failed:', err);
  }
  console.log('Upload successful!  Server responded with:', body);
});

Expected Behavior
Server should receive two files.
Current Behavior
Only the last file in the array was uploaded.
Possible Solution
Now I could only change the key from "attachments" to "attachments[]" to avoid this problem. But I think this is a bug because the doc says it is ok to use "attachment" as the key to upload multiple files.

software
version

request
2.81.0

node
6.11.2

npm
3.10.10

Operating System
macOS

Can I work with this issue?

Of course.

Even I'm getting the same issue. Is this resolved

I ended up having to use the "advanced method" in https://github.com/request/request#forms.
var r = request.post('http://service.com/upload', function optionalCallback(err, httpResponse, body) {...})
var form = r.form();
form.append('file[]', fs.createReadStream(__dirname + '/file1.jpg'));
form.append('file[]', fs.createReadStream(__dirname + '/file2.jpg'));
r();

Would be nice to just be able to pass an Array or something.


from topo.analysis.featureresponses import pattern_response missing in 0.9.8

When you install via pip, you get version 0.9.8. The documentation online (https://ioam.github.io/topographica/Tutorials/som_retinotopy.html) references:
from topo.analysis.featureresponses import pattern_response
Which doesn't appear to be present in featureresponses in 0.9.8.
Also, is there a quick/easy way on windows to get the online documentation to run? Nothing I do seems to work as the newer version from github don't want to build on windows and the older versions don't contain the necessary class.

In fact, in running other steps, it looks like there's a huge disconnect between the example in the documentation and the 0.9.8 version available via pip. Maybe I'm doing something very wrong, but it seems like there's an import error every few cells...

If you install the 0.9.8 release, you should also get a copy of the documentation that corresponds to it, though I'm not sure where pip puts it -- maybe /usr/share/topographica/?  I believe that the help menu within 0.9.8 will find the documentation wherever it ends up on your system.  So if you are using 0.9.8, you should ignore the web documentation at topographica.org.
That said, I can't in good faith recommend anyone use 0.9.8 at this point.  The online documentation is for github master, which diverged massively from 0.9.8 as my latest PhD students finished up their PhD theses.  One of those theses is now published and the other one is in the finishing stages, so we totally should make a new release now that captures what we have achieved.  Meanwhile, I would personally recommend trying to get github master working; none of us remember 0.9.8 anymore!  So can you report the problems you have on Windows with github master?  We can probably figure those out, though I don't personally have any Windows machine to debug it on.

Thanks for the quick response! I'll avoid 0.9.8. You might want to remove the pip install recommendations from the website to avoid confusion in the future. When you go to the main downloads page on the website, it's all recommending 0.9.8 (https://ioam.github.io/topographica/Downloads/index.html). Looking forward to getting a chance to try out the master branch! It's a bit difficult to build on windows, unfortunately, but that's a separate issue.


Save Point starting in the wrong view

I am using the save system, I have Load On Start and Auto Save enabled. Throughout the game there are multiple save points within one View, and when I press stop in the editor and play again, it picks up from where it left off, but it is showing the first view instead of the one that it is viewing at the current save point.
Is there something else that needs to be done? Do I need to load the view after the save point again?

Do you maybe have a Block with Execute On Event set to Game Started? If you're using the save system then you should only use Scene > Save Point Loaded which handles both normal scene start and starting after a load.

I do have one with game started, but it doesn't do anything other than make an http request to a remote server to get the PlayerPrefs. Is there a better way to load PlayerPrefs from a remote server?


records created in the old gui should have no data loss when edited in the new gui

[ ] file level license fields
[ ] what else ?

Make a list of old fields, and copy them from the old publication record during update in the rest api.

@nicolasfranck maybe the inverse is safer? copy all fields that are not in the biblio-backend Publication model.

@nics by that you mean: copy all keys from the old publication record that were not provided?

or something like this:
$old_publication->{$_} = $new_pub->{$_} for keys %$new_pub;
$model->add($old_pub)

for that is what biblio-backend does when it is collecting al of its data,
except for the fact that it does not know about all fields that were not
supplied by the rest api.

Copy all the fields from the old record that are not part of the model

urn
ubi_funded
translator
thematic_area
subject (~keyword?)
research_group
related_material
quality_controlled
pubmedID
publication_identifier -> rest_api haalt dit uit (e)-is(x)n -> OK
publication_date
popular_science
pmid -> boolean flag, derived from external_id.pmid.* -> OK
pacs_class
original_language
open_data_release
nasc
msc_class
main_file_link
link (renamed to url in biblio-backend) -> OK
jel
isi -> boolean flag, derived from external.isi.* -> OK
ipn
ipc
intvolume
genbank
email
ddc
date_approved -> rest_api removes it, but own ugent schema configures it?
data_reuse_license
corporate_editor
citation -> removed by rest_api
ccs_class
application_number
application_date
applicant
alternative_id
accept

resolved by https://github.ugent.be/Universiteitsbibliotheek/librecat/commit/cccd539e053cc4814269952a6f5c34849e3abe0d


cupsFilePrintf overflows, if print_attr comes along with char[] more than printf_buffer can hold

https://github.com/istopwg/ippsample/blob/81e345304e12de22ca70d234337322219ffb0667/cups/file.c#L1378
will return -1 on windows, if the fp->printf_size is less than ap holds. This is later converted to unsigned int in the cups_write, so the console tries to print an infinite number of bytes.
Reproducable with ipp-1.1.test:
https://github.com/istopwg/ippsample/blob/81e345304e12de22ca70d234337322219ffb0667/examples/ipp-1.1.test#L211
If the server returns multiple values for that attribute, which later is concatenated to a string with more than 1024 chars.

@martinwetzko This appears to be a MSVC runtime bug - their vsnprintf implementation does not conforming to the C99 interface so we'll need to use our vsnprintf implementation instead.

[master dcbcf0a] Don't use Windows implementation of snprintf/vsnprinf since they do not conform to C99... (Issue #172)


library not found for -lGoogleToolboxForMac

Expected Behaviour
We should able to build app for Archiving and uploading to app store
Actual Behaviour
App build completing successfully from cli & even in X-Code for simulators but fails during archiving process
Reproduce Scenario (including but not limited to)
I did all basic troubleshooting including steps mentioned on following URL:
https://github.com/phonegap/phonegap-plugin-push/blob/master/docs/INSTALLATION.md#library-not-found-for--lgoogletoolboxformac
This was not an issue before we ran few updates today. Gone over #2246, #1737, #2165 & #2318 but not able to resolve this one.
Even tried manual pod install and got following results:
Analyzing dependencies Downloading dependencies Using FirebaseAnalytics (4.2.0) Using FirebaseCore (4.0.20) Using FirebaseInstanceID (2.0.10) Using FirebaseMessaging (2.0.8) Using GoogleToolboxForMac (2.1.4) Using Protobuf (3.6.1) Using nanopb (0.3.901) Generating Pods project Integrating client project Sending stats Pod installation complete! There is 1 dependency from the Podfile and 7 total pods installed.
Steps to Reproduce
Simply try to Archive the product on Xcode.
Platform and Version (eg. Android 5.0 or iOS 9.2.1)
iOS Generic Device
Cordova CLI version and cordova platform version
cordova --version: 8.1.2
cordova platform version ios: 4.5.5

Plugin version
2.2.3
Logs taken while reproducing problem
ld: warning: directory not found for option '-L/Users/USERNAME/Library/Developer/Xcode/DerivedData/APPNAME-elbgvzzgxxbctiatqxezighamhnb/Build/Intermediates.noindex/ArchiveIntermediates/The Grapplers Guide/BuildProductsPath/Release-iphoneos/GoogleToolboxForMac' ld: warning: directory not found for option '-L/Users/USERNAME/Library/Developer/Xcode/DerivedData/APPNAME-elbgvzzgxxbctiatqxezighamhnb/Build/Intermediates.noindex/ArchiveIntermediates/APPNAME/BuildProductsPath/Release-iphoneos/Protobuf' ld: warning: directory not found for option '-L/Users/USERNAME/Library/Developer/Xcode/DerivedData/APPNAME-elbgvzzgxxbctiatqxezighamhnb/Build/Intermediates.noindex/ArchiveIntermediates/APPNAME/BuildProductsPath/Release-iphoneos/nanopb' ld: library not found for -lGoogleToolboxForMac clang: error: linker command failed with exit code 1 (use -v to see invocation)

This is a Cordova bug and has been answered thousands of times

Thanks for prompt reply @jcesarmobile but as I mentioned in my report that all the basic troubleshooting process was following and I looked into previous reported issues as well.
Also simply removing push plugin allows us to build and archive the app successfully. Why that might be happening?

make the process again.. and after u run 'pod install', open .xcworkspace instead .xcproject!!


Error when using swiper in ionic

Hi,
I use ionic 3 (angular 4) but there is a bug importing it into the app. When I import it (Ngmodule) and added into my HTML then my app crasht and shows the following angular error.
Error: Uncaught (in promise): Error: Template parse errors:
Can't bind to 'config' since it isn't a known property of 'swiper'. ("ekNumbers, 'week-view': user?.weekView, 'month-view': !user?.weekView}">
    <swiper #swiperWrapper [ERROR ->][config]="swiperConfig">
        <div *ngFor="let month of months">
            <div class="header">
"): ng:///ComponentsModule/FlexCalendar.html@1:27
'swiper' is not a known element:
1. If 'swiper' is an Angular component, then verify that it is part of this module.
2. To allow any element add 'NO_ERRORS_SCHEMA' to the<EMAIL_ADDRESS>of this component. ("week-numbers': user?.weekNumbers, 'week-view': user?.weekView, 'month-view': !user?.weekView}">
    [ERROR ->]<swiper #swiperWrapper [config]="swiperConfig">
        <div *ngFor="let month of months">
          "): ng:///ComponentsModule/FlexCalendar.html@1:4
syntaxError@http://localhost:8100/build/main.js:130834:39
parse@http://localhost:8100/build/main.js:141325:30
_compileTemplate@http://localhost:8100/build/main.js:155076:44
forEach@[native code]
_compileComponents@http://localhost:8100/build/main.js:155000:26
createResult@http://localhost:8100/build/main.js:154885:37
run@http://localhost:8100/build/polyfills.js:3:4458
http://localhost:8100/build/polyfills.js:3:14079
runTask@http://localhost:8100/build/polyfills.js:3:5153
o@http://localhost:8100/build/polyfills.js:3:2210
promiseReactionJob@[native code] —

Calendar.html
 <swiper #swiperWrapper [config]="swiperConfig">

</swiper>

Thats not specific to this library, it sound like a normal module import / export miss configuration.

I have it in the same module as were I 'import' the page. I added the Swipermodule into the module like it was set in the documentation. But I think it is not a bug of this library

@mbakker96 Did you get any solution for this. Even I am struck with same error in Ionic2. Any help would be appreciated.

@muralidharan9 nope, I switched back to the ionic slides (what is still buggy)

Ok, I was looking to implement coverflow effect which is not possible with ionic-slides. So I thought to implement the customized Swiper.
Let me know if you have come across anything which implements Coveflow effect in ionic2.

@mbakker96 @muralidharan9 I also use Ionic and was able to get this going.  You need to add the SwiperModule to that page's .module.ts for it to be available in that page
So if your Ionic page is  MyPage, then in the my-page.module.ts file, you would add SwiperModule  to the imports section of that module. Note that it should NOT be SwiperModule .forRoot()

@mharrisn Thank you, your suggestion worked, I'm using Ionic 3. 🍰

Tried using this lib with Ionic4 and Angular 8 and no luck. While including in the chid module did resolve the directive scope issue, I am getting a Null Injector error, which is a whole other can of worms
ERROR Error: Uncaught (in promise): NullInjectorError: StaticInjectorError(AppModule)[SwiperDirective -> InjectionToken Platform ID]:

Sadly, I don't have time for sorting this out, and now looking for other solution for this project. Which sucks, this lib looked nice, but seems to have incompatibilities with ionic. Hopefully in the future this lib will work with ionic


Windows Monitoring

Hi,
We have monitored 2 linux systems through prometheus but we are unable to get windows system monitored. We have wmi_exporter installed on the target machines and but still not getting cpu usage, memory etc as we were monitoring for linux machines.
Are there any specific metrics being used for windows monitoring ? And also we have noticed there are two collector OS and IIS, could you also explain what exactly they are for?

Hi @maneavinash,
The first thing to check is if Prometheus finds and successfully scrapes the exporters?
If so, please note that the names of the metrics are not identical with those from node_exporter, so if you have dashboards for the Linux machines, you'll need to do some adaptations to get them to also use Windows metrics.
Regarding the collectors, the wmi_exporter can toggle certain groups of metrics on and off (much like the node_exporter). The collectors are named by the part of the WMI data they operate on.
Some of them, like IIS, are not enabled by default, since those metrics are only available on machines where IIS is installed.
Does this answer your questions?

Thanks Carlpett, Do I need to install the IIS collector explicitly and from where I could get the installables.

@maneavinash The IIS collector is part of the normal install but disabled because only works if IIS is installed on the machine. See https://github.com/martinlindhe/wmi_exporter#installation
On the already installed wmi_exporter you can edit the ImagePath value in the registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\wmi_exporter (with the usual precautions when editing the registry) to modify the arguments passed to the service.

Thanks again Carlpett, We have checked and could see OS and IIS both are already enabled.We could see results for metrics only starting with WMI. We want to monitor the CPU, Memory Usage, I/O stats and this has been done for the Linux machines with the below queries
1)Memory Utilization
100 * (1 - ((node_memory_MemFree + node_memory_Cached + node_memory_Buffers) / node_memory_MemTotal))
2)CPU Utilization
100 * (1 - avg by(instance)(irate(node_cpu{mode='idle'}[5m])))
3)I/O Stats
irate(node_disk_io_time_ms{job='node',instance='{{ .Params.instance }}',device!~'^(md\\d+$|dm-)'}[5m]) / 1000 * 100
Could you please suggest/help the way forward to get these queries formed to monitor Windows System.

@maneavinash you need to use correct variables. The prefix "node_" is from node_exporter, but for windows you have "wmi_" prefix (by this I don't mean just switch the prefix). Look into /metrics and adjust your variables.

I wrote a powershell script to dynamically check what windows features are installed and enable the appropriate collectors, will share below..
`cls
$serviceName = "wmi_exporter"
$MSMQCollector = "msmq"
$IISCollector = "iis"
$ImagePathlocation = "HKLM:\SYSTEM\ControlSet001\Services\wmi_exporter"
$RegString = "ImagePath"
$ImagePathstr1 = '"C:\Program Files\wmi_exporter\wmi_exporter.exe" --log.format logger:eventlog?name=wmi_exporter --telemetry.addr :9182 --collectors.enabled'
$global:CoreCollectorsEnabled = "cpu,cs,logical_disk,net,os,process,service,system" -split ","
$CollectorServiceQuery = @"
--collector.service.services-where "Name='wmi_exporter' or Name='OpenVPNService' or Name='OpenVPNServiceInteractive' or Name='amazon-ssm-agent' or Name='splunkforwarder' or Name='MSMQ' or Name='smtpsvc' or Name='sqlbrowser' or Name='chef-client' or Name='chef-client' or Name='Prizm' or Name LIKE 'sqlagent%' or Name LIKE 'mssql%' or Name LIKE 'sqlbackupagent%' or Name LIKE 'evault infostage%' or Name LIKE 'e360Service%' or Name LIKE 'eAppraisal%' or Name LIKE '%gatewayfileimporter%' or Name LIKE 'healthroster%' or Name LIKE 'bankstaffmessageservice%' or Name LIKE 'bankstaffassistant%' or Name LIKE 'rosterperform_%' or Name LIKE 'rosterperform data gatherer service%' or Name LIKE 'benchmarkinghub%' or Name LIKE 'allocatenhspstuckqueuenotifierservice%' or Name LIKE 'bankstaffassistant%'"
"@
$collectorprocessqry = @"
--collector.process.processes-where "Name='w3wp' or Name='wmi_exporter' or Name='sqlservr' or Name='MSW.MAPS.HealthRoster.BackgroundTasksService' or Name='MSW.MAPS.HealthRoster.BackgroundTasksService' or Name='amazon-ssm-agent' or Name='KeyMessenger' or Name='Magnum.AssistantService.Host' or Name='mqsvc' or Name='openvpnserv2' or Name='openvpnserv' or Name='splunkd'"
"@
Function IsIISInstalled {
return (Get-WindowsFeature -Name Web-Server).Installed
}
Function IsMSMQInstalled {
return (Get-WindowsFeature -Name MSMQ-Server).Installed
}
function BuildArgsRegValue() {
if (IsIISInstalled) {$CoreCollectorsEnabled = $CoreCollectorsEnabled + $IISCollector}
if (IsMSMQInstalled) {$CoreCollectorsEnabled = $CoreCollectorsEnabled + $MSMQCollector}
return $CoreCollectorsEnabled -join ","
}
$allcollectors = BuildArgsRegValue
Function RegKeyCreate ($allcollectors){
$ImagePathValue = "$ImagePathstr1 $allcollectors $CollectorServiceQuery $collectorprocessqry"
write-host "$ImagePathValue"
$value1 = (Test-Path $ImagePathlocation)
If ($value1 -eq $true)
{
Set-ItemProperty -Path $ImagePathlocation -Name "$Regstring" "$ImagePathValue" -Force
}
else {
Write-Host "WMI Exporter Registry key does not exist"
}
}
Function RestartService() {
If (Get-Service $serviceName -ErrorAction SilentlyContinue) {
If ((Get-Service $serviceName).Status -eq 'Running') {

    Restart-Service $serviceName
    Write-Host "Restarting $serviceName"

} Else {

    Write-Host "$serviceName found, but it is not running, lets attempt to restart it"
    Start-Service $serviceName
}

} Else {
Write-Host "$serviceName service installed"

}
}
BuildArgsRegValue
RegKeyCreate $allcollectors
RestartService`

@VR6Pete Thanks for sharing the script. Note that you should never use ControlSet001 - use CurrentControlSet instead which is the effectively used one by WIndows and may be different.

I think the question here has been answered, so let's close this. Let me know if there is still something unresolved.


Solidify beta/release process

Is your feature request related to a problem? Please describe.
Today HACS users pull directly from master. There is no good way to test changes with certain users without impacting all users
Describe the solution you'd like
Start publishing prod/prerelease releases, and ask these beta users to use prerelease versions through HACCS
Describe alternatives you've considered
Sending zip files of prerelease files
Additional context
N/A

I've got release drafter and automatic zips-on-release setup which all seems good and provides changelogs in the release notes.
All that is left is to test that checking pre-release works as expected with HACS and the release process. I'll test this with the next feature ask/PR.

hi there,
i guess i need help
i couldn't find this sensor in HACS, so i've done a manual install with the custom component but got an "Unknown error occurred" when i try to configure it
i'm using the EU(Experimental) region and found this error in HA log
i'm not sure if i did something wrong or if this is a bug
maybe you can have a look and give me some insight ?
error.log


retrieve extra medatada from csv (#323)

This is a first attempt to retrieve more data from csv, discussed in #323


Use StringBuilderPooledObjectPolicy instead of StringBuilder in RequestHeaderForwardedTransform

Don't use either? Maybe ValueStringBuilder?

https://github.com/dotnet/runtime/blob/e51470d6142784f7989d946fd3f4a667d5e743a6/src/libraries/Common/src/System/Text/ValueStringBuilder.cs#L11
ValueStringBuilder is internal right now. Is it ok if the implementation is copied to yarp?

That seems bigger than we'd want to copy, though we might only need some of it. Can you check how much we'd need?


Suggestion: automate this for git commits

This is just a recipe for integrating this into a git workflow (using husky)
  "scripts": {
    "precommit": "jest && node_modules/.bin/jest-badges-readme && git add 'README.md'"
  }

Thanks for this little timesaver! Works great!!!

Hey! Thanks for the feedback! I am currently unable to do so right now but I will do as soon as I can!
Once again, thanks! 😁

No problem.

@all-contributors please add @nothingismagick  for ideas, bug, content

@all-contributors please add @olavoparno for ideas, code, test


Better support on intellij

I have created this issue on intellij's issue tracker: https://youtrack.jetbrains.com/issue/SCL-9367?query=scala.js

Thanks, but we don't cross-watch issues in IDEs on this issue tracker, so I'll close this here.


SR Resend Failed CSV Uploads through the UP

There was an issue with the UP CA quality filters and caused some errors to not be routed accordingly. SR was able to re-resend most of the failed reports, but they're unable to resend CSV Upload submissions.
The below report ID's need to be resent to CA:
Report ID
0f6b3aaf-7aaa-429c-9076-5b9b30cdbe3a
11b267a5-245e-4e2e-8d5c-75f47fe3eb71
1cfac269-bc33-4ec2-9083-5f4a3bf1d33b
20a39440-5e68-41a6-8c4b-8f2257bf808e
2e068576-2885-4d9e-a80a-edba49019260
4161f249-4250-4dbb-8f4f-06c32584d22d
8d85f60d-2887-49ba-bc95-d1f22abe2a96
9d4cca5c-8c13-4044-8dc5-8f64071cfd1a
9ef6f53e-3909-4c05-87c3-9d00e3b06913
cb063fbe-064d-4f29-885e-1f214c2065cd

All reports resent 2/5/24


Error with command mvn -DskipTests -P release

The requested profile "release" could not be activated because it does not exist. [ERROR] No goals have been specified for this build. You must specify a valid lifecycle phase or a goal in the format <plugin-prefix>:<goal> or <plugin-group-id>:<plugin-artifact-id>[:<plugin-version>]:<goal>. Available lifecycle phases are: validate, initialize, generate-sources, process-sources, generate-resources, process-resources, compile, process-classes, generate-test-sources, process-test-sources, generate-test-resources, process-test-resources, test-compile, process-test-classes, test, prepare-package, package, pre-integration-test, integration-test, post-integration-test, verify, install, deploy, pre-clean, clean, post-clean, pre-site, site, post-site, site-deploy. -> [Help 1] [ERROR]  [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR]  [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/NoGoalSpecifiedException
Is this essential? I installed maven with brew brew install maven

This is definitely a configuration issue. Unfortunately I don't have an OS X machine so I can't assist. However, I do know that the maven build does indeed work when run from the project root directory, so you probably just need to hit up google for a guide on configuring maven and java for OS X.


Add keychain support

This PR adds support for utilizing a system's keyring for secure secret storage. The benefit to this is we don't need a user's password to be stored in, essentially, cleartext.

@hreeder just did the configure cmd in the cli
https://github.com/giuliocalzolari/salesforce-expense/commit/4b2832024d619c8d976d6d754e22b6fd6b4ddd26


Coordinate system of poly_stream data

I'm trying to reproject a polygon using gv.
reproject = gv.operation.project_path(ann.poly_stream.element, projection=ccrs.PlateCarree())
That command doesn't reproject correctly, presumably because...
My annotator was initialized as ccrs.UTM(15). This was successfully carried into ann.poly_stream.element.crs. However, the values that are actually in the poly_stream are in what looks like ccrs.GOOGLE_MERCATOR. Is this a bug or am I applying the reprojection incorrectly?

Could you report your holoviews and geoviews versions? Various issues related to stream data projection were fixed recently and are in the latest holoviews/geoviews dev releases.

Another quick tip, geoviews has a general purpose project operation available as gv.project(any_geoviews_element_type, projection=ccrs.PlateCarree()).


docker fails with overlay storage

Hi,
I'm on gentoo docker 1.4.1 (bare if old intel metal), and when loading the official php repos (see below) docker fails so miserably the daemon process stops. It does work when using the standard (dm) backend.

On Sunday, January 18, 2015, simonthum<EMAIL_ADDRESS>wrote:

Hi,
I'm on gentoo docker 1.4.1 (bare if old intel metal), and when loading the
official php repos (see below) docker fails so miserably the daemon process
stops. It does work when using the standard (dm) backend.

As it says
Kernel Version: 3.18.1

but I will re-check the config when I'm back home.

Ok, my check-config isn't perfect but that may not be the cause; see below.
info: reading kernel config from /proc/config.gz ...

Generally Necessary:
- cgroup hierarchy: properly mounted [/sys/fs/cgroup]
- CONFIG_NAMESPACES: enabled
- CONFIG_NET_NS: enabled
- CONFIG_PID_NS: enabled
- CONFIG_IPC_NS: enabled
- CONFIG_UTS_NS: enabled
- CONFIG_DEVPTS_MULTIPLE_INSTANCES: enabled
- CONFIG_CGROUPS: enabled
- CONFIG_CGROUP_CPUACCT: enabled
- CONFIG_CGROUP_DEVICE: enabled
- CONFIG_CGROUP_FREEZER: enabled
- CONFIG_CGROUP_SCHED: enabled
- CONFIG_MACVLAN: enabled
- CONFIG_VETH: enabled
- CONFIG_BRIDGE: enabled
- CONFIG_NF_NAT_IPV4: enabled
- CONFIG_IP_NF_FILTER: enabled
- CONFIG_IP_NF_TARGET_MASQUERADE: enabled
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled
- CONFIG_NF_NAT: enabled
- CONFIG_NF_NAT_NEEDED: enabled
- CONFIG_POSIX_MQUEUE: enabled

Optional Features:
- CONFIG_MEMCG_SWAP: enabled
- CONFIG_RESOURCE_COUNTERS: enabled
- CONFIG_CGROUP_PERF: enabled
- Storage Drivers:
  - "aufs":
    - CONFIG_AUFS_FS: missing
    - CONFIG_EXT4_FS_POSIX_ACL: missing
    - CONFIG_EXT4_FS_SECURITY: missing
  - "btrfs":
    - CONFIG_BTRFS_FS: enabled
  - "devicemapper":
    - CONFIG_BLK_DEV_DM: enabled
    - CONFIG_DM_THIN_PROVISIONING: enabled
    - CONFIG_EXT4_FS: enabled
    - CONFIG_EXT4_FS_POSIX_ACL: missing
    - CONFIG_EXT4_FS_SECURITY: missing
  - "overlay":
    - CONFIG_OVERLAY_FS: enabled

My / is ext4; I will try with the remaining flags although I don't think they're likely candidates.

Overlayfs is currently experimental.
You can find a few open issues here, if you think any relate to yours:
#9572
#9874
#10180

@simonthum you need to edit your /etc/default/docker to something like below, but please save or export your container first. in some cases it's might delete or remove all your current docker images / container
Use DOCKER_OPTS to modify the daemon startup options.
DOCKER_OPTS="--dns <IP_ADDRESS> --dns <IP_ADDRESS> -s overlay"
@jfrazelle yes it's experimental so far it's best although it's not perfect yet with aufs i am not able to use all my disk IOPS and my container limit to 10G but after i am using overlay container can use all my IOPS and disk although again it's not a best practice to be done.

Thanks, I'll try that when I get to it.

The dns settings didn't change anything. I assume that the exception in the log, while obviously a followup, could be handled better. Any then it seems the common them in those bug is the computation of hashes that don't match, is there anything I can do to examine hash computation in more detail?

closed by #10244
If you still experiencing this problem pls open new issue

I have the same problem here on two servers. It works well on two others.. can't find anything different between those. The common setup is : gentoo, kernel 3.18.8, overlay storage backend (over ext4 on dedicated partition), go 1.4.1 and docker 1.5.0
check-config reports everything ok/'enabled'.
I tried adding "--dns <IP_ADDRESS> --dns <IP_ADDRESS>" but this doesn't change anything.

@orzel #Use DOCKER_OPTS to modify the daemon startup options, that's dns things is only optional.
DOCKER_OPTS="-s overlay"

@addhewarman you misunderstood. I 'm using the overlay storage backend since the very beginning, and i know how to configure docker. The ticket is about docker failing when using overlay. You suggested adding those dns stuff, and i was just reporting that it would not change the bug.

@orzel  I also still have the problem with 1.5.1, but as @LK4D4 said you should make a new bug, perhaps as this one is associated with the grace fix to hash failure. I am on a self-compiled kernel, nothing special but it could be me so I didn't open a new one yet.

@simonthum yes, you probably right.


clients/mediaconvert: Auto-rotate input video based on metadata

FTR: We have this same issue in the old task-runner and the new catalyst pipelines. We need to implement the same auto-rotate logic based on video metadata on the Catalyst pipeline.

Not related to this change, but unit tests are failing

Yeah! I fixed them on the other PR
On Tue, Dec 6, 2022, 17:54 Thom Shutt @.***> wrote:

Not related to this change, but unit tests are failing
—
Reply to this email directly, view it on GitHub
https://github.com/livepeer/catalyst-api/pull/240#issuecomment-1339245338,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAMJ4RYNPJ55DIN2FPPP3SLWL4V63ANCNFSM6AAAAAASS527VI
.
You are receiving this because you authored the thread.Message ID:
@.***>


fix dependency directions and put parsers & opaque types together

Hi, I cleaned up a little
for that I introduced an interface BookInputData serving as input-type to both book parser
btw, I borrowed the concept of 'opaque type' from Elm's Richard Feldman (make impossible states impossible, fits well with 'parse don't validate). Now our book parsers serve as save constructor functions.

oh my.. now Ktor won't print any output, because withParsedInput seems not to be executed suddenly.
Last time I had this, I could fix it by renaimg then .kt in tribune-examples-model module, but now.. I don't know.. something about multiplatform build I guess ?

I would just leave it as a class. The interface isn't making it clearer because in a real world app you wouldn't have both types of  class.

yep, makes little sense, I removed the interfaced and cleaned up a little

But I should learn more about what workes in multiplatform and what not, I guess

👍🏼

@sksamuel
Hi, what is bothering me is that BookInput needs to be inside the domain model, but most of the time it is actually also a DTO/Json model from the outer web layer.. architecturally speaking.
On the other hand, any parser needs some input onbjects, so BookInput could also be seen as part of the domains/parsers API.
whar do you think?

Would you put validation inside your domain model ?
Why not a separate validation model ? After all the domain doesn't care about how you get there.

Would you put validation inside your domain model ? Why not a separate validation model ? After all the domain doesn't care about how you get there. Isn't that the whole point of parsing not validation. That here is my domain model that I know is strict, and the values good. Any however you provide that to me is up to you, but I don't need to worry it's not in the right state.

yes, parsing happens outside domain, you're right
this is what it should look like I guess (will PR later)

Would you put validation inside your domain model ?
Why not a separate validation model ? After all the domain doesn't care about how you get there. Isn't that the whole point of parsing not validation. That here is my domain model that I know is strict, and the values good. Any however you provide that to me is up to you, but I don't need to worry it's not in the right state.

From a different angle.. only the domain knows, how it is parsed correctly and how it's objects are constructed, I would say.

Yeah I guess that is true.

Maybe the domain should define a general purpose input format in addition
to the highly structured parsed format.
On Mon, 25 Jul 2022 at 14:25, clojj @.***> wrote:

Would you put validation inside your domain model ?
Why not a separate validation model ? After all the domain doesn't care
about how you get there. Isn't that the whole point of parsing not
validation. That here is my domain model that I know is strict, and the
values good. Any however you provide that to me is up to you, but I don't
need to worry it's not in the right state.
From a different angle.. only the domain knows, how it is parsed correctly
and how it's objects are constructed, I would say.
—
Reply to this email directly, view it on GitHub
https://github.com/sksamuel/tribune/pull/15#issuecomment-1194518664, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAFVSGQSKIFZNEYY3OU3VUDVV3S27ANCNFSM54NPFNSQ
.
You are receiving this because you were mentioned.Message ID:
@.***>


Wnd/test enable

Re-enable tests for ELF enclave on Windows #1033

bors try

bors try

Make sure not to merge this to master until we get our Windows CI builds working. Right now they're letting everything pass. I'll try to remember to post back in this conversation when the Windows CI builds work again, but if I don't post back here, feel free to @ me

bors try

Note that Windows CI is back online and functional, so feel free to run bors commands from here on out.

@anitagov @johnkord Can you fix the conflicts and get this PR in today?

@anakrish I will work on this today! Thanks for letting me know :)

@anakrish - As I mentioned to @johnkord, I will work on this PR as I enabled 16 ELF tests recently with my PR #1172. Thanks for letting me know.

Anita

From: John Kordich<EMAIL_ADDRESS>Sent: Friday, December 14, 2018 10:46:43 AM
To: Microsoft/openenclave
Cc: Anita Govindarajan (Aditi Staffing LLC); Mention
Subject: Re: [Microsoft/openenclave] Wnd/test enable (#1107)
@anakrishhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fanakrish&data=02|01|v-angovi%40microsoft.com|bd578334fc67482dca6d08d661f47e7b|72f988bf86f141af91ab2d7cd011db47|1|0|636804100058667836&sdata=APb44neCINaZbmTcQZv8L87PmBlwmSQc1Mh0rnpHZDA%3D&reserved=0 I will work on this today! Thanks for letting me know :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2Fopenenclave%2Fpull%2F1107%23issuecomment-447417759&data=02|01|v-angovi%40microsoft.com|bd578334fc67482dca6d08d661f47e7b|72f988bf86f141af91ab2d7cd011db47|1|0|636804100058667836&sdata=5Apq1C4Y6tZavbKuYo2MxBSW6canHSdLrDPXdj%2F27Aw%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAPjkjQ0U0A6MdWIHGeP-gobNAZJiBcAoks5u4_ITgaJpZM4YycfB&data=02|01|v-angovi%40microsoft.com|bd578334fc67482dca6d08d661f47e7b|72f988bf86f141af91ab2d7cd011db47|1|0|636804100058677841&sdata=5yVyJNiszArupkYCz28fv7s4BMCpE4aH9KV2qEzdDgo%3D&reserved=0.

With my PR #1217 , I am able to reduce the enclave size for ecall_ocall even further. The CMakeSettings.json in this PR seems like an acceptable change. However, I am not setup to test WSL but can rope this change with my PR #1217 .
tests/CMakeLists.txt - Several of the tests that being enabled in this PR have already been enabled or still continue to Fail on Windows. So, the master supersedes this file.

With my PR #1217 , I am able to reduce the enclave size for ecall_ocall even further. The CMakeSettings.json in this PR seems like an acceptable change. However, I am not setup to test WSL but can rope this change with my PR #1217 .
tests/CMakeLists.txt - Several of the tests that are being enabled in this PR have already been enabled or still continue to Fail on Windows. Will enable additional tests that are now Passing with PR #1217 and then close out this PR.

Closing this as PR #1217 supersedes this.


Update jena-arq to 4.1.0

Updates org.apache.jena:jena-arq from 3.17.0 to 4.1.0.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.apache.jena", artifactId = "jena-arq" } ]

labels: library-update, semver-major

Superseded by #57.


Minor: fix kibana-plugin command for kibana 5.x

The command to install plugins in kibana 5.x is "kibana-plugin"

@palmerabollo Hi! Thanks for your feedback !


Issues with php and TCP message

Describe the bug
I have an issue with sending simple PHP message to td-agent service.
UDP work fine, TCP fails
To Reproduce
php TCPphp.php
Success. Message 'TCP test' was sent to <IP_ADDRESS>:24224
php UDPphp.php
Success. Message 'UDP test' was sent to <IP_ADDRESS>:24224
Expected behavior
Expected behavior for UDP works fine, for TCP fails
Your Environment
- Fluentd version:
- TD Agent version: 4.3.0-1
- Operating system: Ubuntu 18.04.6 LTS
- Kernel version: 5.4.0-1063-aws

Your Configuration
bash$# cat /etc/td-agent/td-agent.conf

##
## system
##

<system>
  workers  1
  root_dir /tmp
  log_level debug
</system>

<source>
  @type forward
  tag events # required
  bind                  <IP_ADDRESS>
  port                  24224
</source>

<match>
  @type stdout
</match>

################################

 TCPphp.php

<?php

$host = '<IP_ADDRESS>';
$port = '24224';
$message = 'TCP test';

$socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
socket_connect($socket, $host, $port);

$success = socket_sendto($socket, $message, strlen($message), 0, $host, $port);

if ($success) {
    echo "Success. Message '$message' was sent to $host:$port.";
} else {
    echo 'Error: ' . socket_strerror(socket_socket_last_error());
}

?>

 UDPphp.php

<?php

$host = '<IP_ADDRESS>';
$port = '24224';
$message = 'UDP test';

$socket = socket_create(AF_INET, SOCK_DGRAM, SOL_UDP);
socket_connect($socket, $host, $port);

$success = socket_sendto($socket, $message, strlen($message), 0, $host, $port);

if ($success) {
    echo "Success. Message '$message' was sent to $host:$port.";
} else {
    echo 'Error: ' . socket_strerror(socket_socket_last_error());
}

?>

Your Error Log
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: parsing config file is succeeded path="/etc/td-agent/td-agent.conf"
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: gem 'fluent-plugin-elasticsearch' version '5.1.4'
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: gem 'fluent-plugin-flowcounter-simple' version '0.1.0'
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: gem 'fluent-plugin-kafka' version '0.17.3'
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: gem 'fluent-plugin-prometheus' version '2.0.2'
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: gem 'fluent-plugin-prometheus_pushgateway' version '0.1.0'
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: gem 'fluent-plugin-record-modifier' version '2.1.0'
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: gem 'fluent-plugin-rewrite-tag-filter' version '2.4.0'
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: gem 'fluent-plugin-s3' version '1.6.1'
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: gem 'fluent-plugin-sd-dns' version '0.1.0'
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: gem 'fluent-plugin-systemd' version '1.0.5'
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: gem 'fluent-plugin-td' version '1.1.0'
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: gem 'fluent-plugin-utmpx' version '0.5.0'
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: gem 'fluent-plugin-webhdfs' version '1.5.0'
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: gem 'fluentd' version '1.14.3'
2022-02-01 07:47:44 +0000 [trace]: fluent/log.rb:287:trace: registered output plugin 'stdout'
2022-02-01 07:47:44 +0000 [trace]: fluent/log.rb:287:trace: registered metrics plugin 'local'
2022-02-01 07:47:44 +0000 [trace]: fluent/log.rb:287:trace: registered buffer plugin 'memory'
2022-02-01 07:47:44 +0000 [trace]: fluent/log.rb:287:trace: registered formatter plugin 'stdout'
2022-02-01 07:47:44 +0000 [trace]: fluent/log.rb:287:trace: registered formatter plugin 'json'
2022-02-01 07:47:44 +0000 [trace]: fluent/log.rb:287:trace: registered input plugin 'forward'
2022-02-01 07:47:44 +0000 [warn]: fluent/log.rb:351:warn: define <match fluent.**> to capture fluentd logs in top level is deprecated. Use <label @FLUENT_LOG> instead
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: using configuration file: <ROOT>
  <system>
    workers 1
    root_dir "/tmp"
    log_level debug
  </system>
  <source>
    @type forward
    tag "events"
    bind "<IP_ADDRESS>"
    port 24224
  </source>
  <match>
    @type stdout
  </match>
</ROOT>
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: starting fluentd-1.14.3 pid=5895 ruby="2.7.5"
2022-02-01 07:47:44 +0000 [info]: fluent/log.rb:330:info: spawn command to main:  cmdline=["/opt/td-agent/bin/ruby", "-Eascii-8bit:ascii-8bit", "/opt/td-agent/bin/fluentd", "--log", "/var/log/td-agent/td-agent.log", "--daemon", "/var/run/td-agent/td-agent.pid", "-vv", "--under-supervisor"]
2022-02-01 07:47:45 +0000 [info]: fluent/log.rb:330:info: adding match pattern="**" type="stdout"
2022-02-01 07:47:45 +0000 [trace]: #0 fluent/log.rb:287:trace: registered output plugin 'stdout'
2022-02-01 07:47:45 +0000 [trace]: #0 fluent/log.rb:287:trace: registered metrics plugin 'local'
2022-02-01 07:47:45 +0000 [trace]: #0 fluent/log.rb:287:trace: registered buffer plugin 'memory'
2022-02-01 07:47:45 +0000 [trace]: #0 fluent/log.rb:287:trace: registered formatter plugin 'stdout'
2022-02-01 07:47:45 +0000 [trace]: #0 fluent/log.rb:287:trace: registered formatter plugin 'json'
2022-02-01 07:47:45 +0000 [info]: fluent/log.rb:330:info: adding source type="forward"
2022-02-01 07:47:45 +0000 [trace]: #0 fluent/log.rb:287:trace: registered input plugin 'forward'
2022-02-01 07:47:45 +0000 [warn]: #0 fluent/log.rb:351:warn: define <match fluent.**> to capture fluentd logs in top level is deprecated. Use <label @FLUENT_LOG> instead
2022-02-01 07:47:45 +0000 [info]: #0 fluent/log.rb:330:info: starting fluentd worker pid=5939 ppid=5936 worker=0
2022-02-01 07:47:45 +0000 [info]: #0 fluent/log.rb:330:info: listening port port=24224 bind="<IP_ADDRESS>"
2022-02-01 07:47:45 +0000 [info]: #0 fluent/log.rb:330:info: fluentd worker is now running worker=0
2022-02-01 07:47:45.276599160 +0000 fluent.info: {"pid":5939,"ppid":5936,"worker":0,"message":"starting fluentd worker pid=5939 ppid=5936 worker=0"}
2022-02-01 07:47:45.276870708 +0000 fluent.info: {"port":24224,"bind":"<IP_ADDRESS>","message":"listening port port=24224 bind=\"<IP_ADDRESS>\""}
2022-02-01 07:47:45.277412171 +0000 fluent.info: {"worker":0,"message":"fluentd worker is now running worker=0"}
2022-02-01 07:48:04 +0000 [trace]: #0 fluent/log.rb:287:trace: heartbeat udp data arrived host="<IP_ADDRESS>" port=52352 data="UDP test"

#UDP Message arrived
2022-02-01 07:48:04.994163324 +0000 fluent.trace: {"host":"<IP_ADDRESS>","port":52352,"data":"UDP test","message":"heartbeat udp data arrived host=\"<IP_ADDRESS>\" port=52352 data=\"UDP test\""}

#TCP message sent
2022-02-01 07:48:15 +0000 [trace]: #0 fluent/log.rb:287:trace: connected fluent socket addr="<IP_ADDRESS>" port=50522
2022-02-01 07:48:15.322055453 +0000 fluent.trace: {"addr":"<IP_ADDRESS>","port":50522,"message":"connected fluent socket addr=\"<IP_ADDRESS>\" port=50522"}
2022-02-01 07:48:15 +0000 [trace]: #0 fluent/log.rb:287:trace: accepted fluent socket addr="<IP_ADDRESS>" port=50522
2022-02-01 07:48:15.322321600 +0000 fluent.trace: {"addr":"<IP_ADDRESS>","port":50522,"message":"accepted fluent socket addr=\"<IP_ADDRESS>\" port=50522"}
2022-02-01 07:48:15 +0000 [warn]: #0 fluent/log.rb:351:warn: incoming chunk is broken: host="<IP_ADDRESS>" msg=84
2022-02-01 07:48:15.322613870 +0000 fluent.warn: {"host":"<IP_ADDRESS>","msg":84,"message":"incoming chunk is broken: host=\"<IP_ADDRESS>\" msg=84"}
2022-02-01 07:48:15 +0000 [warn]: #0 fluent/log.rb:351:warn: incoming chunk is broken: host="<IP_ADDRESS>" msg=67
2022-02-01 07:48:15.322807663 +0000 fluent.warn: {"host":"<IP_ADDRESS>","msg":67,"message":"incoming chunk is broken: host=\"<IP_ADDRESS>\" msg=67"}
2022-02-01 07:48:15 +0000 [warn]: #0 fluent/log.rb:351:warn: incoming chunk is broken: host="<IP_ADDRESS>" msg=80
2022-02-01 07:48:15.322950045 +0000 fluent.warn: {"host":"<IP_ADDRESS>","msg":80,"message":"incoming chunk is broken: host=\"<IP_ADDRESS>\" msg=80"}
2022-02-01 07:48:15 +0000 [warn]: #0 fluent/log.rb:351:warn: incoming chunk is broken: host="<IP_ADDRESS>" msg=32
2022-02-01 07:48:15.323101532 +0000 fluent.warn: {"host":"<IP_ADDRESS>","msg":32,"message":"incoming chunk is broken: host=\"<IP_ADDRESS>\" msg=32"}
2022-02-01 07:48:15 +0000 [warn]: #0 fluent/log.rb:351:warn: incoming chunk is broken: host="<IP_ADDRESS>" msg=116
2022-02-01 07:48:15.323244910 +0000 fluent.warn: {"host":"<IP_ADDRESS>","msg":116,"message":"incoming chunk is broken: host=\"<IP_ADDRESS>\" msg=116"}
2022-02-01 07:48:15 +0000 [warn]: #0 fluent/log.rb:351:warn: incoming chunk is broken: host="<IP_ADDRESS>" msg=101
2022-02-01 07:48:15.323330172 +0000 fluent.warn: {"host":"<IP_ADDRESS>","msg":101,"message":"incoming chunk is broken: host=\"<IP_ADDRESS>\" msg=101"}
2022-02-01 07:48:15 +0000 [warn]: #0 fluent/log.rb:351:warn: incoming chunk is broken: host="<IP_ADDRESS>" msg=115
2022-02-01 07:48:15.323530676 +0000 fluent.warn: {"host":"<IP_ADDRESS>","msg":115,"message":"incoming chunk is broken: host=\"<IP_ADDRESS>\" msg=115"}
2022-02-01 07:48:15 +0000 [warn]: #0 fluent/log.rb:351:warn: incoming chunk is broken: host="<IP_ADDRESS>" msg=116
2022-02-01 07:48:15.323605739 +0000 fluent.warn: {"host":"<IP_ADDRESS>","msg":116,"message":"incoming chunk is broken: host=\"<IP_ADDRESS>\" msg=116"}

It seems that UDP isn't work as you expected, it's recognized as heartbeat, not a message:
2022-02-01 07:48:04.994163324 +0000 fluent.trace: {"host":"<IP_ADDRESS>","port":52352,"data":"UDP test","message":"heartbeat udp data arrived host=\"<IP_ADDRESS>\" port=52352 data=\"UDP test\""}

Although Fluentd opens both TCP and UDP sockets, UDP socket is used as heartbeat: https://docs.fluentd.org/output/forward#port
2022-02-01 07:48:15 +0000 [trace]: #0 fluent/log.rb:287:trace: connected fluent socket addr="<IP_ADDRESS>" port=50522
2022-02-01 07:48:15.322055453 +0000 fluent.trace: {"addr":"<IP_ADDRESS>","port":50522,"message":"connected fluent socket addr=\"<IP_ADDRESS>\" port=50522"}
2022-02-01 07:48:15 +0000 [trace]: #0 fluent/log.rb:287:trace: accepted fluent socket addr="<IP_ADDRESS>" port=50522
2022-02-01 07:48:15.322321600 +0000 fluent.trace: {"addr":"<IP_ADDRESS>","port":50522,"message":"accepted fluent socket addr=\"<IP_ADDRESS>\" port=50522"}
2022-02-01 07:48:15 +0000 [warn]: #0 fluent/log.rb:351:warn: incoming chunk is broken: host="<IP_ADDRESS>" msg=84

This is because it's doesn't follow Fluentd's forward protocol: https://github.com/fluent/fluentd/wiki/Forward-Protocol-Specification-v1
Please consider using fluent-logger-php: https://github.com/fluent/fluent-logger-php


Crow core rewrite

Rewrite the core to more effectively use lisp constructs. This will have a subtle but important change to how the interpreter works. We will also take this opportunity to align our syntax with scheme.

Depends #83

Depends #84

Depends #85

Depends #87

Depends #86


Tasks

The only violation I see is that watchify is in the dependencies when it's already in devDependencies. I'm fixing that now.

Good to go.


[Feature][Connector-V2] Kylin source connector

Purpose of this pull request
refer to https://github.com/apache/incubator-seatunnel/issues/3018
add Kylin source Connector

hive support datatype
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types
kylin jdbc
https://kylin.apache.org/cn/docs31/howto/howto_jdbc.html

tips
Tip: Since  need to configure the kylin cube model  and build the data in the kylin UI to complete the data preparation, it seems that you cannot add e2e to the program. refer to https://kylin.apache.org/cn/docs/tutorial/create_cube.html

Check list

[x] Code changed are covered with tests, or it does not need tests for reason:
[x] If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
[x] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs

Please add e2e in https://github.com/apache/incubator-seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e

Please add e2e in https://github.com/apache/incubator-seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e

Since need to configure the kylin cube model and build the data in the kylin UI to complete the data preparation, it seems that you cannot add e2e to the program. so I  added a tips to describe this

Please fix the CI problems.

Please add e2e test

Please add e2e test
Since need to configure the kylin cube model and build the data in the kylin UI to complete the data preparation, it seems that you cannot add e2e to the program. so I added a tips to describe this

Please add e2e test
Since need to configure the kylin cube model and build the data in the kylin UI to complete the data preparation, it seems that you cannot add e2e to the program. so I added a tips to describe this

Got it, if e2e testing is very difficult. You can comment below your local testing process in the form of pictures, etc. This will make it easier to troubleshoot later if there is a problem.


Changed info about supported platforms and added shields

Just some small tweaks to the readme, thinking of adding a screenshot too..

@m4gr3d realized we have a readme in the root and one that will be added to the release. Kept the release one simple and added a bit of detail to the one in the root.


Problem with image size and pairing image with mask

It looks like image_size is supposed to be single number according to code; even though documentation mentions  them as  tuples. So it seems to throw error when i run it unmodified.
Also in python3.6 the os.listdir() function for getting file names may result in different order of image names and may lead to improper pairing of the image with mask. I got improper image-mask pair  from dataset (using 'example-use' code) when i used jpg images for source; even though i changed extension in code. I think, it would be better to sort this list to ensure they are properly paired before applying any other operations/shuffling, considering the images are properly named.

@anilsathyan7 Thank you for the information about the image_size variable, it seems you are correct, I will update the docs.
About the example with os.listdir, you are correct. It is upto you to prepare the lists so that each image-mask index corresponds to the same pair. This is why the comment in the docs mention "mask-0 is the mask to image-0". I think I can be more explicit about this. I'll find a way to make it clearer.


Rewrite quadrature class to simplify generation of element kernels

Instead of pushing forward quadrature rules in every kernel, make this a constructor of quadratureelement.
Upstream changes in asimov-contact soon to come to match this.

@IgorBaratta  @SarahRo
Pull rquest updated to not use xtensor blas.
The maths.hpp file has also been simplified, to avoid duplication of code between cuas and dolfinx.


SSO Session implementation

We will not keep a (SSO) session on the gateway. Thus:

Each SP authentication will trigger an authentication request at the IdP (e.g. SURFconext). The IdP may offer SSO.
Each authentication > LoA 1 will trigger an authentication of the 2nd factor

We will not forward the SSO session information from the IdP to the SSP, or provide session information to the SP. So in the Response from GW to SP:

No SessionNotOnOrAfter in the AuthnStatement
No SessionIdentifier in the AuthnStatement

The assertion is to be valid for 5 minutes.
See https://wiki.surfnet.nl/display/stepup/SAML+2.0+messages for sample SAML messages

Checked assertion expiration: https://github.com/SURFnet/Stepup-Gateway/blob/01df5457a82af8c6d8c13b4ee32914331d442b9e/src/Surfnet/StepupGateway/GatewayBundle/Service/ProxyResponseService.php#L94
Ensured LOA>1 = trigger 2nd factor: https://github.com/SURFnet/Stepup-Gateway/blob/01df5457a82af8c6d8c13b4ee32914331d442b9e/src/Surfnet/StepupGateway/GatewayBundle/Controller/GatewayController.php#L149

:+1:


Error loading pytorch_volumetric package

Hi, I run into some error while loading the package in my code. Here is the error information. Any suggestion?
File "/local/mnt/anaconda3/envs/idr/lib/python3.7/site-packages/pytorch_volumetric/sdf.py", line 547
return *res[:-1], cache
^
SyntaxError: invalid syntax

maybe you can try with python>=3.8. I think it's an error related to python changes.


Katib: Fix getting started redirect

I fixed redirect to Katib getting started guide.
/assign @kubeflow/wg-training-leads @thesuperzapper

/approve


Fix issue #29 - Problem on fs.copyFileSync when publishing files

Fix issue on fs.copyFileSync when publishing files to destination directory with node v8.5.0

+1

+1

+1

Thank you!!! I'll release this asap!! 👍

Revert
The fs module used is in jsdoc. Not in native node.js v8.5.
So We should update jsdoc versions to 3.5.5.


Request: Could you add optional parameters  to "fitToSuppliedMarkers()" like in "fitToCoordinates()"

The tip of the markers are being chopped off when adjusted to the new co-ordinates, like in the picture below:
Here is my code:
focus=()=> {
    this.mapRef.fitToSuppliedMarkers([
            "Marker1",
            "Marker2",
        ], true);
}

componentDidMount() {
    animationTimeout = setTimeout(() => {
        this.focus();
    }, 2000);
}

componentDidUpdate() {
    animationTimeout = setTimeout(() => {
        this.focus();
    }, 2000);
}

+1 - It would be fantastic if we could specify some padding on the edges of the map! Thanks for all your work on this library.


feat(tag): add support for explicit workspace

Pull Request
🤨 Rationale
https://dev.azure.com/ni/DevCentral/_workitems/edit/2387085

Lets user provide the workspace that a tag is in

👩💻 Implementation

Created a text box for the workspace id (in the future this will be a dropdown)
Added && workspace = "<workspace>" to the query tags filter

🧪 Testing

Added auto tests and improved the fixtures

✅ Checklist

[x] This PR has a title that follows the commit message format.

:tada: This PR is included in version 1.13.0 :tada:
The release is available on:

v1.13.0
GitHub release

Your semantic-release bot :package::rocket:


crash of the MQTT client when receiving "large" messages

MQTT client is disconnecting or crashing when receiving large messages, ( ~ more than 200, 250 bytes )
Details :
Fatal exception (9):
epc1=0x40208957
epc2=0x00000000
epc3=0x40217c98
excvaddr=0x23232327
depc=0x00000000
excsave1=0x40208ace
Registers:
a0 40208ace a1 3fff9620 a2  3fff96a0 a3  23232323
a4  00000000 a5  00000000 a6  3fff2368 a7  00000000
a8  00000000 a9  3ffeaa68 a10 3ffea610 a11 3ffeaa44
a12 3fff97a8 a13 3fff96a0 SAR 00000020
Stack: SP=0x3fff9620
0x3fff9620: 00000000 3fff97a8 3fff9800 4020872c
0x3fff9630: 40208610 3fff96e3 3fff97a8 40208a68
0x3fff9640: 3fff9680 00000020 3fff12e0 00000000
0x3fff9650: 3fff96a0 3fff97a8 fffffffd 40208ace
0x3fff9660: 00000090 3ffea610 3ffeaa44 00000790
0x3fff9670: 3fff97a8 00000010 00000000 00000020
0x3fff9680: 4021588c 00000000 3fff96a0 4020872c
0x3fff9690: 00000010 3fff97a8 fffffffd 40208c27
Free Heap: 27012
_heap_start 0x3fff1418 brk 0x3fffa4d0 supervisor sp 0x3ffffae0 sp-brk 22032 bytes
arena (total_size) 37048 fordblks (free_size) 4980 uordblocks (used_size) 32068
ets Jan  8 2013,rst cause:2, boot mode:(3,7)
load 0x40100000, len 2292, room 16
tail 4
(gdb) list 0x40208957
0x40208957 is in readPacket (/home/theo/Dev/Embedded/esp8266/esp-open-rtos/extras/paho_mqtt_c/MQTTClient.c:92).
87      int rem_len = 0;
88
89      / 1. read the header byte.  This has the packet type in it /
90      Network ip = c->ipstack;
91
92      int (mqttread) (Network, unsigned char*, int, int) = ip->mqttread;
93
94      if ( mqttread(
95          c->ipstack,
96          c->readbuf,

Hello, I'm the one who wrote the Paho port.To be frank I did not test the large message case. Thanks for the catching. I'll look into this. Did you just publish the message through a mosquito server? I'm building a testing case here.
Btw how you get gdb working?

Hi, yep the message came through a mosquitto server.
This is no live debugging, I loaded my binary with xtensa-thingy-gdb, and looked for the line of code that raised the exception.
I tried to make the same thing by disassembling with objdump  ( and mixing with the source code, with the option -S ), but quite an interesting bug happened : all the functions of MQTTClient.c where disassembled correctly but with no C code, whereas it worked fine for other files. Maybe this behavior in consistent with the possibly alignment problem ? I could not get objdump to get more verbose to know what was happening.

It's unrelated to this bugreport, but since the author of the paho_mqtt port is here I'd like to point out a possible bug in MQTTClient.c:
int  sendPacket(MQTTClient* c, int length, Timer* timer)
{
    int rc = FAILURE,
        sent = 0;

    while (sent < length && !expired(timer))
    {
        rc = c->ipstack->mqttwrite(c->ipstack, &c->buf[sent], length, left_ms(timer));
        if (rc < 0)  // there was an error writing the data
            break;
        sent += rc;
    }
    if (sent == length)
    {
        countdown(&(c->ping_timer), c->keepAliveInterval); // record the fact that we have successfully sent the packet
        rc = SUCCESS;
    }
    else
        rc = FAILURE;
    return rc;
}

in the while loop the code keeps sending the full buffer length, for each iteration, not taking into account that it already sent some data.

This seems to be resolved with https://github.com/SuperHouse/esp-open-rtos/pull/158


Fix apartment creation

When you create new apartment you receive a message with exception: "Apartment id must not be null".

Please add your planning poker estimate with ZenHub @BohdanLysenko


Abandoned?

This gem can't even find an issue in JIRA.  Has it been abandoned?

No it has not been abandoned but you need to use the correct distribution. Some information would be helpful.

JIRA Version?
Ruby Version?
Jiralicious Version?
What errors are you receiving?

Latest commit 2014. Two open PRs.
Can you confirm this is a non-abandoned project?

Added another PR to the list :)

The project is not abandoned but it is for the most part complete in the consumption of the JIRA API. So unless the API changes it's minor tweaks which I rely on the other uses to assist with since I have not had any issues for anything I have used the gem for.


Autoplay not working on Chrome PC... and playback not working on Samsung Browser

Working on Firefox, Edge... but not working on Chrome PC
In addition, can't be played on Samsung Browser (Android). Most streams fail with Clappr. (Error code: html5_video:4)
http://www.rsbdylgy.lucusvirtual.es/dw.htm
Any idea?

@hijuuijj thanks for reporting. This appears to be related to Chrome (and Safari) blocking autoPlay with sound.
This one won't play (when attempting autoPlay with sound): autoPlay with sound
This one will play correctly (starting autoPlay with mute: true): autoPlay without sound
Chrome has some documentation on how it whitelists domains based on its Media Engagement Index (MEI).

The same logic applies to mobile browses as well. The only difference is that most browsers (especially Safari) on mobile devices will always block autoplay with audio.
We'll release a "click to unmute" plugin to help prevent these kind of issues, while warning the user that the video started autoplaying without sound, and an interaction is necessary to enable audio. If you're not able to wait for it, you can develop it yourself, perhaps based on what youtube does on mobile pages.