[Cloudtools APKG HOW-TO] Hubic

User avatar
father.mande
Posts: 1819
youtube meble na wymiar Warszawa
Joined: Sat Sep 12, 2015 2:55 am
Location: La Rochelle (France)

[Cloudtools APKG HOW-TO] Hubic

Post by father.mande »

Hi,

This is a series of messages to explain usage for each cloud target using cloudtools command line or mount tools
Please for support use the "PRODUCT" forum and help
... the source code of the program have not been changed in any manner (except if difference is listed here ... in fact only for Bash interface)

[Hubic]
Prerequisite :
need an hubic account
hubicfuse : NOTHING
HubiC (official client) : mono APKG (install it from AppCentral)

=== in my opinion ... hubicfuse seems to have some difficulty to always succeed to validate credentials and performance are medium to poor (even if LOT of parameters can help) to increase it
=== sometimes all is fine ... others time it's erratic ...
=== so please analyse your need and choice Official Hubic client if you don't clearly need direct access ...

[hubicfuse] (in fact fasthubicfuse version)
Tool to mount as a file system your Hubic account structure
... data are NOT synchronized or copied localy ... work directly on server ... or through a cache
... using cache increase performance but also can generate difference between local view and server view (up to cache is flushed)
The usage in phases :
1 : connect you to https://hubic.com/home/browser/developers/ ... then in developers menu add an application ... (use http://localhost as redirection)
then
use FIRST time and only ONE time the script hubic_token (in /usr/local/AppCentral/cloudtools/ folder) and follow instructions (need to have a browser to join hubic during this phase) call it using ./hubic_token -k (alternatives exist to get the token ... search on fasthubicfuse git site for ex.)
... this provide you list of values
client_id (get from dev. site)
client_secret (get from dev. site)
refresh_token (generated)
2 : create a .hubicfuse file (in the HOME folder of the user ... root ( /root/ ) to facilitate the mount and auto-mount
... you can be helped by using the sample supply (with lot of parameters better for performance (I hope)
... please get time to search on the Web for each value and target :lol: ... I am not an expert just an integrator ...
... IMPORTANT the temp_dir IS IMPORTANT (default is /tmp ... so full in a short time (so crash !!!) in normal usage

Code: Select all

root@AS5002TaPhil:/share/Public/apkg/for_64b/cloudtools # cat .hubicfuse.sample
client_id=api_hubic_xxxxxxxxx_from_hubic_dev_web_app
client_secret=xxxxxxxxxxxxxx_from_hubic_dev_web_app
refresh_token=xxxxxxxx_from_hubic_token_script
temp_dir=/share/cloudtools/tmp

timout = 600
segment_size = 30000000
segment_above = 30000001
get_extended_metadata = true
curl_verbose = false
curl_progress_state = false
cache_statfs_timeout = 30
debug_level = 0
enable_chmod = false
enable_chown = false
option_enable_progressive_upload = false
enable_progressive_download = false
min_speed_limit_progressive = 0
min_speed_timeout = 3
read_ahead = 0
enable_syslog = true
cache_max_size = 1048576000
enable_chaos_test_monkey = false
disable_atime_check=false
http_log_path=""
fast_list_dir_limit=0
async_delete = false
root@AS5002TaPhil:/share/Public/apkg/for_64b/cloudtools #
3 : after that you can use hubicfuse to mount (as a file system) the hubic structure in your file system
... please use the specific share resource created for this : /share/cloudtools/hubic

Hereafter the hubicfuse phase 1 : (please follow response as is or change if you know why ... )
... enter client_id, client_secret and redirect_url provide by developer definition before on hubic developers web site ...

Code: Select all

root@AS5002TaPhil:/volume1/.@plugins/AppCentral/cloudtools # ./hubic_token -k
client_id (the app's id): api_hubic_response_get_from_dev_site_app_definition
client_secret (the app's secret): response_get_from_dev_site_app_secret
redirect_uri (declared at app's creation): http://localhost/

For the scope -what you authorize your app to do-, enter characters as suggested
in parenthesis, or just hit return if you don't need the item authorized.
Get account usage (r): r
Get all published links in one call (r): r
Get OpenStack credentials, eg. access to your files (r): r
Send activation email (w): w
Add new/Get/Delete published link (wrd): wrd

user_login (the e-mail you used to subscribe): xxxxxxxxxx@yyyymail.com
user_pwd (your hubiC's main password):

Success!


# Here is what your app needs to connect to hubiC:
client_id=api_hubic_kxxxnlzyS6FTpLXNhMzPxxxxxxx
client_secret=xxxxxrzsg41tpBkqapi6mwX6PpAoZBGLpK3sQWtD0QXWfxxxxxxx
refresh_token=xxxE18f7QiT7cgkdWzo5M5osrnTgS1Br5Up578WQLuTBmelxxxxxxx
THEN REPORT THE RESULT (last free lines) ... in the .hubicfuse file (to be created) in /root/ folder

Now you can mount your hubic folders :
ALL DATA are under default folder (NEVER CHANGE somthings in others root(s) folder ... )

Code: Select all

root@AS5002TaPhil:/volume1/.@plugins/AppCentral/cloudtools # hubicfuse /share/cloudtools/hubic -o noauto_cache,sync_read,allow_other
root@AS5002TaPhil:/volume1/.@plugins/AppCentral/cloudtools # ll /share/cloudtools/hubic
total 4
drwxr-xr-x    2 root     root           0 Jan  1  1970 ./
drwxr-xr-x    7 root     root        4.0K May 19 17:03 ../
drwxr-xr-x    2 root     root           0 May 19 17:04 default/
drwxr-xr-x    2 root     root           0 May 19 17:04 default_segments/
root@AS5002TaPhil:/volume1/.@plugins/AppCentral/cloudtools # ll /share/cloudtools/hubic/default
total 0
drwxr-xr-x    2 root     root           0 May 19 17:04 ./
drwxr-xr-x    2 root     root           0 Jan  1  1970 ../
drwxr-xr-x    2 root     root           0 Oct 31  2014 Documents/
drwxr-xr-x    2 root     root           0 Oct 31  2014 Images/
drwxr-xr-x    2 root     root           0 Oct 31  2014 Videos/
root@AS5002TaPhil:/volume1/.@plugins/AppCentral/cloudtools # ll /share/cloudtools/hubic/default/Documents/
total 4116
drwxr-xr-x    2 root     root           0 Oct 31  2014 ./
drwxr-xr-x    2 root     root           0 May 19 17:04 ../
drwxr-xr-x    2 root     root           0 Feb 17  2015 .upload_cache/
-rw-rw-rw-    1 root     root        4.0M Oct 31  2014 T100TAAS.227
-rw-rw-rw-    1 root     root        2.9K Feb 17  2015 protocols
-rw-rw-rw-    1 root     root        9.1K Feb 17  2015 trace_qvm.rtf
AS any Fuse network mount ... you can access the mount only using locals tools NOR by default using php, or java in web tools or Samba, etc. ... (due to the fact that folder appear as null (siize 0) files) BUT all of "Linux local" programs works as rsync, tar, etc. etc.

umount is done as usual :
umount /share/cloudtools/hubic

AFTER if you want ... you can add this in the start_stop.sh script in the APKG folder to do it at boot (but remmeber the problem with credentials ... so perhaps add test on status return :) .

[HubiC]official client
It's a C# application (from Windows) ... so require "mono APKG"
HubiC (aka hubic command) is a server (launch first time or on demand when credentials is know) AND the client to send command to the server ...
HubiC use Dbus to exchange
HubiC (hubic program) is a SYNCHRONIZATION tool ... so ALL data are copied to a local folder 'except if you exclude some one)
... this imply that you can have a full access with all application BUT also that you need some time for the first synchronization ...
BUT the major point is to use an "official" client ... so with more support from Hubic side

HubiC use a folder for synchronization proposed is : /share/cloudtools/HubiC (caps dependent)
First login create a credentials under $HOME/.mono/...... /ovh/....
be attention to don't change by error the folder used for synchronization ... or manage it

hubic (a link to HubiC) can also accept parameters (like password_file_path) if you want to use it in bash or script ...
I just show you a short usage example ... but you can do step by step to have more control ...

Code: Select all

root@AS5002TaPhil:/volume1/.@root # hubic login yyyyyy@xxxmail.com /share/cloudtools/HubiC/
Password
root@AS5002TaPhil:/volume1/.@root #
... here NO password file so password is requested ...
This command log you then start synchronization in /share/cloudtools/HubiC folder
status show you the state and download etc.

Code: Select all

root@AS5002TaPhil:/volume1/.@root # hubic status
State: Busy
Up: 0 B/s (0 B/s)       Down: 295.76 KB/s (0 B/s)

Account: yyyyyy@xxxxmail.com
Synchronized directory: /share/cloudtools/HubiC/
Usage: 5.11 MB/25 GB

Queue:
        Uploads: 0 (0 B) + 0 running
        Downloads: 0 (0 B) + 1 running
        Misc: 0 + 0 running

Running operations:
        Download for /share/cloudtools/HubiC/Images/Koala.jpg

Last events:
        [5/22/2016 12:06:32 PM|Info] Click on this icon to access your hubiC.
root@AS5002TaPhil:/volume1/.@root #
when finished ... status is Idle

Code: Select all

root@AS5002TaPhil:/volume1/.@root # hubic status
State: Idle
Up: 0 B/s (0 B/s)       Down: 0 B/s (0 B/s)

Account: yyyyyy@xxxxmail.com
Synchronized directory: /share/cloudtools/HubiC/
Usage: 5.11 MB/25 GB

Last events:
        [5/22/2016 12:06:32 PM|Info] Click on this icon to access your hubiC.
root@AS5002TaPhil:/volume1/.@root #
NOW your local copy of the data are available for any usage ... from FTP, Samba, Web file manager, Rsync, etc.
(is not like a fuse network mount ... data are really here)
After play as you want with hubic command
... remember to use : hubic help command ... to get explain ...

Code: Select all

root@AS5002TaPhil:/volume1/.@root # ll /share/cloudtools/HubiC/
total 20
drwsrwsrwt    5 root     root        4.0K May 22 12:06 ./
drwxrwxrwx    6 admin    administ    4.0K May 21 18:28 ../
drwxr-sr-x    3 root     root        4.0K May 22 12:06 Documents/
drwxr-sr-x    3 root     root        4.0K May 22 12:07 Images/
drwxr-sr-x    2 root     root        4.0K May 22 12:06 Videos/
root@AS5002TaPhil:/volume1/.@root # ll /share/cloudtools/HubiC/Documents/
total 4128
drwxr-sr-x    3 root     root        4.0K May 22 12:06 ./
drwsrwsrwt    5 root     root        4.0K May 22 12:06 ../
drwxr-sr-x    2 root     root        4.0K May 22 12:06 .upload_cache/
-rw-r--r--    1 root     root        4.0M May 22 12:06 T100TAAS.227
-rw-r--r--    1 root     root        2.9K Feb 17  2015 protocols
-rw-r--r--    1 root     root        9.1K Feb 17  2015 trace_qvm.rtf
root@AS5002TaPhil:/volume1/.@root 
Linux status (of running server can be also obtain by

Code: Select all

root@AS5002TaPhil:/volume1/.@root # ps -eaf | grep hub
root      5552 24860  0 12:11 pts/1    00:00:00 grep hub
root     31857     1  2 12:06 ?        00:00:09 mono /usr/local/AppCentral/cloudtools/lib/hubic/hubiC.exe main-loop
root@AS5002TaPhil:/volume1/.@root #
Have fun.
Philippe.
AS6602T / AS5202T /AS5002T / AS1002T / FS6706T
damien599901
Posts: 575
Joined: Mon Dec 30, 2013 2:53 am

Re: [Cloudtools APKG HOW-TO] Hubic

Post by damien599901 »

Dear Philippe,

could explain how to proceed?

"HubiC use a folder for synchronization proposed is : /share/cloudtools/HubiC (caps dependent)
First login create a credentials under $HOME/.mono/...... /ovh/....
be attention to don't change by error the folder used for synchronization ... or manage it"
Storage: AS7004T & AS5002T
Network: FREEBOX REVOLUTION V6
Laptop: Apple MACBOOK Pro OS X El Capitan & Windows 10
Media Player: ASUSTOR NAS with Kodi 16 Beta & HD_Engine 1 (thanks Fathe_Mande)
Portable: iPhone 6S Plus, iPad Mini 2 & iPad Air
User avatar
father.mande
Posts: 1819
Joined: Sat Sep 12, 2015 2:55 am
Location: La Rochelle (France)

Re: [Cloudtools APKG HOW-TO] Hubic

Post by father.mande »

Hi,
damien599901 wrote:Dear Philippe,
could explain how to proceed?
"HubiC use a folder for synchronization proposed is : /share/cloudtools/HubiC (caps dependent)
First login create a credentials under $HOME/.mono/...... /ovh/....
be attention to don't change by error the folder used for synchronization ... or manage it"
Not sure I understand your question ???

HubiC (official client) use a folder to clone the data from the hubic account
... so as it's show in the topic ... you must put this local folder in the command to inform HubiC where it must clone the data ...
... I have created a folder in the shared resource /share/cloudtools ... for that
BUT as usual in any Linux you can create your own folder and use it in HubiC command line (hubic program)
When you start the first time HubiC ... mono create a credentials structure associated to the mono application ... here HubiC in the HOME folder of the user starting the application ...
... this information is just for as usual in Linux :
... inform you where are the credential (for expert to save it and reuse it in case of ... )
... inform you that, as other, the synchronization is associated to a specific user ... so changing user ... can't be done without regeneration ALL from another one ...

Each tools that clone and synchronize data ... manage an index with deterministic information ... to don't clone twice and manage modification
... so if you change the original target folder (where data are cloned) this don't works ... by incoherence structure ...
SO my text is only a warning ... to prevent potential problem ...

Philippe.
NB at all I don't write a full documentation ... only a first basic approach ... so you can have more detail on developer site and forum (in hubic web site) :roll: :)
AS6602T / AS5202T /AS5002T / AS1002T / FS6706T
damien599901
Posts: 575
Joined: Mon Dec 30, 2013 2:53 am

Re: [Cloudtools APKG HOW-TO] Hubic

Post by damien599901 »

brilliant again Philippe !!!
but i see this error message (probably my fault)
"Cannot contact daemon, are you sure it is running?"
Storage: AS7004T & AS5002T
Network: FREEBOX REVOLUTION V6
Laptop: Apple MACBOOK Pro OS X El Capitan & Windows 10
Media Player: ASUSTOR NAS with Kodi 16 Beta & HD_Engine 1 (thanks Fathe_Mande)
Portable: iPhone 6S Plus, iPad Mini 2 & iPad Air
User avatar
father.mande
Posts: 1819
Joined: Sat Sep 12, 2015 2:55 am
Location: La Rochelle (France)

Re: [Cloudtools APKG HOW-TO] Hubic

Post by father.mande »

Hi,
damien599901 wrote: but i see this error message (probably my fault)
"Cannot contact daemon, are you sure it is running?"

Hum! ... NO it's not your fault .. it's generally done by a problem with dbus ... I have try to solve this (and it works on my AS5002T ADM 2.6 ) but don't be sure for the rest ...

1 did you use root or admin to connect you ? ... all is manage to use root ONLY ... or it's need another command line ...
2 do and provide output :
ps -eaf | grep hub and verify you have somethings like this

Code: Select all

root@AS5002TaPhil:/volume1/.@root # ps -eaf | grep hub
root     28554 28215  0 13:37 pts/2    00:00:00 grep hub
root     31857     1  0 May22 ?        00:02:11 mono /usr/local/AppCentral/cloudtools/lib/hubic/hubiC.exe main-loop
root@AS5002TaPhil:/volume1/.@root #
3 do and provide output :
ps -eaf | grep dbus (don't be afraid if you don't have all the line the interesting (for mz) is the line with --print-address=3

Code: Select all

root@AS5002TaPhil:/volume1/.@root # ps -eaf | grep dbus
root       585 28215  0 13:41 pts/2    00:00:00 grep dbus
nobody    2946     1  0 May22 ?        00:00:00 /usr/bin/dbus-daemon --system --fork
root      5376     1  0 May22 ?        00:00:00 /usr/local/AppCentral/xorg/usr/bin/dbus-launch --autolaunch 2b8402487876264543e3118656db087f --binary-syntax --close-stderr
root      5401     1  0 May22 ?        00:00:21 /usr/local/AppCentral/xorg/bin/xorg-dbusd --fork --print-pid 5 --print-address 7 --session
101       6387     1  0 May22 ?        00:00:00 /bin/dbus-daemon --system
65503     7107     1  0 May22 ?        00:00:00 /usr/bin/dbus-launch --sh-syntax --exit-with-session /usr/bin/xfce4-session
65503     7112     1  0 May22 ?        00:00:00 //bin/dbus-daemon --fork --print-pid 5 --print-address 7 --session
root      7228     1  0 May22 ?        00:00:00 /usr/bin/dbus-daemon --session --fork --print-address=3
65503     7578  7572  0 May22 ?        00:00:00 /bin/dbus-daemon --config-file=/etc/at-spi2/accessibility.conf --nofork --print-address 3
root@AS5002TaPhil:/volume1/.@root #
3 also provide content of /usr/local/AppCentral/cloudtools/.dbus_session ...

A least a reboot can solve this problem ... if something goes wrong and a loop exist ...

Explain :
Dbus is a function in Linux to manage inter-process exchange using "socket" (like network but in fact through file (in /tmp )) . Due to the lack of dbus-launch program ...
hubic is NOT able to launch itself a dbus process to manage exchange between hubic acting as a server and hubic acting as a client
To solve this :
during the start process (so under root) I will start a dbus-daemon manually and get the dbus requested information ... that I store in .dbus_session of cloudtools
on each command I get the content of this file to set a specific environment value : DBUS_SESSION_BUS_ADDRESS ... so hubic server now is launch with this SESSION value and hubic client use also this value to speak with hubic server ... so they are able to speak together even the "normal" command don't exist ...
This work (generally) even it's a sort of bad hack ...
But if for any reason this DBUS SESSSION don't match ... you receive this error ...
"Cannot contact daemon, are you sure it is running?"

Philippe.
AS6602T / AS5202T /AS5002T / AS1002T / FS6706T
damien599901
Posts: 575
Joined: Mon Dec 30, 2013 2:53 am

Re: [Cloudtools APKG HOW-TO] Hubic

Post by damien599901 »

Hi Philippe

1) by root
2) nothing happens
3) 4281 nobody 0:32 /usr/bin/dbus-daemon --system --fork
6237 root 0:00 /usr/local/AppCentral/xorg/usr/bin/dbus-launch --autolaunch 8a4b3f7c4384803eebbbddf557077f08 --binary-syntax --close-stderr
6305 root 0:06 /usr/local/AppCentral/xorg/bin/xorg-dbusd --fork --print-pid 5 --print-address 7 --session
9562 root 0:00 grep dbus
9770 104 0:00 /usr/bin/dbus-daemon --system
14856 root 0:00 looksgood-dbusd --session --print-address --fork
Storage: AS7004T & AS5002T
Network: FREEBOX REVOLUTION V6
Laptop: Apple MACBOOK Pro OS X El Capitan & Windows 10
Media Player: ASUSTOR NAS with Kodi 16 Beta & HD_Engine 1 (thanks Fathe_Mande)
Portable: iPhone 6S Plus, iPad Mini 2 & iPad Air
User avatar
father.mande
Posts: 1819
Joined: Sat Sep 12, 2015 2:55 am
Location: La Rochelle (France)

Re: [Cloudtools APKG HOW-TO] Hubic

Post by father.mande »

Hi,

Can you manage to reboot ...
if we don't have a line in dbus list like this :
"root 7228 1 0 May22 ? 00:00:00 /usr/bin/dbus-daemon --session --fork --print-address=3"
nothing can run ...

can you also check the content of
/usr/local/AppCentral/cloudtools/.dbus_session
... sorry I duplicate point 3 ... so you jump this point ...

this imply that dbus-daemon is not started for hubic ...

I will check my script (perhaps an error)
did you use ADM 2.6 ??? because if dbus management change with early version ... I can't test it ...

Philippe.
AS6602T / AS5202T /AS5002T / AS1002T / FS6706T
User avatar
father.mande
Posts: 1819
Joined: Sat Sep 12, 2015 2:55 am
Location: La Rochelle (France)

Re: [Cloudtools APKG HOW-TO] Hubic

Post by father.mande »

Hi Damien,

Hum! seem I use a hammer to stick a thumbtack ...

reboot is not necessary (I think) just disabling the APKG then re-enabling must be enough.

sorry.
Philippe.
AS6602T / AS5202T /AS5002T / AS1002T / FS6706T
damien599901
Posts: 575
Joined: Mon Dec 30, 2013 2:53 am

Re: [Cloudtools APKG HOW-TO] Hubic

Post by damien599901 »

father.mande wrote:Hi,

Can you manage to reboot ...
works now Philippe ! you rock again
it s a pity that i can't create nothing hahahaha. indeed a web UI could be great :cry:
Storage: AS7004T & AS5002T
Network: FREEBOX REVOLUTION V6
Laptop: Apple MACBOOK Pro OS X El Capitan & Windows 10
Media Player: ASUSTOR NAS with Kodi 16 Beta & HD_Engine 1 (thanks Fathe_Mande)
Portable: iPhone 6S Plus, iPad Mini 2 & iPad Air
damien599901
Posts: 575
Joined: Mon Dec 30, 2013 2:53 am

Re: [Cloudtools APKG HOW-TO] Hubic

Post by damien599901 »

hi Philippe

i try to use Hubicfuse

"1 : connect you to https://hubic.com/home/browser/developers/ ... then in developers menu add an application ... (use http://localhost as redirection) "

The app name of my choice? Indeed, if yes, i ve tried with NAS etc and finally succeeded with Cloudtools :lol:

"2 : create a .hubicfuse file (in the HOME folder of the user ... root ( /root/ ) to facilitate the mount and auto-mount"
=> same question than in the other topic => an automation is possible ?
Storage: AS7004T & AS5002T
Network: FREEBOX REVOLUTION V6
Laptop: Apple MACBOOK Pro OS X El Capitan & Windows 10
Media Player: ASUSTOR NAS with Kodi 16 Beta & HD_Engine 1 (thanks Fathe_Mande)
Portable: iPhone 6S Plus, iPad Mini 2 & iPad Air
Post Reply

Return to “Cloodtools”