conpot.core package¶
Subpackages¶
- conpot.core.loggers package
- Submodules
- conpot.core.loggers.helpers module
- conpot.core.loggers.hpfriends module
- conpot.core.loggers.json_log module
- conpot.core.loggers.log_worker module
- conpot.core.loggers.mysql_log module
- conpot.core.loggers.sqlite_log module
- conpot.core.loggers.stix_transform module
- conpot.core.loggers.syslog module
- conpot.core.loggers.taxii_log module
- Module contents
Submodules¶
conpot.core.attack_session module¶
conpot.core.databus module¶
conpot.core.filesystem module¶
-
class
conpot.core.filesystem.
AbstractFS
(src_path: str, create_mode: int = 511, temp_dir: Optional[str] = None, identifier: Optional[str] = '__conpot__', auto_clean: Optional[bool] = True, ignore_clean_errors: Optional[bool] = True)¶ Bases:
fs.wrapfs.WrapFS
AbstractFS distinguishes between “real” filesystem paths and “virtual” ftp paths emulating a UNIX chroot jail where the user can not escape its home directory (example: real “/home/user” path will be seen as “/” by the client)
This class exposes common fs wrappers around all os.* calls involving operations against the filesystem like creating files or removing directories (such as listdir etc.)
Implementation Note: When doing I/O - Always with the check_access and set_access context managers for safe operations.
-
access
(path: str, name_or_id: Union[int, str] = None, required_perms: str = None)¶ Returns bool w.r.t the a user/group has permissions to read/write/execute a file. This is a wrapper around os.access. But it would accept name or id instead of of just ids. Also it can accept required permissions in the form of strings rather than os.F_OK, os.R_OK, os.W_OK etc.
Implementation Note: First we would check whether the current user has the required permissions. If not, then we check the group to which this user belongs to. Finally if the user’s group also does not meet the perms we check for other permissions.
-
add_users_to_group
(gid: int, uids: List[T]) → None¶ Add list of users to an existing group :param gid: Group id of the group. :param uids: List of registers users that belong to this group
-
check_access
(path=None, user=None, perms=None)¶ Checks whether the current user has permissions to do a specific operation. Raises FSOperationNotPermitted exception in case permissions are not satisfied. Handy utility to check whether the user with uid provided has permissions specified. Examples:
>>> import conpot.core as conpot_core >>> _vfs, _ = conpot_core.get_vfs('ftp') >>> with _vfs.check_access(path='/', user=13, perms='rwx'): >>> _vfs.listdir('/')
>>> with _vfs.check_access(path='/', user=45, perms='w'): >>> with _vfs.open('/test', mode='wb') as _file: >>> _file.write(b'Hello World!')
-
chmod
(path: str, mode: oct, recursive: bool = False) → None¶ Change file/directory mode. :param path: Path to be modified. :param mode: Operating-system mode bitfield. Must be in octal’s form. Eg: chmod with (mode=0o755) = Permissions(user=’rwx’, group=’rx’, other=’rx’) :param recursive: If the path is directory, setting recursive to true would change permissions to sub folders and contained files. :type recursive: bool
-
chown
(fs_path: str, uid: int, gid: int, recursive: Optional[bool] = False) → None¶ Change the owner of a specified file. Wrapper for os.chown :param fs_path: path or directory in the VFS where chown would be executed. :param uid: The uid of the user. **User must be a registered user on the filesystem or an exception would be thrown. :param gid: The gid of the group **Group must be a registered group on the filesystem or an exception would be thrown. :param recursive: If the given path is directory, then setting the recursive option to true would walk down the tree and recursive change permissions in the cache.
** fs_path needs to be the absolute path w.r.t to the vfs. If you are in a sub file system, please use subvfs.getcwd() to get the current directory. **
-
clean
()¶ Clean (delete) temporary files created by this filesystem.
-
copy
(src_path, dst_path, overwrite=False)¶ Copy file contents from
src_path
todst_path
.- Arguments:
src_path (str): Path of source file. dst_path (str): Path to destination file. overwrite (bool): If True, overwrite the destination file
if it exists (defaults to False).- Raises:
- fs.errors.DestinationExists: If
dst_path
exists, - and
overwrite
is False. - fs.errors.ResourceNotFound: If a parent directory of
dst_path
does not exist.
- fs.errors.DestinationExists: If
-
create_group
(name: str, gid: int) → None¶ Store all group related data for the file system. :param name: Name of the group :param gid: gid of the group
-
create_jail
(path)¶ Returns chroot jail sub system for a path
-
format_list
(basedir, listing)¶ Return an iterator object that yields the entries of given directory emulating the “/bin/ls -lA” UNIX command output. This is how output should appear: -rw-rw-rw- 1 owner group 7045120 Sep 02 3:47 music.mp3 drwxrwxrwx 1 owner group 0 Aug 31 18:50 e-books -rw-rw-rw- 1 owner group 380 Sep 02 3:40 module.py
Parameters: - basedir – (str) must be protocol relative path
- listing – (list) list of files to needed for output.
-
get_permissions
(path)¶ Get permissions for a particular user on a particular file/directory in ‘rwxrx—’ format
-
getcwd
()¶
-
getfile
(path, file, chunk_size=None, **options)¶ Copies a file from the filesystem to a file-like object.
This may be more efficient that opening and copying files manually if the filesystem supplies an optimized method.
- Arguments:
path (str): Path to a resource. file (file-like): A file-like object open for writing in
binary mode.- chunk_size (int, optional): Number of bytes to read at a
- time, if a simple copy is used, or None to use sensible default.
- **options: Implementation specific options required to open
- the source file.
Note that the file object
file
will not be closed by this method. Take care to close it after this method completes (ideally with a context manager).- Example:
>>> with open('starwars.mov', 'wb') as write_file: ... my_fs.download('/movies/starwars.mov', write_file)
- Note:
Deprecated since version 2.2.0: Please use ~download
-
getinfo
(path: str, get_actual: bool = False, namespaces=None)¶ Get information about a resource on a filesystem.
- Arguments:
path (str): A path to a resource on the filesystem. namespaces (list, optional): Info namespaces to query
(defaults to [basic]).- Returns:
- ~fs.info.Info: resource information object.
For more information regarding resource information, see info.
-
getmeta
(namespace='standard')¶ Get meta information regarding a filesystem.
- Arguments:
- namespace (str): The meta namespace (defaults
- to
"standard"
).
- Returns:
- dict: the meta information.
Meta information is associated with a namespace which may be specified with the
namespace
parameter. The default namespace,"standard"
, contains common information regarding the filesystem’s capabilities. Some filesystems may provide other namespaces which expose less common or implementation specific information. If a requested namespace is not supported by a filesystem, then an empty dictionary will be returned.The
"standard"
namespace supports the following keys:key Description case_insensitive True if this filesystem is case insensitive. invalid_path_chars A string containing the characters that may not be used on this filesystem. max_path_length Maximum number of characters permitted in a path, or None for no limit. max_sys_path_length Maximum number of characters permitted in a sys path, or None for no limit. network True if this filesystem requires a network. read_only True if this filesystem is read only. supports_rename True if this filesystem supports an os.rename operation. Most builtin filesystems will provide all these keys, and third- party filesystems should do so whenever possible, but a key may not be present if there is no way to know the value.
- Note:
- Meta information is constant for the lifetime of the filesystem, and may be cached.
-
getmtime
(path)¶ Return the last modified time as a number of seconds since the epoch.
-
groups
¶
-
listdir
(path)¶ Get a list of the resource names in a directory.
This method will return a list of the resources in a directory. A resource is a file, directory, or one of the other types defined in ~fs.ResourceType.
- Arguments:
- path (str): A path to a directory on the filesystem
- Returns:
- list: list of names, relative to
path
. - Raises:
- fs.errors.DirectoryExpected: If
path
is not a directory. fs.errors.ResourceNotFound: Ifpath
does not exist.
-
makedir
(path, permissions=None, recreate=True)¶ Make a directory.
- Arguments:
path (str): Path to directory from root. permissions (~fs.permissions.Permissions, optional): a
Permissions instance, or None to use default.- recreate (bool): Set to True to avoid raising an error if
- the directory already exists (defaults to False).
- Returns:
- ~fs.subfs.SubFS: a filesystem whose root is the new directory.
- Raises:
- fs.errors.DirectoryExists: If the path already exists. fs.errors.ResourceNotFound: If the path is not found.
-
mount_fs
(dst_path: str, fs_url: str = None, owner_uid: Optional[int] = 0, group_gid: Optional[int] = 0, perms: Union[fs.permissions.Permissions, int, None] = 493) → fs.subfs.SubFS¶ To be called to mount individual filesystems. :param fs_url: Location/URL for the file system that is to be mounted. :param dst_path: Place in the Conpot’s file system where the files would be placed. This should be relative to FS root. :param owner_uid: The owner user UID of the directory and the sub directory. Default is root/ :param group_gid: The group ‘group` to which the directory beings. Defaults to root. :param perms: Permission UMASK
-
move
(src_path, dst_path, overwrite=False)¶ Move a file from
src_path
todst_path
.- Arguments:
src_path (str): A path on the filesystem to move. dst_path (str): A path on the filesystem where the source
file will be written to.- overwrite (bool): If True, destination path will be
- overwritten if it exists.
- Raises:
- fs.errors.FileExpected: If
src_path
maps to a - directory instead of a file.
- fs.errors.DestinationExists: If
dst_path
exists, - and
overwrite
is False. - fs.errors.ResourceNotFound: If a parent directory of
dst_path
does not exist.
- fs.errors.FileExpected: If
-
norm_path
(path)¶
-
open
(path, mode='r', buffering=-1, encoding=None, newline='', line_buffering=False, **options)¶ Open a file.
- Arguments:
path (str): A path to a file on the filesystem. mode (str): Mode to open the file object with
(defaults to r).- buffering (int): Buffering policy (-1 to use
- default buffering, 0 to disable buffering, 1 to select line buffering, of any positive integer to indicate a buffer size).
- encoding (str): Encoding for text files (defaults to
utf-8
)- errors (str, optional): What to do with unicode decode errors
- (see codecs module for more information).
newline (str): Newline parameter. **options: keyword arguments for any additional information
required by the filesystem (if any).- Returns:
- io.IOBase: a file-like object.
- Raises:
fs.errors.FileExpected: If the path is not a file. fs.errors.FileExists: If the file exists, and exclusive mode
is specified (x
in the mode).fs.errors.ResourceNotFound: If the path does not exist.
-
openbin
(path, mode='r', buffering=-1, **options)¶ Open a file in the ConpotFS in binary mode.
-
opendir
(path, factory=<class 'conpot.core.fs_utils.SubAbstractFS'>)¶ Get a filesystem object for a sub-directory.
- Arguments:
path (str): Path to a directory on the filesystem. factory (callable, optional): A callable that when invoked
with an FS instance andpath
will return a new FS object representing the sub-directory contents. If nofactory
is supplied then ~fs.subfs.SubFS will be used.- Returns:
- ~fs.subfs.SubFS: A filesystem representing a sub-directory.
- Raises:
- fs.errors.DirectoryExpected: If
dst_path
does not - exist or is not a directory.
- fs.errors.DirectoryExpected: If
-
readlink
(path)¶ Perform a readlink() system call. Return a string representing the path to which a symbolic link points. :param path: (str) must be protocol relative path
-
register_user
(name: str, uid: int) → None¶ Store all user related data for the file system.
-
remove
(path)¶ Remove a file from the file system.
-
removedir
(path, rf=True)¶ Remove a directory from the file system. :param path: directory path :param rf: remove directory recursively and forcefully. This removes directory even if there is any data in it. If set to False, an exception would be raised
-
root
¶ The root directory - where the filesystem is stored
-
setbinfile
(path, file)¶ Set a file to the contents of a binary file object.
This method copies bytes from an open binary file to a file on the filesystem. If the destination exists, it will first be truncated.
- Arguments:
path (str): A path on the filesystem. file (io.IOBase): a file object open for reading in
binary mode.- chunk_size (int, optional): Number of bytes to read at a
- time, if a simple copy is used, or None to use sensible default.
- **options: Implementation specific options required to open
- the source file.
Note that the file object
file
will not be closed by this method. Take care to close it after this method completes (ideally with a context manager).- Example:
>>> with open('~/movies/starwars.mov', 'rb') as read_file: ... my_fs.upload('starwars.mov', read_file)
- Note:
Deprecated since version 2.2.0: Please use ~upload
-
setinfo
(path, info)¶ Higher level function to directly change values in the file system. Dictionary specified here changes cache values. :param path: path of the file that is to be changed :param info: Raw Info object. Please check pyfilesystem2’s docs for more info.
-
settimes
(path, accessed=None, modified=None)¶ Set the accessed and modified time on a resource.
- Arguments:
path: A path to a resource on the filesystem. accessed (datetime, optional): The accessed time, or
None (the default) to use the current time.- modified (datetime, optional): The modified time, or
- None (the default) to use the same time as the
accessed
parameter.
-
stat
(path)¶ Perform a stat() system call on the given path. :param path: (str) must be protocol relative path
-
take_snapshot
()¶ Take snapshot of entire filesystem. :rtype: dict
-
user_groups
¶ gid: {set of uid of users.}
-
users
¶
-
conpot.core.fs_utils module¶
Utils related to ConpotVFS
-
exception
conpot.core.fs_utils.
FSOperationNotPermitted
(msg=None)¶ Bases:
fs.errors.FSError
Custom class for filesystem-related exceptions.
-
exception
conpot.core.fs_utils.
FilesystemError
(msg=None)¶ Bases:
fs.errors.FSError
Custom class for filesystem-related exceptions.
-
class
conpot.core.fs_utils.
SubAbstractFS
(parent_fs, path)¶ Bases:
fs.subfs.SubFS
,typing.Generic
Creates a chroot jail sub file system. Each protocol can have an instance of this class. Use AbstractFS’s create_jail method to access this. You won’t be able to cd into an up directory.
-
access
(path: str, name_or_id: Union[int, str] = None, required_perms: str = None)¶
-
check_access
(path=None, user=None, perms=None)¶
-
chmod
(path: str, mode: oct, recursive: bool = False) → None¶
-
chown
(fs_path: str, uid: int, gid: int, recursive: Optional[bool] = False)¶
-
default_gid
¶
-
default_group
¶
-
default_perms
¶
-
default_uid
¶
-
default_user
¶
-
format_list
(basedir, listing)¶
-
get_permissions
(path)¶
-
getcwd
()¶
-
getinfo
(path: str, get_actual: bool = False, namespaces=None)¶ Get information about a resource on a filesystem.
- Arguments:
path (str): A path to a resource on the filesystem. namespaces (list, optional): Info namespaces to query
(defaults to [basic]).- Returns:
- ~fs.info.Info: resource information object.
For more information regarding resource information, see info.
-
getmtime
(path)¶
-
move
(src_path, dst_path, overwrite=True)¶ Move a file from
src_path
todst_path
.- Arguments:
src_path (str): A path on the filesystem to move. dst_path (str): A path on the filesystem where the source
file will be written to.- overwrite (bool): If True, destination path will be
- overwritten if it exists.
- Raises:
- fs.errors.FileExpected: If
src_path
maps to a - directory instead of a file.
- fs.errors.DestinationExists: If
dst_path
exists, - and
overwrite
is False. - fs.errors.ResourceNotFound: If a parent directory of
dst_path
does not exist.
- fs.errors.FileExpected: If
-
readlink
(path)¶
-
remove
(path)¶ Remove a file from the filesystem.
- Arguments:
- path (str): Path of the file to remove.
- Raises:
- fs.errors.FileExpected: If the path is a directory. fs.errors.ResourceNotFound: If the path does not exist.
-
removedir
(path, rf=False)¶ Remove a directory from the filesystem.
- Arguments:
- path (str): Path of the directory to remove.
- Raises:
- fs.errors.DirectoryNotEmpty: If the directory is not empty (
- see ~fs.base.FS.removetree for a way to remove the directory contents.).
- fs.errors.DirectoryExpected: If the path does not refer to
- a directory.
- fs.errors.ResourceNotFound: If no resource exists at the
- given path.
- fs.errors.RemoveRootError: If an attempt is made to remove
- the root directory (i.e.
'/'
)
-
root
¶
-
stat
(path)¶
-
-
conpot.core.fs_utils.
copy_files
(source, dest, buffer_size=1048576)¶ Copy a file from source to dest. source and dest must be file-like objects.
conpot.core.internal_interface module¶
-
class
conpot.core.internal_interface.
DotDict
¶ Bases:
dict
-
class
conpot.core.internal_interface.
Network
¶ Bases:
object
conpot.core.session_manager module¶
conpot.core.virtual_fs module¶
-
class
conpot.core.virtual_fs.
VirtualFS
(data_fs_path=None)¶ Bases:
object
Conpot’s virtual file system. Based on Pyfilesystem2, it would allow us to have arbitrary file uploads while sand boxing them for later analysis. This is how it should look like:
Parameters: data_fs_path – Path for storing data_fs. A dictionary with attribute name _protocol_vfs stores all the fs folders made by all the individual protocols. :type data_fs_path: fs.open_fs
-
add_protocol
(protocol_name: str, data_fs_subdir: str, vfs_dst_path: str, src_path=None, owner_uid=0, group_gid=0, perms=493) -> (<class 'conpot.core.fs_utils.SubAbstractFS'>, <class 'fs.subfs.SubFS'>)¶ Method that would be used by protocols to initialize vfs. May be called by each protocol individually. This creates a chroot jail sub file system env which makes easier handling. It also creates a data_fs sub file system for managing protocol specific uploads. :param protocol_name: name of the protocol for which VFS is being created. :param data_fs_subdir: sub-folder name within data_fs that would be storing the uploads for later analysis :param vfs_dst_path: protocol specific sub-folder path in the fs. :param src_path: Source from where the files are to copied. :param owner_uid: UID of a registered user. This is the default owner in the sub file system :param group_gid: GID of a existing group. :param perms: Default permissions of the sub file system. :return: fs object
Note: The owner_uid and group_gid must be already registered with the fs. Otherwise an exception would be raised.
-
close
(force=False)¶ Close the filesystem properly. Better and more graceful than __del__ :param force: Force close. This would close the AbstractFS instance - without close closing data_fs File Systems
-
initialize_vfs
(fs_path=None, data_fs_path=None, temp_dir=None)¶
-
Module contents¶
-
conpot.core.
add_protocol
(protocol_name: str, data_fs_subdir: str, vfs_dst_path: str, src_path=None, owner_uid: Optional[int] = 0, group_gid: Optional[int] = 0, perms: Optional[oct] = 493) → Tuple¶
-
conpot.core.
close_fs
()¶ Close the file system. Remove all the temp files.
-
conpot.core.
get_databus
()¶
-
conpot.core.
get_interface
()¶
-
conpot.core.
get_session
(*args, **kwargs)¶
-
conpot.core.
get_sessionManager
()¶
-
conpot.core.
get_vfs
(protocol_name: Optional[str] = None) → Union[conpot.core.filesystem.AbstractFS, Tuple]¶ Get the File System. :param protocol_name: Name of the protocol to be fetched
-
conpot.core.
initialize_vfs
(fs_path=None, data_fs_path=None, temp_dir=None)¶